Confusing yourself.

All of Sipsey Wilderness is Bankhead NF operated federal property. Sipsey is part of Bankhead National Forest. One odd feature of our National Forests is that a huge portion of their lands are scattered with private land, and in many cases, small lots of NF land are actually not connected physically with other NF properties. A lot of these little outlyers get sold or traded, to consolidate the forest or generate revenue, and a lot of property gets stuck inside the forest as well. My understanding is that nobody actually LIVES on any of the property within the Sipsey Wilderness boundary, but that the land that is not owned by the federal gov't was for some reason not ceded to the gov't. GSMNP had similar issues, with stragglers hanging on for a while in Cades Cove and other areas, until full buyouts were accomplished. There are houses in the corner not too far from Thompson Trailhead, but they aren't within the Wilderness Boundary.
The gray areas on the Cart-o-craft map are private property within Bankhead National Forest. There are a few areas of non-Forest Service land along the edges of Sipsey (the area near the Thompson Creek Trailhead), and there are small parcels (at least two) within the actual boundaries, including one house with a windmill, which is fairly deep into the Wilderness.