Ticket prices are one factor, but also long lines. The latter is the bigger issue for us*. Some of that can be blamed on childless, too, though from what I recall kids dominated the lines at the rides.
* -We're not so rich that it doesn't hurt, but you can budget for the once-in-a-childhood trip. The lines make it so that it either (a) isn't as fun as it should be, or (b) an exercise in planning how to optimize lines and fast passes.
Disney World isn’t “full of families” anymore because most of the people walking around don’t have any. We're talking about only 37% of visitors being households with kids. Childless adults are the single biggest slice and the highest spenders. Ticket prices turned the place into a luxury good right as the birth rate collapsed, so the DINKs and the perpetual adolescents fill the gap. Contemporary culture is increasingly organized around adult consumption rather than marriage and family formation.