Numerous studies have documented declines in the abundance of reef-building corals over the last several decades and in some but not all cases, phase shifts to dominance by macroalgae have occurred. These assessments, however, often ignore the remainder of the benthos and thus provide limited information on the present-day structure and function of coral reef communities. Here, using an unprecedentedly large dataset collected within the last 10 years across 56 islands spanning five archipelagos in the central Pacific, we examine how benthic reef communities differ in the presence and absence of human populations. Using islands as replicates, we examine whether benthic community structure is associated with human habitation within and among archipelagos and across latitude. While there was no evidence for coral to macroalgal phase shifts across our dataset we did find that the majority of reefs on inhabited islands were dominated by fleshy non-reef-building organisms (turf algae, fleshy macroalgae and non-calcifying invertebrates). By contrast, benthic communities from uninhabited islands were more variable but in general supported more calcifiers and active reef builders (stony corals and crustose coralline algae). Our results suggest that cumulative human impacts across the central Pacific may be causing a reduction in the abundance of reef builders resulting in island scale phase shifts to dominance by fleshy organisms.