Industrialization has transformed the gut microbiota, reducing the prevalence of Prevotella relative to Bacteroides. Here, we isolate Bacteroides and Prevotella strains from the microbiota of Hadza huntergatherers in Tanzania, a population with high levels of Prevotella. We demonstrate that plant-derived microbiota-accessible carbohydrates (MACs) are required for persistence of Prevotella copri but not Bacteroides thetaiotaomicron in vivo. Differences in carbohydrate metabolism gene content, expression, and in vitro growth reveal that Hadza Prevotella strains specialize in degrading plant carbohydrates, while Hadza Bacteroides isolates use both plant and host-derived carbohydrates, a difference mirrored in Bacteroides from non-Hadza populations. When competing directly, P. copri requires plant-derived MACs to maintain colonization in the presence of B. thetaiotaomicron, as a no-MAC diet eliminates P. copri colonization. Prevotella’s reliance on plant-derived MACs and Bacteroides’ ability to use host mucus carbohydrates could explain the reduced prevalence of Prevotella in populations consuming a low-MAC, industrialized diet.