Diets higher in plants are associated with lower risks of chronic diseases. However, animal foods, which are rich in protein, are also rich in some important minerals and vitamins. Using data from a representative survey in France (INCA3, n = 1,125), we used path analyses as a mediation-like approach to decipher the importance of plant and animal proteins in the relationship between the plant-based diet index (PDI) and diet quality. We used three types of diet quality scores, namely, nutrient security, positive nutrient adequacy, and long-term mortality risk of four diet-related diseases (i.e., coronary heart disease, stroke, type 2 diabetes, and colorectal cancer). We identified positive and negative mediations, i.e., changes in plant/animal protein intake that are associated with changes in PDI, and favor or limit the association with the diet quality score. The PDI was positively associated with the risk of long-term mortality but not significantly with nutrient adequacy or nutrient security. A positive mediation by plant protein was found for all diet quality scores (specific indirect effects (SIEs) ranging from 0.04 to 0.10 SD). Conversely, the association between PDI and nutrient adequacy (but not nutrient security) was negatively mediated by animal protein intake (SIE: −0.06 SD). In further detailed models, the association between PDI and diet quality was mainly positively mediated by protein foods from the fruit-vegetables-legumes group (0.01 SD for the nutrient security and 0.02 SD for the nutrient adequacy) and whole grains (0.02 SD for the nutrient adequacy). Our data suggest that the positive impact of plant-based diets on diet quality is largely driven by higher intakes of plant protein foods, especially from fruits-vegetables-legumes and whole grains. Conversely, lower animal protein intake tends to limit the positive impact of plant-based diets on overall positive nutrient adequacy but not security. Protein sources appear critical to healthy plant-based diets.