The use of the perennial grain intermediate wheatgrass (Thinopyrum intermedium (Host) Barkworth & D.R. Dewey) may have the potential to sustain soil health and fertility through the development of an extensive root system. However, references are scarce to demonstrate its potential influence in a context of a limited perennial grain growth phase, integrated into annual grain crops succession. This study aims at determining how early a perennial crop rooting system differs from that of an annual crop through root development and root traits and microbial indicators. Our results indicate that the two-year-old intermediate wheatgrass promotes a denser and deeper rooting system with proportionally more root biomass and length deeper in the soil profile. From the first growing season, the perennial grain demonstrated a suite of root traits typical of a more resource-conservative strategy, and more belowground-oriented resource allocation. Soil fungal biomass indicators were enhanced. Arbuscular mycorrhizal fungi (AMF) indicators were notably found to be improved at 1 m depth during the second growing season. This study provides evidence that grain-based agriculture can benefit from the potential of deeper and long-lived root systems of intermediate wheatgrass to manage soils. The periodic use of a short-term perennial phase in the crop rotation has the potential to improve soil functioning in the long term.