The tangential force required to observe slip across a whole frictional interface can increase over time under a constant load, due to any combination of creep, chemical, or structural changes of the interface. In macroscopic rate-andstate models, these frictional aging processes are lumped into an ad hoc state variable. Here we explain, for a frictional system exclusively undergoing structural aging, how the macroscopic friction response emerges from the interplay between the surface roughness and the molecular motion within adsorbed monolayers. The existence of contact junctions and their friction dynamics are studied through coupled experimental and computational approaches. The former provides detailed measurements of how the friction force decays, after the stiction peak, to a steady-state value over a few nanometers of sliding distance, while the latter demonstrates how this memory distance is related to the evolution of the number of cross-surface attractive physical links, within contact junctions, between the molecules adsorbed on the rough surfaces. We also show that roughness is a sufficient condition for the appearance of structural aging. Using a unified model for friction between rough adsorbed monolayers, we show how contact junctions are a key component in structural aging and how the infrajunction molecular motion can control the macroscopic response.