Both time and number can be represented in spatial terms. While their representation in terms of spatial magnitude (distance or size) might be innate, their representation in terms of spatial position (left/right or up/down) is acquired. In Western culture, the mental timeline represents past/future events or short/long duration on the left/right sides of space, respectively. We conducted two developmental studies to pinpoint the age at which the mental timeline for duration begins to be acquired. Children (aged 5–6, 8, or 10 years old) and adults performed temporal bisection tasks in which relative spatial position (left/right) was manipulated by either arrow direction (Experiment 1) and/or lateralized stimulus location (Experiments 1 and 2). Results first confirmed previous findings that the symbolic representation of spatial position conveyed by arrow stimuli influences the perception of duration in older children. Both 8 and 10 year olds judged the duration of leftward arrows to be shorter than that of rightward arrows. We also showed for the first time that as long as position is manipulated in a non-symbolic way by the visual eccentricity of the stimuli, then even 5–6 year olds' perception of duration is influenced by spatial position. These children judged the duration of left-lateralized stimuli to be shorter than that of either right-lateralized or centrally located stimuli. These data are consistent with the use of a mental timeline for stimulus duration from the age of 5 years old, with short duration being represented on the left side of space and long duration on the right. Nevertheless, the way in which left and right were manipulated determined the age at which spatial position influenced duration judgment: physical spatial location influenced duration perception from the age of 5 years old whereas arrow direction influenced it from the age of 8. This age-related dissociation may reflect distinct developmental trajectories of automatic versus voluntary spatial attentional mechanisms and, more generally highlights the importance of accounting for attentional ability when interpreting results of duration judgment tasks.