Dichotic listening is the high-level auditory process which enables the perception of different verbal stimuli delivered simultaneously to the right and left ears (binaural integration), as well as the perception of a verbal stimulus presented to one ear while ignoring a different stimulus in the other ear (binaural separation). Deficits in central auditory processing have been reported in children with learning disabilities. The present study aimed to compare dichotic listening performances in right-handed impaired readers (IR) and non-impaired readers (non-IR) according to age. For this, a cross-sectional study was conducted in 120 IR (56 males and 64 females) divided into five age groups and 120 non-IR (63 male and 57 female) matched on chronological age (8 to 9 years; 9 to 10 years; 10 to 12 years; 12 to 18 years; adult). They were tested for binaural integration and binaural separation, allowing for the calculation of dichotic aptitude (DA), ear prevalence (EP), and attentional shift index (ASI). A series of ANOVAs showed an effect of age and of the reading group for all the dichotic-related measures, except for EP. Binaural separation scores were lower in IR who also showed more intrusive responses compared to non-IR. These intrusive responses, which were more frequent on the right ear for IR, decreased with age in both groups. Overall, these results suggest that dichotic listening scores improve with age as the central auditory pathways mature. However, whatever the age, performances are lower in IR than in non-IR. This might be explained by an incomplete maturation of the auditory pathways in IR; an early start for long-term follow-up and auditory training is suggested.