The age-correlated gains and losses in visual identification under backward pattern masking were studied in a representative sample of 226 individuals ranging from 6 to 88 years of age. Participants identified masked symbols at leisure under high and low stimulus quality and at varying Stimulus Onset Asynchronies. Performance increased from childhood to early adulthood and then decreased, describing the common inverted U-shaped function. However, measures of general processing speed accounted for the gains in childhood and adolescence but not for losses in older age. This asymmetry between child development and aging is inconsistent with general-factor lifespan theories of cognitive development and suggests that specific mechanisms underlying visual identification during child development and aging are different.