In the present study, we investigated brain morphological signatures of dyslexia by using a voxel-based asymmetry analysis. Dyslexia is a developmental disorder that affects the acquisition of reading and spelling abilities and is associated with a phonological deficit. Speech perception disabilities have been associated with this deficit, particularly when listening conditions are challenging, such as in noisy environments. These deficits are associated with known neurophysiological correlates, such as a reduction in the functional activation or a modification of functional asymmetry in the cortical regions involved in speech processing, such as the bilateral superior temporal areas. These functional deficits have been associated with macroscopic morphological abnormalities, which potentially include a reduction in gray and white matter volumes, combined with modifications of the leftward asymmetry along the perisylvian areas. The purpose of this study was to investigate gray/white matter distribution asymmetries in dyslexic adults using automated image processing derived from the voxel-based morphometry technique. Correlations with speech-in-noise perception abilities were also investigated. The results confirmed the presence of gray matter distribution abnormalities in the superior temporal gyrus (STG) and the superior temporal Sulcus (STS) in individuals with dyslexia. Specifically, the gray matter of adults with dyslexia was symmetrically distributed over one particular region of the STS, the temporal voice area, whereas normal readers showed a clear rightward gray matter asymmetry in this area. We also identified a region in the left posterior STG in which the white matter distribution asymmetry was correlated to speech-in-noise comprehension abilities in dyslexic adults. These results provide further information concerning the morphological alterations observed in dyslexia, revealing the presence of both gray and white matter distribution anomalies and the potential involvement of these defects in speech-in-noise deficits.