We evaluated the relationship between binaural hearing deficits and quality of life. The study included 49 adults with asymmetric hearing loss (AHL), and 11 adult normal-hearing listeners (NHL) served as controls. Speech reception thresholds (SRT) were assessed with the French Matrix Test. Quality of life was evaluated with the Speech, Spatial and Qualities of Hearing Scale (SSQ) and the Glasgow Health Status Inventory. Speech recognition in noise was significantly poorer for AHL subjects [-0.12 dB signal-to-noise ratio (SNR) in dichotic (with speech presented to the poorer ear and noise to the better ear), -1.72 dB in diotic and -6.84 dB in reverse-dichotic conditions] compared to NHL (-4.98 dB in diotic and -9.58 dB in dichotic conditions). Scores for quality-of-life questionnaires were significantly below norms. Significant correlations were found between the SRT for the dichotic condition and the SSQ total score (r = -0.38, p = 0.01), and pure-tone average thresholds for both groups.