This study was aimed at assessing listening effort (LE) in quiet and in noisy daily life situations, in 481 adults with hearing aids (HAs) and 62 adults with normal hearing, using an Extended version of the Effort Assessment Scale (EEAS). Participants were invited to self-assess their LE in daily life, on a visual analog scale graded from 0 (no effort) to 10. The EEAS's internal structure identified two separate constructs pertaining to LE in quiet and LE in noise, each with good consistency (Cronbach's α > 0.83). A three-factor model explained 12% of the variance of the EEAS scores, with HA experience the most important one, and better ear hearing threshold (averaged across 0.5–4 kHz) and ear asymmetry as the other two factors. The EEAS subscales differed in behavior, with the LE in noise being the most dependent on HA experience, whereas LE in quiet depended more on better ear hearing threshold. In a subgroup of people with 6 months to less than 24 months HA experience, a significant decrease in LE in noise was observed with increasing HA experience (0.26 points decrease per year of HA experience), whereas in a group of people with at least 24 months of HA experience, a small increase in LE in noise was observed. This effect was not mediated by age, nor hearing threshold. The extended Effort Assessment Scale is therefore offering an assessment of both LE in quiet and LE in noise, with different dependence on HA experience and hearing thresholds.