This article provides new evidence on various factors that affect residential energy consumption in France. We model the consumption of residential energy in dwellings and apply a bottom-up statistical approach. Our quantile regression model uses an innovative variable selection method via the adaptive elastic net regularization technique. The empirical estimates are based on responses to the PHEBUS survey. The aim is to untangle the effects of dwelling, socio-economic and behavior-related factors on the household energy consumption, for different levels of energy use. Our findings demonstrate that cross- sectional variation in residential energy consumption is a function of the building’s technical characteristics and the household’s socio-economic attributes and behavior. The analysis suggests that the effect of the household’s dwelling, demographic and socioeconomic attributes on its energy consumption differs across quantiles. We propose some measures and empirical methods that allow a deeper understanding of the factors affecting energy use which should be informative for policy making aimed at reducing residential energy demand.