Obesity and metabolic-related diseases among which diabetes are prominent public health challenges of the 21 st century. It is now well acknowledged that pollutants are part of the equation, especially endocrine disrupting chemicals (EDCs) which interfere with any hormonal aspect. The aim of the review was to focus on adipose tissue, a central regulator of energy balance and metabolic homeostasis and to highlight the significant differences in the endocrine and metabolic aspects of adipose tissue between males and females which likely underlie the differences of the response to exposure to EDCs between the sexes. It was also to present an overview of several mechanisms of action by which pollutants could cause adipose tissue dysfunction. Indeed, a better understanding of the mechanism by which environmental chemicals target adipose tissue and cause metabolic disturbances, how these mechanisms interact and sex specificities is essential for developing mitigating and sex-specific strategies against metabolic diseases of chemical origin. In particular, considering that a scenario without pollutant exposure is not a realistic option in our current societies, attenuating the deleterious effects of exposure to pollutants by acting on the gut-adipose tissue axis may constitute a new direction of research.