Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model - Archive ouverte HAL
Article Dans Une Revue Journal of Machine Learning Research Année : 2023

Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

Résumé

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes of CO 2 eq if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based operational consumption. We also carry out an empirical study to measure the energy requirements and carbon emissions of its deployment for inference via an API endpoint receiving user queries in real-time. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of ML models and future research directions that can contribute towards improving carbon emissions reporting.
Fichier principal
Vignette du fichier
23-0069.pdf (243.4 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04288059 , version 1 (15-11-2023)

Identifiants

  • HAL Id : hal-04288059 , version 1

Citer

Alexandra Sasha Luccioni, Sylvain Viguier, Anne-Laure Ligozat. Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model. Journal of Machine Learning Research, 2023, 24 (253), pp.1-15. ⟨hal-04288059⟩
186 Consultations
138 Téléchargements

Partager

More