← Go Back

The Environmental Cost of Generative AI

First Planted: 11 December 2023 Last Tended: 14 December 2023

Generative Tasks are power hungry

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of “generality” comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. [1]

Since GPT-3, we've been seeing a "paradigm shift away from smaller models finetuned for a specific task towards models that are meant to carry out a multitude of tasks at once, deployed to respond to a barrage of user queries in real time." [1]. One of the key findings from Luccioni et al's study was that:

Generative tasks are more energy- and carbon-intensive compared to discriminative tasks. [...] the most energy- and carbon-intensive tasks are those that generate new content: text generation, summarization, image captioning, and image generation. [1]

The study also found that "Using multi-purpose models for discriminative tasks is more energy-intensive compared to task-specific models for these same tasks.". This was especially the case for question answering, and "the difference between multi-purpose models and task-specific models is amplified as the length of output gets longer". [1]

Image generation in particular requires a lot of power

To give a tangible example:

[...] charging the average smartphone requires 0.012 kWh of energy 4 , which means that the most efficient text generation model uses as much energy as 16% of a full smartphone charge for 1,000 inferences, whereas the least efficient image generation model uses as much energy as 950 smartphone charges (11.49 kWh), or nearly 1 charge per image generation. [1]

Measuring the environmental impact of ML is not easy

Due in part to a lack of transparency, it's quite difficult to measure (and regulate) the environmental impact in this particular branch of the industry. Recent work has focused on "quantifying the operational energy and carbon required to perform the training phase [...] due to the relative ease of measuring" [1] the energy required. Examples include:

  • Energy and policy considerations for deep learning in NLP, 2019 [2]
  • Carbon emissions and large neural network training, 2021 [3]
  • Measuring the carbon intensity of AI in cloud instances, 2022 [4]
  • Counting carbon: A survey of factors influencing the emissions of machine learning, 2023 [5]

But what about other phases of the ML model life cycle? Phases that "stand to impact the environment just as much, or more, than training due to the computational resources required to deploy modern models at scale"? [1]

We need more transparency

Given our findings and the increased deployment of generative, multi-purpose AI models, we hope that both ML researchers and practitioners will practice transparency regarding the nature and impacts of their models, to enable better understanding of their environmental impacts. [1]

References:

[1] Power Hungry Processing: Watts Driving the Cost of AI Deployment? - A. Luccioni, Y. Jernite, E. Strubell, Allen Institute for AI. 2023.
[2] Energy and Policy Considerations for Deep Learning in NLP. - Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019.
[3] Carbon Emissions and Large Neural Network Training - David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. 2021.
[4] Measuring the Carbon Intensity of AI in Cloud Instances - Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A Smith, Nicole DeCario, and Will Buchanan. 2022.
[5] Counting carbon: A survey of factors influencing the emissions of machine learning - Alexandra Sasha Luccioni and Alex Hernandez-Garcia. 2023.