Carbon, Water, and Compute
Training large language models consumes staggering amounts of energy. The exact figures are closely guarded by AI companies — competitive intelligence wrapped in PR sensitivity — but the numbers that have leaked or been published paint a concerning picture.
Training GPT-3 (175 billion parameters) was estimated by researchers at the University of Copenhagen to consume approximately 1,287 megawatt-hours of electricity — equivalent to the annual consumption of about 120 average US households. That training run emitted roughly 502 metric tons of CO2 equivalent if powered by a standard grid mix. GPT-4’s training run, with an estimated 1.76 trillion parameters (mixture-of-experts), was likely 5-10x more energy-intensive.
Meta’s LLaMA 3 405B training reportedly consumed approximately 39 million GPU-hours on H100-80GB GPUs. At roughly 700W per GPU (including data center overhead), that’s about 27.3 gigawatt-hours — enough to power about 2,500 US homes for a year. And that’s just one training run. Most models go through multiple training runs and extensive fine-tuning.
Water: The Overlooked Cost
Data centers don’t just consume electricity — they consume water for cooling. A 2023 study from UC Riverside estimated that training GPT-3 in Microsoft’s US data centers consumed approximately 700,000 liters of freshwater (185,000 gallons) directly for cooling. The researchers further estimated that a conversation with ChatGPT of 20-50 questions consumes about 500ml of water — roughly a standard water bottle — per interaction, when accounting for the data center cooling required to run inference at scale.
Microsoft’s water consumption increased 34% from 2021 to 2022, to nearly 1.7 billion gallons, a spike the company attributed partly to AI workloads. Google’s water consumption increased 20% over the same period, to 5.6 billion gallons. Data centers in drought-prone regions (Arizona, Chile, Spain, parts of India) face growing community opposition over water usage.
These numbers have significant uncertainty ranges — water consumption depends heavily on data center location, cooling technology, and time of year. A data center in Iowa in January uses far less water for cooling than the same center in Arizona in July. But the trend line is clear: as models get larger and AI usage grows, the environmental footprint grows with it.
Inference: The Bigger Long-Term Problem
Training gets the headlines, but inference — actually running the model to answer user queries — may represent the larger environmental cost over the full lifecycle of an AI model. GPT-4 serves hundreds of millions of queries per day. Each query consumes energy. If those queries continue growing at current rates, the cumulative inference energy cost of a popular model will exceed its training cost within months of deployment.
SemiAnalysis, an industry research firm, estimated in early 2024 that NVIDIA shipped approximately 1.5 million H100 GPUs in 2023, with a total power draw of about 1.05 gigawatts when fully utilized. A single H100 server node can draw 10.2 kW. At scale, the AI industry is building infrastructure with power requirements comparable to mid-sized cities. Microsoft and OpenAI’s planned “Stargate” supercomputer, reportedly targeting 2028, is speculated to require up to 5 gigawatts of power — roughly the output of five nuclear reactors.
Efficiency Improvements and Their Limits
The AI industry isn’t ignoring this problem — but the solutions are partial. Newer hardware is dramatically more efficient: NVIDIA’s H100 GPU delivers roughly 3x the AI performance per watt compared to the previous-generation A100. The upcoming B200 (“Blackwell”) promises another 2-4x improvement. Training algorithms are getting more efficient too — DeepSeek’s V3 model reportedly achieved GPT-4-class performance with roughly one-tenth the training compute budget through innovations in model architecture and training methodology.
But efficiency gains are being outpaced by scale. When training runs expand from 10^23 to 10^25 floating-point operations, a 3x hardware efficiency improvement doesn’t close the gap. This dynamic — known as Jevons paradox in energy economics — means that as AI gets more efficient, demand tends to increase faster than efficiency improves, driving up total resource consumption.
What Can Be Done
Several approaches could mitigate AI’s environmental impact. Transparency is the first step — mandatory reporting of energy consumption and carbon emissions for large training runs would allow researchers and policymakers to track the problem. Google, Microsoft, and Amazon have committed to carbon neutrality or negativity by 2030, but these commitments were made before the AI boom reshaped their energy footprints.
Data center siting matters enormously. A training run powered by Quebec’s hydroelectric grid has a fraction of the carbon footprint of the same run powered by Virginia’s natural-gas-heavy grid. Water consumption is lowest in cool, water-rich locations like Sweden, Finland, and Norway, where several hyperscale data centers are being built.
Smaller models with specialized training — rather than ever-larger general models — may be the most practical path. A fine-tuned 7B parameter model can match GPT-3.5 on specific tasks while using a tiny fraction of the energy. As the marginal utility of additional parameters diminishes, the environmental calculus may naturally steer the industry toward more efficient architectures.
The AI industry has an environmental problem. It’s not catastrophic yet, but it’s growing fast. Acknowledging it — with real data, not corporate sustainability reports — is the first step toward managing it.