What is the CO2 Footprint of ChatGPT and how does it compare to the emissions of a classic search engine like google.

1. qaware.de
CO2 Footprint of ChatGPT
Andreas Weber
andreas.weber@qaware.de

2. 2
QAware

3. 3
QAware

4. CO2 Footprint of ChatGPT
■ GPT-3 [1]
– 175 billion (10^9) parameters
– Training:
• 10 000 * V100 GPU (TDP 300W)
• 14.8 days training
• 1287 MWh
• 550t CO2e (0,429 kg CO2e/kWh)
– Inference [2] (Data for BLOOM, 176 B params)
• 16 * A100 GPU (TDP 400W)
• 18 days, 230,768 requests
• 914 kWh total, 3,96Wh/Request
• 1,56g CO2e / Request (0,394 kg CO2e/kWh)
• Note: Idle times!
4
QAware
[1] https://doi.org/10.48550/arXiv.2104.10350
[2] https://doi.org/10.48550/arXiv.2211.02001
[3] https://medium.com/@chrispointon/the-carbon-footprint-of-chatgpt-e1bc14e4cc2a
– Inference (Estimations from [3])
• 1 * A100 GPU (407W)
• 1 Request = 10,5s (0,35s / word * 30 words)
• 0,38g CO2e/ Request (0,322 kg CO2e/kWh)
• 13 million users per day with 5 requests each
• 24,8t CO2e / day
• Note: Is a single A100 GPU (80GB RAM)
capable of 175 billion parameters?
❭ 4 bit quantization -> 81,5 GB

5. CO2 Footprint of ChatGPT
5
QAware
[4] https://archive.ph/2RQ8X
■ GPT-4 [4]
– 1.8 trillion (10^12) parameters
– Training
• 25 000 * A100 GPUs (TDP 400W)
• 100 initial days training (after tests) (cost: ~$63 million)
• 24 000 MWh
• 7 728t CO2e (0,322 kg CO2e/kWh)
– Inference
• 8-16 * A100 GPU
• ~10s-15s / request -> 8,89 Wh/ request
• 2,86g CO2e / Request (0,322 kg CO2e/kWh)

6. CO2 Footprint of a Google Search
■ Google Search incl. indexing effort
– 0.3 Wh / Search [5.1]
– 0.2g CO2e / Search [3] (-> 0,67 kg CO2e/kWh) [5.1]
– 0.1g CO2e / Search (0,322 kg CO2e/kWh)
6
QAware
[5.1] https://googleblog.blogspot.com/2009/01/powering-google-search.htm
[5.2] https://greenspector.com/en/search-engines/
■ Comparison of different search engines & use cases [5.2]
– methodology: measure energy consumption + network
input/output → estimate emissions via OneByte Model
– use cases: app vs. browser, weather, definition, local
search / interactive map

7. Footprint Comparison
7
QAware
[6] https://co2.myclimate.org/
[7] https://de.statista.com/statistik/daten/studie/167877/umfrage/co-emissionen-nach-laendern-je-einwohner/
[8] https://de.statista.com/statistik/daten/studie/399048/umfrage/entwicklung-der-co2-emissionen-von-neuwagen-deutschland/
Training
■ GPT-3: ~ 100 Flights MUC <-> Sydney
■ GPT-4: ~1000 Germans living 1 year
Usage
■ GPT-3: ~ 4 Google Searches
■ GPT-4: ~ 29 Google Searches

8. What’s missing in the analysis
■ Non-GPU emissions like networking, RAM and SSDs
■ Power Usage Effectiveness (PUE) of datacenter
■ The embodied carbon of a datacenter
■ Emissions from network transfer
■ Emissions from end-user equipment
8
QAware

9. We need more transparency!
Green Software Development Manifesto: https://www.greenmanifesto.de/en/
9
QAware

10. Helpful Transparency Tools
■ Infrastructure at Cloud-Provider-Level
– Emission Dashboards from AWS, Azure, Google
■ Cluster-Level Energy consumption
– Kepler: https://github.com/sustainable-computing-io/kepler
■ Estimate Emissions for ML Usecases
– ML CO2 Impact: https://mlco2.github.io/impact/
• based on GPU hardware, usage hours, infrastructure provider, location
■ Estimate CO2 Emissions for Network Transfer, Device Usage, Embodied Carbon
– CO2.js: https://developers.thegreenwebfoundation.org/co2js/overview/
■ Emission Factors
– Climatiq data explorer & api: https://www.climatiq.io/data
• emission factors for various sectors
– Carbon aware sdk: https://github.com/Green-Software-Foundation/carbon-aware-sdk
• grid intensity data & forecasts
10
QAware