26th Annual CESM Workshop - Machine Learning: CESM-Related Efforts In this study, we use an explainable artificial intelligence met hod to identify climate signals that are found in a new set of single-forcing large ensembles from CESM1. To compare patterns between simulations, we adopt an artificial neural network (ANN) that predicts the year from input maps of near-surface temperature. We find that the North Atlantic Ocean is an important region for the ANN to make its prediction, especially for the simulation forced without time-evolving industrial aerosols.

1. CLIMATE SIGNALS IN CESM1
SINGLE-FORCING LARGE ENSEMBLES
REVEALED BY EXPLAINABLE NEURAL NETWORKS
@ZLabe
Zachary M. Labe
with Dr. Elizabeth A. Barnes
Department of Atmospheric Science
17 June 2021
Machine Learning: CESM-Related Efforts Cross Working Group
26th Annual CESM Workshop

2. THE REAL WORLD
(Observations)
What is the annual mean temperature of Earth?

3. What is the annual mean temperature of Earth?
THE REAL WORLD
(Observations)
Anomaly is relative to 1951-1980

4. What is the annual mean temperature of Earth?
THE REAL WORLD
(Observations)
CESM1 LARGE
ENSEMBLE

5. What is the annual mean temperature of Earth?
THE REAL WORLD
(Observations)
Range of ensembles
= natural variability (noise)
Mean of ensembles
= forced response (climate change)
CESM1 LARGE
ENSEMBLE

6. What is the annual mean temperature of Earth?
• Increasing greenhouse gases (CO2, CH4, N2O)
• Changes in industrial aerosols (SO4, BC, OC)
• Changes in biomass burning (aerosols)
• Changes in land-use & land-cover (albedo)

7. What is the annual mean temperature of Earth?
• Increasing greenhouse gases (CO2, CH4, N2O)
• Changes in industrial aerosols (SO4, BC, OC)
• Changes in biomass burning (aerosols)
• Changes in land-use & land-cover (albedo)
Plus everything else…
(Natural/internal variability)

8. What is the annual mean temperature of Earth?

9. Greenhouse gases fixed to 1920 levels
All forcings (CESM-LE)
Industrial aerosols fixed to 1920 levels
[Deser et al. 2020, JCLI]
Fully-coupled CESM1.1
20 Ensemble Members
Run from 1920-2080
Observations

10. So what?
Greenhouse gases = warming
Aerosols = ?? (though mostly cooling)
What are the relative responses
between greenhouse gas
and aerosol forcing?

11. Surface Temperature Map
ARTIFICIAL NEURAL NETWORK (ANN)

12. INPUT LAYER
Surface Temperature Map
ARTIFICIAL NEURAL NETWORK (ANN)

13. INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
Surface Temperature Map
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”
ARTIFICIAL NEURAL NETWORK (ANN)

14. INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
Surface Temperature Map
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”
BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI
ARTIFICIAL NEURAL NETWORK (ANN)

15. INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
Layer-wise Relevance Propagation
Surface Temperature Map
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”
BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI
ARTIFICIAL NEURAL NETWORK (ANN)
[Barnes et al. 2020, JAMES]
[Labe and Barnes 2021, JAMES]

16. LAYER-WISE RELEVANCE PROPAGATION (LRP)
Volcano
Great White
Shark
Timber
Wolf
Image Classification LRP
https://heatmapping.org/ [Geoscience examples in Toms et al. 2020, JAMES]
LRP heatmaps show regions
of “relevance” that
contribute to the neural
network’s decision-making
process for a sample
belonging to a particular
output category
Neural Network
WHY
WHY
WHY
Backpropagation – LRP

17. LAYER-WISE RELEVANCE PROPAGATION (LRP)
Volcano
Great White
Shark
Timber
Wolf
Image Classification LRP
https://heatmapping.org/ [Geoscience examples in Toms et al. 2020, JAMES]
LRP heatmaps show regions
of “relevance” that
contribute to the neural
network’s decision-making
process for a sample
belonging to a particular
output category
Neural Network
WHY
WHY
WHY
Backpropagation – LRP

18. LAYER-WISE RELEVANCE PROPAGATION (LRP)
Volcano
Great White
Shark
Timber
Wolf
Image Classification LRP
https://heatmapping.org/ [Geoscience examples in Toms et al. 2020, JAMES]
LRP heatmaps show regions
of “relevance” that
contribute to the neural
network’s decision-making
process for a sample
belonging to a particular
output category
Neural Network
Backpropagation – LRP
WHY
WHY
WHY

19. LAYER-WISE RELEVANCE PROPAGATION (LRP)
Image Classification LRP
https://heatmapping.org/ [Geoscience examples in Toms et al. 2020, JAMES]
NOT PERFECT
Crock
Pot
Neural Network
Backpropagation – LRP
WHY

20. OUTPUT LAYER
Layer-wise Relevance Propagation
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”
BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI
WHY?
= LRP HEAT MAPS
[Labe and Barnes 2021, JAMES]

21. Layer-wise Relevance Propagation
BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI
WHY?
= LRP HEAT MAPS
Machine Learning
Black Box
[Labe and Barnes 2021, JAMES]

22. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS
Greenhouse gases fixed
to 1920 levels
[AEROSOLS PREVAIL]
Industrial aerosols fixed
to 1920 levels
[GREENHOUSE GASES PREVAIL]
All forcings
[STANDARD CESM-LE]
DATA

23. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS
Greenhouse gases fixed
to 1920 levels
[AEROSOLS PREVAIL]
Industrial aerosols fixed
to 1920 levels
[GREENHOUSE GASES PREVAIL]
All forcings
[STANDARD CESM-LE]
DATA

24. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS
Greenhouse gases fixed
to 1920 levels
[AEROSOLS PREVAIL]
Industrial aerosols fixed
to 1920 levels
[GREENHOUSE GASES PREVAIL]
All forcings
[STANDARD CESM-LE]
DATA

25. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS
Greenhouse gases fixed
to 1920 levels
[AEROSOLS PREVAIL]
Industrial aerosols fixed
to 1920 levels
[GREENHOUSE GASES PREVAIL]
All forcings
[STANDARD CESM-LE]
DATA

26. CLIMATE MODEL DATA PREDICT THE YEAR FROM MAPS OF TEMPERATURE
AEROSOLS
PREVAIL
GREENHOUSE GASES
PREVAIL
STANDARD
CLIMATE MODEL
[Labe and Barnes 2021, JAMES]

27. OBSERVATIONS PREDICT THE YEAR FROM MAPS OF TEMPERATURE
AEROSOLS
PREVAIL
GREENHOUSE GASES
PREVAIL
STANDARD
CLIMATE MODEL
[Labe and Barnes 2021, JAMES]

28. OBSERVATIONS
SLOPES
PREDICT THE YEAR FROM MAPS OF TEMPERATURE
AEROSOLS
PREVAIL
GREENHOUSE GASES
PREVAIL
STANDARD
CLIMATE MODEL
[Labe and Barnes 2021, JAMES]

29. [Labe
and
Barnes
2021,
JAMES]
ARE THE RESULTS ROBUST?
YES!
COMBINATIONS OF TRAINING/TESTING DATA

30. HOW DID THE ANN
MAKE ITS
PREDICTIONS?

31. HOW DID THE ANN
MAKE ITS
PREDICTIONS?
WHY IS THERE
GREATER SKILL
FOR GHG+?

32. [Labe and Barnes 2021, JAMES]

33. Higher LRP values indicate greater relevance
for the ANN’s prediction
AVERAGED OVER 1960-2039
Aerosol-driven
Greenhouse gas-driven
All forcings
Low High
[Labe and Barnes 2021, JAMES]

34. Greenhouse gas-driven
Aerosol-driven
All forcings
AVERAGED OVER 1960-2039
[Labe and Barnes 2021, JAMES]

35. DISTRIBUTIONS OF LRP
AVERAGED OVER 1960-2039
[Labe and Barnes 2021, JAMES]

36. DISTRIBUTIONS OF LRP
AVERAGED OVER 1960-2039
[Labe and Barnes 2021, JAMES]

37. KEY POINTS
Zachary Labe
zmlabe@rams.colostate.edu
@ZLabe
1. Using explainable AI methods with artificial neural networks (ANN) reveals climate patterns in
large ensemble simulations
2. A metric is proposed for quantifying the uncertainty of an ANN visualization method that
extracts signals from different external forcings
3. Predictions from an ANN trained using a large ensemble without time-evolving aerosols show
the highest correlation with actual observations

38. QUESTIONS
Zachary Labe
1. Using explainable AI methods with artificial neural networks (ANN) reveals climate patterns in
large ensemble simulations
2. A metric is proposed for quantifying the uncertainty of an ANN visualization method that
extracts signals from different external forcings
3. Predictions from an ANN trained using a large ensemble without time-evolving aerosols show
the highest correlation with actual observations
Labe, Z.M. and E.A. Barnes (2021), Detecting climate signals using
explainable AI with single-forcing large ensembles. Journal of
Advances in Modeling Earth Systems, DOI:10.1029/2021MS002464