Using explainable machine learning for evaluating patterns of climate change

USING EXPLAINABLE MACHINE
LEARNING FOR EVALUATING PATTERNS
OF CLIMATE CHANGE
https://zacklabe.com/ @ZLabe
Zachary M. Labe
NOAA GFDL and Princeton University; Atmospheric and Oceanic Science
with Dr. Elizabeth A. Barnes at Colorado State University
22 February 2023
Washington State University Vancouver
Natural Sciences Group

Machine Learning
is not new!
But…

https://doi.org/10.1175/1520-0450(1964)003%3C0513:AADPSF%3E2.0.CO;2

Artificial Intelligence
Machine Learning
Deep Learning
Computer/Data Science

Computer/Data Science
Machine Learning
Deep Learning
Supervised
Learning
Unsupervised
Learning
Labeled data
Classification
Regression
Unlabeled data
Clustering
Dimension reduction

Do it better
e.g., parameterizations in climate models are not
perfect, use ML to make them more accurate
Do it faster
e.g., code in climate models is very slow (but we
know the right answer) - use ML methods to speed
things up
Do something new
• e.g., go looking for non-linear relationships you
didn’t know were there
Very relevant for
research: may be
slower and worse,
but can still learn
something
WHY SHOULD WE CONSIDER
MACHINE LEARNING?

Adapted from: Kotamarthi, R., Hayhoe, K., Mearns, L., Wuebbles, D., Jacobs, J., & Jurado, J.
(2021). Global Climate Models. In Downscaling Techniques for High-Resolution Climate
Projections: From Global Change to Local Impacts (pp. 19-39). Cambridge: Cambridge University
Press. doi:10.1017/9781108601269.003
CLIMATE MODELS
Horizontal Grid
Vertical Levels
Past/Present/Future
Fully-Coupled System
20-40 Petabytes of data

ADAPTED FROM EYRING ET AL. 2016
CMIP6

Python tools for machine learning

Programming Languages Python – Machine Learning Libraries
https://insights.stackoverflow.com/trends

Machine learning for weather
IDENTIFYING SEVERE THUNDERSTORMS
Molina et al. 2021
Toms et al. 2021
CLASSIFYING PHASE OF MADDEN-JULLIAN OSCILLATION
SATELLITE DETECTION
Lee et al. 2021
DETECTING TORNADOES
McGovern et al. 2019

Machine learning for oceanography
CLASSIFYING ARCTIC OCEAN ACIDIFICATION
Krasting et al. 2022
TRACK AND REVEAL DEEP WATER MASSES
Sonnewald and Lguensat, 2021
ESTIMATING OCEAN SURFACE CURRENTS
Sinha and Abernathey, 2021

Machine learning for climate
FINDING FORECASTS OF OPPORTUNITY
Mayer and Barnes, 2021
PREDICTING CLIMATE MODES OF VARIABILITY
Gordon et al. 2021
TIMING OF CLIMATE CHANGE
Barnes et al. 2019

INPUT
[DATA]
PREDICTION
Machine
Learning

INPUT
[DATA]
PREDICTION
~Statistical
Algorithm~

Machine Learning
Deep Learning

X1
X2
INPUTS
Artificial Neural Networks [ANN]

Linear regression!
X1
X2
W1
W2
∑ = X1W1+ X2W2 + b
INPUTS
NODE

X1
X2
W1
W2
∑
INPUTS
NODE
Linear regression with non-linear
mapping by an “activation function”
Training of the network is merely
determining the weights “w” and
bias/offset “b"
= factivation(X1W1+ X2W2 + b)

X1
X2
W1
W2
∑
INPUTS
NODE
= factivation(X1W1+ X2W2 + b)
ReLU Sigmoid Linear

X1
X2
∑
inputs
HIDDEN LAYERS
X3
∑
∑
∑
OUTPUT
= predictions
: : ::
INPUTS

Complexity and nonlinearities of the ANN allow it to learn many
different pathways of predictable behavior
Once trained, you have an array of weights and biases which can be
used for prediction on new data
INPUT
[DATA]
PREDICTION

X
Y
Our data
Just an exercise in curve fitting… (overfitting!)

TEMPERATURE
We know some metadata…
+ What year is it?
+ Where did it come from?

+ What year is it?
[Labe and Barnes, 2022; ESS]
TEMPERATURE

TEMPERATURE
Neural network learns nonlinear
combinations of forced climate
patterns to identify the year
+ What year is it?

----ANN----
2 Hidden Layers
10 Nodes each
Ridge Regularization
Early Stopping
[e.g., Barnes et al. 2019, 2020]
[e.g., Labe and Barnes, 2021]
TIMING OF EMERGENCE
(COMBINED VARIABLES)
RESPONSES TO
EXTERNAL CLIMATE
FORCINGS
PATTERNS OF
CLIMATE INDICATORS
[e.g., Rader et al. 2022]
Surface Temperature Map Precipitation Map
+
TEMPERATURE
+ What year is it?

----ANN----
2 Hidden Layers
10 Nodes each
Ridge Regularization
Early Stopping
[e.g., Barnes et al. 2019, 2020]
[e.g., Labe and Barnes, 2021]
TIMING OF EMERGENCE
(COMBINED VARIABLES)
RESPONSES TO
EXTERNAL CLIMATE
FORCINGS
PATTERNS OF
CLIMATE INDICATORS
Surface Temperature Map Precipitation Map
+
TEMPERATURE
[e.g., Rader et al. 2022]
+ What year is it?

THE REAL WORLD
(Observations)
What is the annual mean temperature of Earth?

THE REAL WORLD
(Observations)
Anomaly is relative to 1951-1980

THE REAL WORLD
(Observations)
Let’s run a
climate model

THE REAL WORLD
(Observations)
Let’s run a
climate model
again

THE REAL WORLD
(Observations)
Let’s run a
climate model
again & again

THE REAL WORLD
(Observations)
CLIMATE MODEL
ENSEMBLES

THE REAL WORLD
(Observations)
CLIMATE MODEL
ENSEMBLES
Range of ensembles
= internal variability (noise)
Mean of ensembles
= forced response (climate change)

Range of ensembles
Mean of ensembles
But let’s remove
climate change…

Range of ensembles
Mean of ensembles
After removing the
forced response…
anomalies/noise!

• Increasing greenhouse gases (CO2, CH4, N2O)
• Changes in industrial aerosols (SO4, BC, OC)
• Changes in biomass burning (aerosols)
• Changes in land-use & land-cover (albedo)

• Increasing greenhouse gases (CO2, CH4, N2O)
• Changes in industrial aerosols (SO4, BC, OC)
• Changes in biomass burning (aerosols)
• Changes in land-use & land-cover (albedo)
Plus everything else…
(Natural/internal variability)

Greenhouse gases fixed to 1920 levels
All forcings (CESM-LE)
Industrial aerosols fixed to 1920 levels
[Deser et al. 2020, JCLI]
Fully-coupled CESM1.1
20 Ensemble Members
Run from 1920-2080
Observations

So what?
Greenhouse gases = warming
Aerosols = ?? (though mostly cooling)
What are the relative responses
between greenhouse gas
and aerosol forcing?

Surface Temperature Map
ARTIFICIAL NEURAL NETWORK (ANN)
Input data from one of the
three single forcing large
ensemble simulations
(AER+, GHG+, ALL)

INPUT LAYER

INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”

INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”
BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI

INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
Layer-wise Relevance Propagation
“2000-2009”
DECADE CLASS
“2070-2079”
“1920-1929”
[Barnes et al. 2020, JAMES]
[Labe and Barnes 2021, JAMES]

LAYER-WISE RELEVANCE PROPAGATION (LRP)
Volcano
Great White
Shark
Timber
Wolf
Image Classification LRP
https://heatmapping.org/
LRP heatmaps show regions
of “relevance” that
contribute to the neural
network’s decision-making
process for a sample
belonging to a particular
output category
Neural Network
WHY
WHY
WHY
Backpropagation – LRP

Volcano
Great White
Shark
Timber
Wolf
LRP heatmaps show regions
of “relevance” that
contribute to the neural
network’s decision-making
process for a sample
belonging to a particular
output category
Neural Network
WHY
WHY
WHY

NOT PERFECT
Crock
Pot
Neural Network
WHY

[Adapted from Adebayo et al., 2020]
EXPLAINABLE AI (XAI) IS
NOT PERFECT
THERE ARE MANY
METHODS
A bird!
XAI

[Adapted from Adebayo et al., 2020]
THERE ARE MANY
METHODS
EXPLAINABLE AI (XAI) IS
NOT PERFECT

Visualizing something we already know…
ENSO

Neural
Network
[0] La Niña [1] El Niño
[Toms et al. 2020, JAMES]
Input a map of sea surface temperatures

Visualizing something we already know…
Input maps of sea surface
temperatures (SST) to
identify El Niño or La Niña
Use ‘LRP’ to see how the
neural network is making
its decision
[Toms et al. 2020, JAMES]
Layer-wise Relevance Propagation
Composite SST Observations
LRP [Relevance]
SST Anomaly [°C]
0.00 0.75
0.0 1.5
-1.5

1960-1999: ANNUAL MEAN TEMPERATURE TRENDS
Greenhouse gases fixed
to 1920 levels
[AEROSOLS PREVAIL]
Industrial aerosols fixed
to 1920 levels
[GREENHOUSE GASES PREVAIL]
All forcings
[STANDARD CESM-LE]
DATA
Warming
Cooling

CLIMATE MODEL DATA PREDICT THE YEAR FROM MAPS OF TEMPERATURE
AEROSOLS
PREVAIL
GREENHOUSE GASES
PREVAIL
STANDARD
CLIMATE MODEL

OBSERVATIONS PREDICT THE YEAR FROM MAPS OF TEMPERATURE
AEROSOLS
PREVAIL
GREENHOUSE GASES
PREVAIL
STANDARD
CLIMATE MODEL

OBSERVATIONS
SLOPES
PREDICT THE YEAR FROM MAPS OF TEMPERATURE
AEROSOLS
PREVAIL
GREENHOUSE GASES
PREVAIL
STANDARD
CLIMATE MODEL

HOW DID THE ANN
MAKE ITS
PREDICTIONS?

HOW DID THE ANN
MAKE ITS
PREDICTIONS?
WHY IS THERE
GREATER SKILL
FOR GHG+?

RESULTS FROM LRP
Low High

RESULTS FROM LRP
Low High
+

Higher LRP values indicate greater relevance
for the ANN’s prediction
AVERAGED OVER 1960-2039
Aerosol-driven
Greenhouse gas-driven
All forcings
Low High

DISTRIBUTIONS OF LRP

DISTRIBUTIONS OF LRP
Distribution shifted right = more relevant region

KEY POINTS FROM EXAMPLE #1
1. Using explainable AI methods with artificial neural networks (ANN)
reveals climate patterns in large ensemble simulations
2. A metric is proposed for quantifying the uncertainty of an ANN
visualization method that extracts signals from different external
forcings
3. Predictions from an ANN trained using a large ensemble without
time-evolving aerosols show the highest correlation with actual
observations
Labe, Z.M. and E.A. Barnes (2021), Detecting climate signals using explainable AI with single-forcing
large ensembles. Journal of Advances in Modeling Earth Systems, DOI:10.1029/2021MS002464

NASA/GISS/GISTEMPv4
“Hiatus”

Global Warming
Hiatus?
…in research

Global Warming
Hiatus?
…in the media, etc.

Are slowdowns (“hiatus”) in decadal
warming predictable?
• Statistical construct?
• Lack of surface temperature observations in the Arctic?
• Phase transition of the Interdecadal Pacific Oscillation (IPO)?
• Influence of volcanoes and other aerosol forcing?
• Weaker solar forcing?
• Lower equilibrium climate sensitivity (ECS)?
• Other combinations of internal variability?
FUTURE
WARMING

Select one ensemble
member and calculate
the annual mean
global mean surface
temperature (GMST)
2-m TEMPERATURE
ANOMALY
[Labe and Barnes, 2022; GRL]

Calculate 10-year
moving (linear) trends
2-m TEMPERATURE
ANOMALY

Plot the slope of the
linear trends
START OF 10-YEAR
TEMPERATURE TREND
2-m TEMPERATURE
ANOMALY

Calculate a threshold
for defining a slowdown
in decadal warming

Repeat this exercise for
each ensemble
member in CESM2-LE

Compare warming
slowdowns with
reanalysis (ERA5)

OCEAN HEAT CONTENT – 100 M
Start with anomalous ocean heat…

INPUT LAYER
Start with anomalous ocean heat…

INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
YES
SLOWDOWN
NO
SLOWDOWN
Will a slowdown begin?

INPUT LAYER
HIDDEN LAYERS
OUTPUT LAYER
YES
SLOWDOWN
NO
SLOWDOWN
LAYER-WISE RELEVANCE PROPAGATION
Will a slowdown begin?

So how well does the neural network do?

Low High Colder Warmer

What about observations?
Future (2012-)
so-called “hiatus”
Comparing
observations to
the IPO

Future (2012-)
so-called “hiatus”
2021
Looking ahead
to the near-
future…
?
2022
Early 2000s
slowdown

Colder Warmer
[2003, 2004] [2016, 2017]
LRP Relevance

KEY POINTS FROM EXAMPLE #2
1.Artificial neural network predicts the onset of slowdowns in
decadal warming trends of global mean temperature
2.Explainable AI reveals the neural network is leveraging
tropical patterns of ocean heat content anomalies
3.Transitions in the phase of the Interdecadal Pacific Oscillation
are frequently associated with warming slowdown trends in
CESM2-LE
Labe, Z.M. and E.A. Barnes (2022), Predicting slowdowns in decadal climate warming trends with
explainable neural networks. Geophysical Research Letters, DOI:10.1029/2022GL098173

INPUT
[DATA]
PREDICTION
Machine
Learning
Explainable AI
Learn new
climate science!

MACHINE LEARNING IS JUST
ANOTHER TOOL TO ADD TO OUR
WORKFLOW.
1)

MACHINE LEARNING IS
NO LONGER A BLACK BOX.
2)

WE CAN LEARN NEW SCIENCE
FROM EXPLAINABLE AI.
3)

KEY POINTS
1. Machine learning is just another tool to consider for our scientific workflow
2. We can use explainable AI (XAI) methods to peer into the black box of machine learning
3. We can learn new science by using XAI methods in conjunction with existing statistical tools
Zachary Labe
zachary.labe@noaa.gov
Labe, Z.M. and E.A. Barnes (2021), Detecting climate signals using explainable AI with single-forcing
large ensembles. Journal of Advances in Modeling Earth Systems, DOI:10.1029/2021MS002464
Labe, Z.M. and E.A. Barnes (2022), Predicting slowdowns in decadal climate warming trends with
explainable neural networks. Geophysical Research Letters, DOI:10.1029/2022GL098173

Using explainable machine learning for evaluating patterns of climate change

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Using explainable machine learning for evaluating patterns of climate change

Similar to Using explainable machine learning for evaluating patterns of climate change (20)

More from Zachary Labe

More from Zachary Labe (20)

Recently uploaded

Recently uploaded (20)

Using explainable machine learning for evaluating patterns of climate change