Download as PDF, PPTX
![INPUT
[DATA]
PREDICTION
Machine
Learning](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-24-320.jpg)
![CLIMATE MODEL
MAP
[DATA]
Machine
Learning
CLASSIFICATION](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-28-320.jpg)
![CLASSIFICATION
Machine
Learning
CLIMATE MODEL
MAP
[DATA]](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-29-320.jpg)
![CLASSIFICATION
Machine
Learning
CLIMATE MODEL
MAP
[DATA]
Explainable AI
Learn new
science!](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-30-320.jpg)
![EXPLAINABLE AI (XAI)
THERE ARE MANY
METHODS
A bird!
XAI
[Adapted from Adebayo et al., 2020]](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-33-320.jpg)
![THERE ARE MANY
METHODS
EXPLAINABLE AI (XAI)
[Adapted from Adebayo et al., 2020]](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-34-320.jpg)
![Neural
Network
[0] La Niña [1] El Niño
Input a map of sea surface temperatures
[Toms et al. 2020, JAMES]](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-36-320.jpg)
![Visualizing something we already know…
Input maps of sea surface
temperatures (SST) to
identify El Niño or La Niña
Use ‘LRP’ to see how the
neural network is making
its decision
[Toms et al. 2020, JAMES]
Layer-wise Relevance Propagation
Composite SST Observations
LRP [Relevance]
SST Anomaly [°C]
0.00 0.75
0.0 1.5
-1.5
Warmer
Colder
High
Low](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-37-320.jpg)
![APPLY SOFTMAX OPERATOR
IN THE OUTPUT LAYER
[ 0.71 ]
[ 0.05 ]
[ 0.01 ]
[ 0.01 ]
[ 0.03 ]
[ 0.11 ]
[ 0.08 ]
RANK
LABE AND BARNES 2022, ESS](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-41-320.jpg)
![APPLY SOFTMAX OPERATOR
IN THE OUTPUT LAYER
[ 0.71 ]
[ 0.05 ]
[ 0.01 ]
[ 0.01 ]
[ 0.03 ]
[ 0.11 ]
[ 0.08 ]
RANK
[ 1 ]
[ 4 ]
[ 7 ]
[ 6 ]
[ 5 ]
[ 2 ]
[ 3 ]
LABE AND BARNES 2022, ESS](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-42-320.jpg)
![APPLY SOFTMAX OPERATOR
IN THE OUTPUT LAYER
[ 0.71 ]
[ 0.05 ]
[ 0.01 ]
[ 0.01 ]
[ 0.03 ]
[ 0.11 ]
[ 0.08 ]
RANK
[ 1 ]
[ 4 ]
[ 7 ]
[ 6 ]
[ 5 ]
[ 2 ]
[ 3 ]
Confidence/Probability
LABE AND BARNES 2022, ESS](https://image.slidesharecdn.com/zlabeiarpcpresentation-240314161148-9ee71281/85/Explainable-AI-approach-for-evaluating-climate-models-in-the-Arctic-43-320.jpg)
Uploaded byZachary Labe
Explainable AI approach for evaluating climate models in the Arctic
The document discusses an explainable AI approach for evaluating climate models in the Arctic, emphasizing the use of machine learning to enhance model accuracy and speed. It details how artificial neural networks can be employed to classify climate model outputs and compares them to actual observations, providing insights into temperature patterns. The methodology includes layer-wise relevance propagation to understand the decision-making processes of the neural networks implemented.
More Related Content
Similar to Explainable AI approach for evaluating climate models in the Arctic
![ANIMAL_CELL_,_TISSUE_AND_ORGAN_CULTURE[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/animalcelltissueandorganculture1-260204172026-4462b440-thumbnail.jpg?width=640&height=640&fit=bounds)
![Polymer [ बहुलक ] Chemistry Notes PDF - Irfanullah Mehar - JJ Sir Chemistry.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/polymerchemistrynotespdf-irfanullahmehar-jjsirchemistry-260210172118-3f9b37f7-thumbnail.jpg?width=640&height=640&fit=bounds)
Explainable AI approach for evaluating climate models in the Arctic
- 1. Explainable AI approachfor evaluating climate models in the Arctic Zachary M. Labe Postdoc at NOAA GFDL and Princeton University; Atmospheric and Oceanic Science with Elizabeth A. Barnes (Colorado State University) 27 March 2024 IARPC Collaborations Modelers’ Community of Practice https://zacklabe.com/ @ZLabe
- 2.
- 4. STANDARD EVALUATION OF CLIMATEMODELS Pattern correlation RMSE EOFs Trends, anomalies, mean state Climate modes of variability Negative Correlation Positive Correlation PATTERN CORRELATION – T2M PATTERN CORRELATION : NEAR-SURFACE AIR TEMPERATURE
- 5. INPUT PREDICTION SO, WHATABOUT MACHINE LEARNING?
- 6. Machine Learning is notnew! “A Bayesian Neural Network for Severe-Hail Prediction (2000)” “Classification of Convective Areas Using Decision Trees (2009)” “A Neural Network for Damaging Wind Prediction (1998)” “Generative Additive Models versus Linear Regression in Generating Probabilistic MOS Forecasts of Aviation Weather Parameters (1995)” ”A Neural Network for Tornado Prediction Based on Doppler Radar-Derived Attributes (1996)” ”The Diagnosis of Upper-Level Humidity (1968)”
- 7. Do it better e.g.,parameterizations in climate models are not perfect, use ML to make them more accurate Do it faster e.g., code in climate models is very slow (but we know the right answer) - use ML methods to speed things up Do something new • e.g., go looking for non-linear relationships you didn’t know were there WHY ELSE SHOULD WE CONSIDER MACHINE LEARNING?
- 8. Do it better e.g.,parameterizations in climate models are not perfect, use ML to make them more accurate Do it faster e.g., code in climate models is very slow (but we know the right answer) - use ML methods to speed things up Do something new • e.g., go looking for non-linear relationships you didn’t know were there WHY ELSE SHOULD WE CONSIDER MACHINE LEARNING? Very relevant for research: may be slower and worse, but can still learn something
- 9. What is theannual mean temperature of Earth?
- 10. THE REAL WORLD (Observations) Whatis the annual mean temperature of Earth? Data from Berkeley Earth Surface Temperature 1930 2022
- 11. What is theannual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model One ensemble member 2022 1930 2050 Data from NO AA/G FDL SPEAR
- 12. What is theannual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model again! Two ensemble members Data from NO AA/G FDL SPEAR
- 13. What is theannual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model again & again! Three ensemble members Data from NO AA/G FDL SPEAR
- 14. What is theannual mean temperature of Earth? THE REAL WORLD (Observations) CLIMATE MODEL LARGE ENSEMBLE 30 ensemble members in NOAA/GFDL SPEAR
- 15. What is theannual mean temperature of Earth? Mean of ensembles = forced response (climate change) THE REAL WORLD (Observations) CLIMATE MODEL LARGE ENSEMBLE 30 ensemble members In GFDL SPEAR
- 16. What is theannual mean temperature of Earth? Mean of ensembles = forced response (climate change) THE REAL WORLD (Observations) CLIMATE MODEL LARGE ENSEMBLE 30 ensemble members In GFDL SPEAR Range of ensembles = internal variability (noise) Mean of ensembles = forced response (climate change)
- 17. But let’s remove climatechange… Climate Change Signal (ensemble mean) Observations Ensemble Members
- 18. Ensemble Members Mean of anomalies After removingthe forced response… = anomalies/noise!
- 19. Ensemble members in GFDL SPEAR Mapsof a given time period for each ensemble Inputs for machine learning
- 20. Ensemble members in GFDL SPEAR TrainingData: 24 ensemble members Maps of a given time period for each ensemble
- 21. Training Data: 24 ensemblemembers Validation Data: 4 ensemble members
- 22. Training Data: 24 ensemblemembers Validation Data: 4 ensemble members Testing Data: 2 ensemble members
- 23. 2-m Actual AirTemperature (°C) THERE ARE MANY CLIMATE MODEL LARGE ENSEMBLES… Annual mean 2-m temperature 7 global climate models 16 ensembles each ERA5 (observations)
- 24.
- 25. ----ANN---- 2 Hidden Layers 10Nodes each Ridge Regularization Early Stopping TEMPERATURE We know some metadata… + What year is it? (Labe & Barnes, 2021) + Where did it come from?
- 26. TEMPERATURE We know somemetadata… + What year is it? (Labe & Barnes, 2021) + Where did it come from? Train on data from the Multi-Model Large Ensemble Archive
- 27. TEMPERATURE We know somemetadata… + What year is it? (Labe & Barnes, 2021) + Where did it come from? NEURAL NETWORK CLASSIFICATION TASK HIDDEN LAYERS INPUT LAYER INPUT LAYER OUTPUT LAYER HIDDEN LAYERS
- 28.
- 29.
- 30.
- 31. WHY LAYER-WISE RELEVANCE PROPAGATION(LRP) Volcano Timber Wolf Image Classification LRP LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY Backpropagation – LRP https://heatmapping.org/
- 32. WHY LAYER-WISE RELEVANCE PROPAGATION(LRP) Volcano Timber Wolf Image Classification LRP LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY Backpropagation – LRP https://heatmapping.org/
- 33. EXPLAINABLE AI (XAI) THEREARE MANY METHODS A bird! XAI [Adapted from Adebayo et al., 2020]
- 34. THERE ARE MANY METHODS EXPLAINABLEAI (XAI) [Adapted from Adebayo et al., 2020]
- 35. Visualizing something wealready know… ENSO
- 36. Neural Network [0] La Niña[1] El Niño Input a map of sea surface temperatures [Toms et al. 2020, JAMES]
- 37. Visualizing something wealready know… Input maps of sea surface temperatures (SST) to identify El Niño or La Niña Use ‘LRP’ to see how the neural network is making its decision [Toms et al. 2020, JAMES] Layer-wise Relevance Propagation Composite SST Observations LRP [Relevance] SST Anomaly [°C] 0.00 0.75 0.0 1.5 -1.5 Warmer Colder High Low
- 38. APPLYING METHODOLOGY TOTHE ARCTIC
- 39. NEURAL NETWORK CLASSIFICATION TASK HIDDENLAYERS INPUT LAYER OUTPUT LAYER TEMPERATURE MAP LABE AND BARNES 2022, ESS
- 40. APPLY SOFTMAX OPERATOR INTHE OUTPUT LAYER RANK LABE AND BARNES 2022, ESS
- 41. APPLY SOFTMAX OPERATOR INTHE OUTPUT LAYER [ 0.71 ] [ 0.05 ] [ 0.01 ] [ 0.01 ] [ 0.03 ] [ 0.11 ] [ 0.08 ] RANK LABE AND BARNES 2022, ESS
- 42. APPLY SOFTMAX OPERATOR INTHE OUTPUT LAYER [ 0.71 ] [ 0.05 ] [ 0.01 ] [ 0.01 ] [ 0.03 ] [ 0.11 ] [ 0.08 ] RANK [ 1 ] [ 4 ] [ 7 ] [ 6 ] [ 5 ] [ 2 ] [ 3 ] LABE AND BARNES 2022, ESS
- 43. APPLY SOFTMAX OPERATOR INTHE OUTPUT LAYER [ 0.71 ] [ 0.05 ] [ 0.01 ] [ 0.01 ] [ 0.03 ] [ 0.11 ] [ 0.08 ] RANK [ 1 ] [ 4 ] [ 7 ] [ 6 ] [ 5 ] [ 2 ] [ 3 ] Confidence/Probability LABE AND BARNES 2022, ESS
- 44. RANKING CLIMATE MODELPREDICTIONS FOR EACH YEAR IN OBSERVATIONS LABE AND BARNES 2022, ESS
- 45. COMPARING CLIMATE MODELSIN THE ARCTIC High Low RECENT ARCTIC AMPLIFICATION LABE AND BARNES 2022, ESS
- 46. High Low HISTORICAL PERIOD COMPARING CLIMATEMODELS IN THE ARCTIC LABE AND BARNES 2022, ESS
- 47. High Low DIFFERENCE IN LAYER-WISERELEVANCE PROPAGATION COMPARING CLIMATE MODELS IN THE ARCTIC LABE AND BARNES 2022, ESS
- 48. KEY POINTS 1. Artificialneural network can identify which climate model produced an annual mean map of near-surface temperature in the Arctic 2. Classification network is evaluated using input from atmospheric reanalysis as a method of comparing climate models and observations 3. XAI method reveals regional temperature patterns the artificial neural network is using to classify observations with different climate models Zachary Labe zachary.labe@noaa.gov Labe, Z. M., & Barnes, E. A. (2022). Comparison of climate model large ensembles with observations in the Arctic using simple neural networks. Earth and Space Science, 9(7), e2022EA002348 https://doi.org/10.1029/2022EA002348