Exploring climate change signals with explainable AI
•
0 likes•167 views
This document provides an overview of using machine learning and explainable AI techniques to analyze climate model data and make predictions about climate signals and forcings. It discusses: 1) The growing use of machine learning tools like neural networks in weather and climate research to better understand models, find relationships in data, and make predictions. 2) An example of using a neural network to classify decade from surface temperature maps and explaining the predictions using Layer-wise Relevance Propagation. 3) Results showing the neural network had greater skill in identifying patterns related to greenhouse gas forcing versus aerosol forcing, and LRP helped identify the relevant regions driving these predictions.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Exploring climate change signals with explainable AI
Similar to Exploring climate change signals with explainable AI (20)
More from Zachary Labe
More from Zachary Labe (18)
Recently uploaded
Recently uploaded (20)
Exploring climate change signals with explainable AI
- 1. Exploring climate change signals with explainable AI @ZLabe Zachary M. Labe with Elizabeth A. Barnes in the Department of Atmospheric Science at Colorado State University 9 December 2021 Carbon Club NASA Jet Propulsion Laboratory (JPL)
- 2. Machine Learning is not new! But…
- 3. Machine Learning is not new!
- 4. Artificial Intelligence Machine Learning Deep Learning Computer Science
- 5. Computer Science Artificial Intelligence Machine Learning Deep Learning Supervised Learning Unsupervised Learning Labeled data Classification Regression Unlabeled data Clustering Dimension reduction
- 6. • Do it better • e.g., parameterizations in climate models are not perfect, use ML to make them more accurate • Do it faster • e.g., code in climate models is very slow (but we know the right answer) - use ML methods to speed things up • Do something new • e.g., go looking for non-linear relationships you didn’t know were there Very relevant for research: may be slower and worse, but can still learn something WHY SHOULD WE CONSIDER MACHINE LEARNING?
- 7. GROWING DATA
- 12. Python tools for machine learning
- 13. Machine learning for weather IDENTIFYING SEVERE THUNDERSTORMS Molina et al. 2021 Toms et al. 2021 CLASSIFYING PHASE OF MADDEN-JULLIAN OSCILLATION SATELLITE DETECTION Lee et al. 2021 DETECTING TORNADOES McGovern et al. 2019
- 14. Machine learning for climate FINDING FORECASTS OF OPPORTUNITY Mayer and Barnes, 2021 PREDICTING CLIMATE MODES OF VARIABILITY Gordon et al. 2021 TIMING OF CLIMATE CHANGE Barnes et al. 2019
- 17. NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography (AI2ES) https://www.ai2es.org/
- 18. E.g., Research to Operations (R2O) Tornado Warning Special Marine Warning Severe Thunderstorm Warning Flash Flood Warning
- 19. E.g., Establish robust, responsible AI for severe weather detection Tornado Warning Special Marine Warning Severe Thunderstorm Warning Flash Flood Warning
- 20. Artificial Intelligence Machine Learning Deep Learning
- 21. X1 X2 INPUTS Artificial Neural Networks [ANN]
- 22. Linear regression! Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ = X1W1+ X2W2 + b INPUTS NODE
- 23. Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ INPUTS NODE Linear regression with non-linear mapping by an “activation function” Training of the network is merely determining the weights “w” and bias/offset “b" = factivation(X1W1+ X2W2 + b)
- 24. Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ INPUTS NODE = factivation(X1W1+ X2W2 + b) ReLU Sigmoid Linear
- 25. X1 X2 ∑ inputs HIDDEN LAYERS X3 ∑ ∑ ∑ OUTPUT = predictions Artificial Neural Networks [ANN] : : :: INPUTS
- 26. Complexity and nonlinearities of the ANN allow it to learn many different pathways of predictable behavior Once trained, you have an array of weights and biases which can be used for prediction on new data INPUT [DATA] PREDICTION Artificial Neural Networks [ANN]
- 27. TEMPERATURE
- 28. TEMPERATURE We know some metadata… + What year is it? + Where did it come from?
- 29. We know some metadata… + What year is it? + Where did it come from? TEMPERATURE
- 30. We know some metadata… + What year is it? + Where did it come from? TEMPERATURE Neural network learns nonlinear combinations of forced climate patterns to identify the year
- 31. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping We know some metadata… + What year is it? + Where did it come from? [e.g., Barnes et al. 2019, 2020] [e.g., Labe and Barnes, 2021] TIMING OF EMERGENCE (COMBINED VARIABLES) RESPONSES TO EXTERNAL CLIMATE FORCINGS PATTERNS OF CLIMATE INDICATORS [e.g., Rader et al. 2021, in prep] Surface Temperature Map Precipitation Map + TEMPERATURE
- 32. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping We know some metadata… + What year is it? + Where did it come from? [e.g., Barnes et al. 2019, 2020] [e.g., Labe and Barnes, 2021] TIMING OF EMERGENCE (COMBINED VARIABLES) RESPONSES TO EXTERNAL CLIMATE FORCINGS PATTERNS OF CLIMATE INDICATORS [e.g., Rader et al. 2021, in prep] Surface Temperature Map Precipitation Map + TEMPERATURE
- 33. THE REAL WORLD (Observations) What is the annual mean temperature of Earth?
- 34. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Anomaly is relative to 1951-1980
- 35. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) CLIMATE MODEL ENSEMBLES
- 36. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) CLIMATE MODEL ENSEMBLES Range of ensembles = internal variability (noise) Mean of ensembles = forced response (climate change)
- 37. What is the annual mean temperature of Earth? • Increasing greenhouse gases (CO2, CH4, N2O) • Changes in industrial aerosols (SO4, BC, OC) • Changes in biomass burning (aerosols) • Changes in land-use & land-cover (albedo)
- 38. What is the annual mean temperature of Earth? • Increasing greenhouse gases (CO2, CH4, N2O) • Changes in industrial aerosols (SO4, BC, OC) • Changes in biomass burning (aerosols) • Changes in land-use & land-cover (albedo) Plus everything else… (Natural/internal variability)
- 39. What is the annual mean temperature of Earth?
- 40. Greenhouse gases fixed to 1920 levels All forcings (CESM-LE) Industrial aerosols fixed to 1920 levels [Deser et al. 2020, JCLI] Fully-coupled CESM1.1 20 Ensemble Members Run from 1920-2080 Observations
- 41. So what? Greenhouse gases = warming Aerosols = ?? (though mostly cooling) What are the relative responses between greenhouse gas and aerosol forcing?
- 42. Surface Temperature Map ARTIFICIAL NEURAL NETWORK (ANN)
- 43. INPUT LAYER Surface Temperature Map ARTIFICIAL NEURAL NETWORK (ANN)
- 44. INPUT LAYER Surface Temperature Map ARTIFICIAL NEURAL NETWORK (ANN) Collection of nodes (neurons) that adjust their weights and biases across layers in order to learn signals for making predictions Learns nonlinear processes through selected parameters in the model
- 45. INPUT LAYER HIDDEN LAYERS OUTPUT LAYER Surface Temperature Map “2000-2009” DECADE CLASS “2070-2079” “1920-1929” ARTIFICIAL NEURAL NETWORK (ANN)
- 46. INPUT LAYER HIDDEN LAYERS OUTPUT LAYER Surface Temperature Map “2000-2009” DECADE CLASS “2070-2079” “1920-1929” BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI ARTIFICIAL NEURAL NETWORK (ANN)
- 47. INPUT LAYER HIDDEN LAYERS OUTPUT LAYER Layer-wise Relevance Propagation Surface Temperature Map “2000-2009” DECADE CLASS “2070-2079” “1920-1929” BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI ARTIFICIAL NEURAL NETWORK (ANN) [Barnes et al. 2020, JAMES] [Labe and Barnes 2021, JAMES]
- 48. LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Great White Shark Timber Wolf Image Classification LRP https://heatmapping.org/ LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY WHY WHY Backpropagation – LRP
- 49. LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Great White Shark Timber Wolf Image Classification LRP https://heatmapping.org/ LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY WHY WHY Backpropagation – LRP
- 50. LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Great White Shark Timber Wolf Image Classification LRP https://heatmapping.org/ LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY WHY WHY Backpropagation – LRP
- 51. LAYER-WISE RELEVANCE PROPAGATION (LRP) Image Classification LRP https://heatmapping.org/ NOT PERFECT Crock Pot Neural Network WHY Backpropagation – LRP
- 52. [Adapted from Adebayo et al., 2020] EXPLAINABLE AI IS NOT PERFECT THERE ARE MANY METHODS
- 53. [Adapted from Adebayo et al., 2020] THERE ARE MANY METHODS EXPLAINABLE AI IS NOT PERFECT
- 54. Visualizing something we already know…
- 55. Neural Network [0] La Niña [1] El Niño [Toms et al. 2020, JAMES] Input a map of sea surface temperatures
- 56. Visualizing something we already know… Input maps of sea surface temperatures to identify El Niño or La Niña Use ‘LRP’ to see how the neural network is making its decision [Toms et al. 2020, JAMES] Layer-wise Relevance Propagation Composite Observations LRP [Relevance] SST Anomaly [°C] 0.00 0.75 0.0 1.5 -1.5
- 57. OUTPUT LAYER Layer-wise Relevance Propagation “2000-2009” DECADE CLASS “2070-2079” “1920-1929” BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI WHY? = LRP HEAT MAPS [Labe and Barnes 2021, JAMES]
- 58. Layer-wise Relevance Propagation BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI WHY? = LRP HEAT MAPS Machine Learning Black Box [Labe and Barnes 2021, JAMES]
- 59. Layer-wise Relevance Propagation BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI WHY? = LRP HEAT MAPS Find regions of “relevance” that contribute to the neural network’s decision-making process [Labe and Barnes 2021, JAMES]
- 60. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 61. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 62. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 63. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 64. CLIMATE MODEL DATA PREDICT THE YEAR FROM MAPS OF TEMPERATURE AEROSOLS PREVAIL GREENHOUSE GASES PREVAIL STANDARD CLIMATE MODEL [Labe and Barnes 2021, JAMES]
- 65. OBSERVATIONS PREDICT THE YEAR FROM MAPS OF TEMPERATURE AEROSOLS PREVAIL GREENHOUSE GASES PREVAIL STANDARD CLIMATE MODEL [Labe and Barnes 2021, JAMES]
- 66. OBSERVATIONS SLOPES PREDICT THE YEAR FROM MAPS OF TEMPERATURE AEROSOLS PREVAIL GREENHOUSE GASES PREVAIL STANDARD CLIMATE MODEL [Labe and Barnes 2021, JAMES]
- 67. Low High HOW DID THE ANN MAKE ITS PREDICTIONS?
- 68. Low High HOW DID THE ANN MAKE ITS PREDICTIONS? WHY IS THERE GREATER SKILL FOR GHG+?
- 69. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 70. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 71. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 72. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 73. Higher LRP values indicate greater relevance for the ANN’s prediction AVERAGED OVER 1960-2039 Aerosol-driven Greenhouse gas-driven All forcings Low High [Labe and Barnes 2021, JAMES]
- 74. Greenhouse gas-driven Aerosol-driven All forcings AVERAGED OVER 1960-2039 [Labe and Barnes 2021, JAMES]
- 75. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping TEMPERATURE We know some metadata… + What year is it? + Where did it come from?
- 76. TEMPERATURE We know some metadata… + What year is it? + Where did it come from? Train on data from the Multi-Model Large Ensemble Archive
- 77. 2-m Temperature (°C) THERE ARE MANY CLIMATE MODEL LARGE ENSEMBLES… Annual mean 2-m temperature 7 global climate models 16 ensembles each ERA5-BE (observations)
- 78. STANDARD EVALUATION OF CLIMATE MODELS Pattern correlation RMSE EOFs Trends, anomalies, mean state Climate modes of variability
- 79. STANDARD EVALUATION OF CLIMATE MODELS Pattern correlation RMSE EOFs Trends, anomalies, mean state Climate modes of variability CORRELATION [R]
- 80. STANDARD EVALUATION OF CLIMATE MODELS Pattern correlation RMSE EOFs Trends, anomalies, mean state Climate modes of variability CORRELATION [R]
- 81. STANDARD EVALUATION OF CLIMATE MODELS Pattern correlation RMSE EOFs Trends, anomalies, mean state Climate modes of variability Negative Correlation Positive Correlation PATTERN CORRELATION – T2M
- 82. TEMPERATURE We know some metadata… + What year is it? + Where did it come from? NEURAL NETWORK CLASSIFICATION TASK HIDDEN LAYERS INPUT LAYER
- 83. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 84. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 85. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 86. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 87. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 88. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 89. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 90. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low EXPLAINABLE AI
- 91. What climate model does the neural network predict for each year of observations?
- 92. APPLY SOFTMAX OPERATOR IN THE OUTPUT LAYER RANK
- 93. APPLY SOFTMAX OPERATOR IN THE OUTPUT LAYER [ 0.71 ] [ 0.05 ] [ 0.01 ] [ 0.01 ] [ 0.03 ] [ 0.11 ] [ 0.08 ] RANK
- 94. APPLY SOFTMAX OPERATOR IN THE OUTPUT LAYER [ 0.71 ] [ 0.05 ] [ 0.01 ] [ 0.01 ] [ 0.03 ] [ 0.11 ] [ 0.08 ] RANK [ 1 ] [ 4 ] [ 7 ] [ 6 ] [ 5 ] [ 2 ] [ 3 ]
- 95. APPLY SOFTMAX OPERATOR IN THE OUTPUT LAYER [ 0.71 ] [ 0.05 ] [ 0.01 ] [ 0.01 ] [ 0.03 ] [ 0.11 ] [ 0.08 ] RANK [ 1 ] [ 4 ] [ 7 ] [ 6 ] [ 5 ] [ 2 ] [ 3 ] Confidence/Probability
- 96. RANKING CLIMATE MODEL PREDICTIONS FOR EACH YEAR IN OBSERVATIONS
- 97. RANKING CLIMATE MODEL PREDICTIONS FOR EACH YEAR IN OBSERVATIONS
- 99. MACHINE LEARNING IS JUST ANOTHER TOOL TO ADD TO OUR WORKFLOW. 1)
- 100. MACHINE LEARNING IS NO LONGER A BLACK BOX. 2)
- 101. WE CAN LEARN NEW SCIENCE FROM EXPLAINABLE AI. 3)
- 102. KEY POINTS Zachary Labe zmlabe@rams.colostate.edu @ZLabe 1. Machine learning is just another tool to add to our scientific workflow 2. We can use explainable AI (XAI) methods to peer into the black box of machine learning 3. We can learn new science by using XAI methods in conjunction with existing statistical tools