Using explainable AI to identify key regions of climate change in GFDL SPEAR large ensembles
•
0 likes•52 views
15 March 2023… GFDL Lunchtime Seminar Series (Presentation): Using explainable AI to identify key regions of climate change in GFDL SPEAR large ensembles, Princeton, NJ. References... Labe, Z.M., E.A. Barnes, and J.W. Hurrell (2023). Identifying the regional emergence of climate patterns in the ARISE-SAI-1.5 simulations. https://doi.org/10.31223/X5394Z (submitted)
Recommended
Recommended
More Related Content
Similar to Using explainable AI to identify key regions of climate change in GFDL SPEAR large ensembles
Similar to Using explainable AI to identify key regions of climate change in GFDL SPEAR large ensembles (20)
More from Zachary Labe
More from Zachary Labe (20)
Recently uploaded
Recently uploaded (20)
Using explainable AI to identify key regions of climate change in GFDL SPEAR large ensembles
- 1. Using explainable AI to identify key regions of climate change in GFDL SPEAR large ensembles https://zacklabe.com/ @ZLabe Zachary Labe Postdoc in Seasonal-to-Decadal (S2D) Variability and Predictability Division with Nathaniel Johnson and Thomas Delworth 15 March 2023 GFDL Lunchtime Seminar
- 2. 1 1) Where do we go from here?
- 3. 2 1) Where do we go from here? 2) How do we disentangle internal climate variability? Feb/Mar 2016
- 4. 3 3) How do we account for regional patterns of change?
- 5. Explainable machine learning can distinguish between regional patterns of time-evolving climate change in GFDL models. 4 SIGNIFICANCE
- 6. Machine Learning is not new! “A Bayesian Neural Network for Severe-Hail Prediction (2000)” “Classification of Convective Areas Using Decision Trees (2009)” “A Neural Network for Damaging Wind Prediction (1998)” “Generative Additive Models versus Linear Regression in Generating Probabilistic MOS Forecasts of Aviation Weather Parameters (1995)” ”A Neural Network for Tornado Prediction Based on Doppler Radar-Derived Attributes (1996)” ”The Diagnosis of Upper-Level Humidity (1968)” 5
- 7. “An adaptive data processing system for weather forecasting” It’s a neural network! [Hu and Root (1964), APME] 6
- 8. Artificial Intelligence Machine Learning Deep Learning Computer/Data Science 7
- 9. Computer/Data Science Supervised Learning Unsupervised Learning Labeled data Classification Regression Unlabeled data Clustering Dimension reduction 8 Artificial Intelligence Machine Learning Deep Learning DATA-HUNGRY!
- 10. Do it better e.g., parameterizations in climate models are not perfect, use ML to make them more accurate Do it faster e.g., code in climate models is very slow (but we know the right answer) - use ML methods to speed things up Do something new • e.g., go looking for non-linear relationships you didn’t know were there WHY ELSE SHOULD WE CONSIDER MACHINE LEARNING? 9
- 11. Do it better e.g., parameterizations in climate models are not perfect, use ML to make them more accurate Do it faster e.g., code in climate models is very slow (but we know the right answer) - use ML methods to speed things up Do something new • e.g., go looking for non-linear relationships you didn’t know were there Very relevant for research: may be slower and worse, but can still learn something 10 WHY ELSE SHOULD WE CONSIDER MACHINE LEARNING?
- 12. Machine learning for meteorology IDENTIFYING SEVERE THUNDERSTORMS Molina et al. 2021 Martin et al. 2022 CLASSIFYING PHASE OF MADDEN-JULLIAN OSCILLATION DETECTING CONVECTION FROM SATELLITES Lee et al. 2021 LOCATING COLD FRONTS Dagon et al. 2022 11
- 13. Machine learning for oceanography CLASSIFYING ARCTIC OCEAN ACIDIFICATION Krasting et al. 2022 LARGE-SCALE OCEAN CIRCULATION Clare et al. 2022 ESTIMATING OCEAN SURFACE CURRENTS Sinha and Abernathey, 2021 12
- 14. Machine learning for climate PHYSICAL DRIVERS OF ENSO DYNAMICS Shin et al. 2022 IDENTIFYING DECADAL STATE DEPENDENCE Gordon and Barnes, 2022 INTERNAL/EXTERNAL CLIMATE FORCING Po-Chedley et al. 2022 13
- 18. Artificial Intelligence Machine Learning Deep Learning Artificial Neural Networks 17 Computer/Data Science
- 19. X1 X2 INPUTS Artificial Neural Networks [ANN] 18
- 20. Linear regression! Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ = X1W1+ X2W2 + b INPUTS NODE 19
- 21. Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ INPUTS NODE Linear regression with non-linear mapping by an “activation function” Training of the network is merely determining the weights “w” and bias/offset “b" = factivation(X1W1+ X2W2 + b) 20
- 22. Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ INPUTS NODE = factivation(X1W1+ X2W2 + b) ReLU Sigmoid Linear 21
- 23. X1 X2 ∑ inputs HIDDEN LAYERS X3 ∑ ∑ ∑ OUTPUT = predictions Artificial Neural Networks [ANN] : : :: INPUTS 22
- 24. Complexity and nonlinearities of the ANN allow it to learn many different pathways of predictable behavior Once trained, you have an array of weights and biases which can be used for prediction on new data INPUT [DATA] PREDICTION Artificial Neural Networks [ANN] 23
- 25. What is the annual mean temperature of Earth? 24
- 26. THE REAL WORLD (Observations) What is the annual mean temperature of Earth? Data from Berkeley Earth Surface Temperature 1930 2022 25
- 27. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model One ensemble member 2022 1930 2050 26 Data from SPEAR_M ED
- 28. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model again! Two ensemble members 27 Data from SPEAR_M ED
- 29. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model again & again! Three ensemble members 28 Data from SPEAR_M ED
- 30. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) CLIMATE MODEL LARGE ENSEMBLE 30 ensemble members in GFDL SPEAR 29
- 31. What is the annual mean temperature of Earth? Mean of ensembles = forced response (climate change) THE REAL WORLD (Observations) CLIMATE MODEL LARGE ENSEMBLE 30 ensemble members In GFDL SPEAR 30
- 32. What is the annual mean temperature of Earth? Mean of ensembles = forced response (climate change) THE REAL WORLD (Observations) CLIMATE MODEL LARGE ENSEMBLE 30 ensemble members In GFDL SPEAR Range of ensembles = internal variability (noise) Mean of ensembles = forced response (climate change) 31
- 33. But let’s remove climate change… Climate Change Signal (ensemble mean) Observations Ensemble Members 32
- 34. Ensemble Members Mean of anomalies After removing the forced response… = anomalies/noise! 33
- 35. Ensemble members in GFDL SPEAR Maps of a given time period for each ensemble Inputs for machine learning 34
- 36. Ensemble members in GFDL SPEAR 35 Training Data: 24 ensemble members Maps of a given time period for each ensemble
- 37. 36 Training Data: 24 ensemble members Validation Data: 4 ensemble members
- 38. 37 Training Data: 24 ensemble members Validation Data: 4 ensemble members Testing Data: 2 ensemble members
- 39. Historical Forcing – GFDL SPEAR Future Scenarios – GFDL SPEAR 38 Can a neural network learn unique patterns of climate change related to each future emission scenario? 1930 2010 2020 2100
- 40. 39 Train a neural network to predict 5 classes (climate scenarios)
- 41. Yearly Maps of T2M Yearly Maps of T2M Neural Network Classify Climate Scenario Artificial Neural Network Output = 5 Classes Yearly Maps of T2M Neural Network Binary Output Binary Output 40 Step #1 Read in gridded maps of a climate variable from SPEAR simulations
- 42. Yearly Maps of T2M Yearly Maps of T2M Neural Network Classify Climate Scenario Artificial Neural Network Output = 5 Classes Yearly Maps of T2M Neural Network Binary Output Binary Output 41 Step #2 Feed data into an artificial neural network with three hidden layers
- 43. Yearly Maps of T2M Yearly Maps of T2M Neural Network Classify Climate Scenario Artificial Neural Network Output = 5 Classes Yearly Maps of T2M Neural Network Binary Output Binary Output 42 Step #3 Classify which climate scenario (n=5) is associated with each map
- 44. Yearly Maps of T2M Yearly Maps of T2M Neural Network Classify Climate Scenario Artificial Neural Network Output = 5 Classes Yearly Maps of T2M Neural Network Binary Output Binary Output 43 Step #4 Why? à XAI
- 45. WHY LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Timber Wolf Image Classification LRP LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY Backpropagation – LRP https://heatmapping.org/ 44
- 46. WHY LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Timber Wolf Image Classification LRP LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY Backpropagation – LRP https://heatmapping.org/ 45
- 47. Image Classification LRP https://heatmapping.org/ NOT PERFECT Crock Pot Neural Network Backpropagation – LRP WHY 46 LAYER-WISE RELEVANCE PROPAGATION (LRP)
- 48. EXPLAINABLE AI (XAI) THERE ARE MANY METHODS A bird! XAI 47 [Adapted from Adebayo et al., 2020]
- 49. THERE ARE MANY METHODS EXPLAINABLE AI (XAI) 48 [Adapted from Adebayo et al., 2020]
- 50. Visualizing something we already know… ENSO 49
- 51. Neural Network [0] La Niña [1] El Niño Input a map of sea surface temperatures 50 [Toms et al. 2020, JAMES]
- 52. Visualizing something we already know… Input maps of sea surface temperatures (SST) to identify El Niño or La Niña Use ‘LRP’ to see how the neural network is making its decision [Toms et al. 2020, JAMES] Layer-wise Relevance Propagation Composite SST Observations LRP [Relevance] SST Anomaly [°C] 0.00 0.75 0.0 1.5 -1.5 51 Warmer Colder High Low
- 53. 52 Returning to our application…
- 54. 53 Predictions for SPEAR_MED Testing Data Accuracy=92% Nearer to predicted class Further from predicted class
- 55. 54 Predictions for SPEAR_MED Testing Data Accuracy=92% Nearer to predicted class Further from predicted class
- 56. Global Mean Surface Temperature Can we identify changes in future climate impacts after rapid mitigation? 55
- 57. 56 30 ensembles for GFDL SPEAR_MED 9 ensembles for GFDL SPEAR_MED Input maps from out-of-sample ensembles into classification network 2020 2030 2100
- 58. 57 2031 2040 Rapid Mitigation Rapid Mitigation 30 ensembles for GFDL SPEAR_MED 9 ensembles for GFDL SPEAR_MED
- 59. 58 Predictions for a single ensemble member from SSP5-3.4OS SSP5-8.5 SSP2-4.5 2015 2055 2065 2095
- 60. 59 Predictions for a single ensemble member from SSP5-3.4OS SSP5-8.5 SSP2-4.5 2055-2060 rapid mitigation begins 2015 2095
- 61. 60 Are these predictions robust across ensemble members? (n=30) SSP5-3.4OS Transition from SSP5-8.5 to SSP2-4.5 2015 2060 2100
- 62. 61 What if we start mitigation 10 years earlier? Transition from SSP5-8.5 to SSP2-4.5 Transition from SSP5-8.5 to SSP2-4.5 Transition from SSP2-4.5 to SSP1-1.9 2015 2060 2100 2015 2060 2100
- 63. 62 What climate patterns are associated with these transitions? Transition from SSP5-8.5 to SSP2-4.5 Transition from SSP5-8.5 to SSP2-4.5 Transition from SSP2-4.5 to SSP1-1.9 Rapid mitigation Rapid mitigation
- 64. 63 Difficult to distinguish the patterns associated with scenario transitions Composites of relevance maps for the mitigation predictions SSP5-3.4OS example for 2015 to 2100 Nearer to predicted scenario Further from predicted scenario
- 65. Yearly Maps of T2M Yearly Maps of T2M Neural Network Classify Climate Scenario Artificial Neural Network Output = 5 Classes Yearly Maps of T2M Neural Network Binary Output Binary Output 64 Steps #4-5
- 66. 65 XAI composites of years associated with the transition from SSP5-8.5 to SSP2-4.5 (a) approx. 2055-2060 (b) approx. 2040-2045
- 67. 66 North Atlantic is an important indictor region for climate signals related to identifying from SSP5-8.5 to SSP2-4.5 Future Climate Change Rapid Mitigation
- 68. 67 Framework can be applied to different geographic regions and climate variables Parallel approach for detecting climate intervention scenarios Labe, Z.M., E.A. Barnes, and J.W. Hurrell (2023). Identifying the regional emergence of climate patterns in the ARISE-SAI-1.5 simulations. EarthArXiv, DOI: 10.31223/X5394Z
- 69. 1) Explainable AI methods identify regions of climate change that indicate the emission scenario 2) Framework for outlining benefits from mitigation by testing out of sample ensembles from SSP5-3.4OS 3) Patterns of fast and slow responses to rapid climate mitigation are revealed by explainable AI KEY FINDINGS • Indicator patterns of climate change • Quantify benefits of rapid climate mitigation • Regions of rapid or slow response to mitigation Labe, Z.M., N.C. Johnson, and T.L. Delworth (2023), A data-driven approach to identifying key regions of change associated with rapid climate mitigation, in prep
- 70. 1) Explainable AI methods identify regions of climate change that indicate the emission scenario 2) Framework for outlining benefits from mitigation by testing out of sample ensembles from SSP5-3.4OS 3) Patterns of fast and slow responses to rapid climate mitigation are revealed by explainable AI KEY FINDINGS • Indicator patterns of climate change • Quantify benefits of rapid climate mitigation • Regions of rapid or slow response to mitigation Labe, Z.M., N.C. Johnson, and T.L. Delworth (2023), A data-driven approach to identifying key regions of change associated with rapid climate mitigation, in prep
- 71. 1) Explainable AI methods identify regions of climate change that indicate the emission scenario 2) Framework for outlining benefits from mitigation by testing out of sample ensembles from SSP5-3.4OS 3) Patterns of fast and slow responses to rapid climate mitigation are revealed by explainable AI KEY FINDINGS • Indicator patterns of climate change • Quantify benefits of rapid climate mitigation • Regions of rapid or slow response to mitigation Labe, Z.M., N.C. Johnson, and T.L. Delworth (2023), A data-driven approach to identifying key regions of change associated with rapid climate mitigation, in prep
- 73. 72
- 75. [Eischeid et al. 2023, revised] 74
- 76. Temperature anomalies [ °C ] relative to 1981-2010 Observations from NClimGrid Climate model data from GFDL SPEAR_MED United States – Summer 75 1920 2020
- 77. TEMPERATURE 76
- 78. TEMPERATURE We know some metadata… + What year is it? + Where did it come from? 77
- 79. TEMPERATURE We know some metadata… + What year is it? + Where did it come from? 78
- 80. We know some metadata… + What year is it? + Where did it come from? [Labe and Barnes, 2022; ESS] TEMPERATURE 79
- 81. TEMPERATURE Neural network learns nonlinear combinations of forced climate patterns to identify the year We know some metadata… + What year is it? + Where did it come from? [Labe and Barnes, 2022; ESS] 80
- 82. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping [e.g., Barnes et al. 2019, 2020] [e.g., Labe and Barnes, 2021] TIMING OF EMERGENCE (COMBINED VARIABLES) RESPONSES TO EXTERNAL CLIMATE FORCINGS PATTERNS OF CLIMATE INDICATORS [e.g., Rader et al. 2022] Surface Temperature Map Precipitation Map + TEMPERATURE We know some metadata… + What year is it? + Where did it come from? [Labe and Barnes, 2022; ESS] 81
- 83. Seasonal Maps of T2M, TMAX, TMIN Input 82
- 84. Seasonal Maps of T2M, TMAX, TMIN Hidden Layers Artificial Neural Network Input 83
- 85. Seasonal Maps of T2M, TMAX, TMIN Hidden Layers Output 1921 2100 Artificial Neural Network Input 84
- 86. Backpropagation Seasonal Maps of T2M, TMAX, TMIN Hidden Layers Output 1921 2100 Artificial Neural Network Input 85 Post hoc – XAI methods
- 87. Temperature anomalies [ °C ] relative to 1981-2010 TMIN TMAX 1921 to 2022 1921 to 2022 86 Summer (June-August)
- 89. 88 1921 2021 2100 ACTUAL YEARS PREDICTED YEARS NOAA Monthly U.S. NClimGrid v1.0 1921 2021 2100
- 90. Max year predicted in the 1921-1950 baseline for observations Timing of Emergence 89 1921-1950 Skill 1:1 Perfect Prediction
- 92. June – August – Timing of Emergence (ToE) for observations 91 June – August – skill for observations
- 93. How is the neural network able to detect the year prior to ~1990? Temperature anomalies [ °C ] relative to 1981-2010 Machine learning predictions GFDL SPEAR_MED simulation 92
- 94. Machine Learning Explainability Methods – Attribution 93 Decrease likelihood of year Increase likelihood of year
- 95. Western USA Central USA Eastern USA 94
- 96. Western USA Central USA Eastern USA 95
- 97. Western USA Central USA Eastern USA 96
- 99. First year that the 10-year running-mean temperature exceeds and stays above the mean 1921–1950 reference temperature by more than two standard deviations Most Areas = 2000s 98
- 100. Natural Forcings 99
- 102. 1) Is it aerosols? 101 Only available from 1921 to 2020
- 103. Proof of Concept 102
- 104. 50 km resolution 100 km resolution 103 2) Is it related to resolution? SPEAR_MED SPEAR_LOW MAE (years)
- 105. 3) Is it systematic in CMIP6? 104
- 106. Machine Learning Explainability Methods (Attribution) for CESM2-LE 105 Decrease likelihood of year Increase likelihood of year
- 107. 4) So, what is it? The land? 106
- 108. TRENDS FROM 1921 TO 1950 Fully-Coupled [Historical + SSP5-8.5] SPEAR_MED, but NO anthropogenic aerosols SPEAR_MED, but NO anthropogenic forcings 107 Warmer Colder
- 109. TRENDS FROM 1921 TO 1950 Fully-Coupled [Historical + SSP5-8.5] SPEAR_MED, but NO anthropogenic aerosols SPEAR_MED, but NO anthropogenic forcings 108 Warmer Colder
- 110. TRENDS FROM 1921 TO 1950 Fully-Coupled [Historical + SSP5-8.5] SPEAR_MED, but NO anthropogenic aerosols SPEAR_MED, but NO anthropogenic forcings 109 Warmer Colder
- 111. TRENDS IN EVAPORATION Fully-Coupled [Historical + SSP5-8.5] SPEAR_MED, but NO anthropogenic aerosols SPEAR_MED, but NO anthropogenic forcings 110 Increase Decrease
- 112. 111 ERA5 1979-2021 Input maps of P-E for Western USA
- 114. 113 What about where we live? Summer (June-August) Eastern USA X = mean
- 115. 1. Increasing spatial resolution improves the neural network skill for identifying climate signals 2. Externally-forced temperature signals have emerged in observations in the United States 3. Trends in western United States land surface fields are linked to timing of emergence in SPEAR KEY FINDINGS • Machine learning prediction skill • ToE in observations • Physical drivers of climate signals Labe, Z.M., N.C. Johnson, and T.L. Delworth (2023), Forced signals in United States summer temperatures revealed by explainable neural networks in climate models and observations, in prep
- 116. 1. Increasing spatial resolution improves the neural network skill for identifying climate signals 2. Externally-forced temperature signals have emerged in observations in the United States 3. Trends in western United States land surface fields are linked to timing of emergence in SPEAR KEY FINDINGS • Machine learning prediction skill • ToE in observations • Physical drivers of climate signals Labe, Z.M., N.C. Johnson, and T.L. Delworth (2023), Forced signals in United States summer temperatures revealed by explainable neural networks in climate models and observations, in prep
- 117. 1. Increasing spatial resolution improves the neural network skill for identifying climate signals 2. Externally-forced temperature signals have emerged in observations in the United States 3. Trends in western United States land surface fields are linked to timing of emergence in SPEAR KEY FINDINGS • Machine learning prediction skill • ToE in observations • Physical drivers of climate signals Labe, Z.M., N.C. Johnson, and T.L. Delworth (2023), Forced signals in United States summer temperatures revealed by explainable neural networks in climate models and observations, in prep
- 118. INPUT [DATA] PREDICTION Machine Learning Explainable (or interpretable) AI Learn new climate science! 117
- 119. XAI can identify regional patterns of climate change & variability in GFDL large ensembles. 1) 118
- 120. Method can identify differences in time-evolving climate signals between other climate model large ensembles. 2) 119
- 121. Framework can be adapted for monitoring and predicting patterns of climate change in observations. 3) 120
- 122. DECADAL CLIMATE PREDICTION Explainable machine learning for improving prediction skill and identifying physical drivers DETECTION AND ATTRIBUTION Classification neural network for monitoring extreme events in climate models and observations FUTURE DIRECTIONS AT GFDL DIAGNOSTIC TOOL FOR MODEL BIASES Method for extracting forced climate signals across Earth system models
- 123. TAKEAWAYS 1. XAI can identify regional patterns of climate change & variability in GFDL large ensembles. 2. Method can identify differences in time-evolving forced climate signals between other climate model large ensembles. 3. Framework can be adapted for monitoring and predicting patterns of climate change in observations. Zack Labe zachary.labe@noaa.gov 122 15 March 2023 GFDL Lunchtime Seminar