![Single Column Dynamics
x = [qv, s]
y = [SHF,LHF, TOASW,#]
dx
dt
= f(x, y(t)) + gLS(t)
Prognostic variables
Auxiliary variables](https://image.slidesharecdn.com/plenary-190730155604/75/coupling-neural-networks-to-gcms-18-2048.jpg?cb=1668248890)
Coupling Neural Networks to GCMs
•1 like•456 views
Report
Share
Download to read offline
Discusses recent work on developing machine learning parametrization based on high-resolution models.
Recommended
More Related Content
What's hot
What's hot(20)
Similar to Coupling Neural Networks to GCMs
Similar to Coupling Neural Networks to GCMs(20)
Recently uploaded
Recently uploaded(20)
Coupling Neural Networks to GCMs
- 1. Noah Brenowitz July 30, 2019 Princeton AOS Workshop, Princeton, NJ NOAH D. BRENOWITZ - NOAHB@VULCAN.COM Coupling Neural Networks to GCMs 1
- 2. Climate Modeling at Vulcan Inc. NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 2 Christopher Bretherton University of Washington Oliver Fuhrer, MeteoSwiss More people to come…. Research will be open source and openly published Collaborating with the FV3 Team at GFDL
- 3. Coarse-resolution dynamics Apparent heating (K/day) Apparent moistening (g/kg/day) s = T + g cp z q = Mass water vapor Mass dry air @s @t + v · rs = Q1 @q @t + v · rq = Q2 @u @t + v · ru + f ⇥ u 1 ⇢ rp = Q3 SW+ LW radiation, latent heating, etc https://trello.com/c/wV03fVXiz NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 4. One climate model grid box. NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 4Photo Credit: Becky Hornbrook, published on WCRP webpage
- 5. Traditional method for parameterization 5 GigaLES, Khairtoudinov
- 6. Climate models have biases in mean state CMIP5 (models) GPCP (observations) Hwang and Frierson (2013)
- 7. Machine learning builds black boxes Many 1000s of parameters Need a lot of data Designed to be trained not interpreted Examples: Decision trees, neural networks, support vector machines Easy to tune/train Easy to interpret Many parameters Few parameters NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 8. Neural networks are a popular machine learning model x1 = (A1x0 + b1) x2 = (A2x1 + b2) ... y = out(Anxn 1 + bn) Input #1 Input #2 Input #3 Input #4 Output #1 Output #2 Output #3 Output #4 Hidden layer Input layer Activation layer Output layer Activation function
- 9. Optimum Traditional parametrizations How too few parameters may hurt. NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 10. 1 0 A climate model time step Radiative Transfer Cloud Microphysics Cumulus convection And many more Dry Dynamics 4980 50405100 51605160 5220 5220 5220 5280 5280 5340 5400 5460 5520 5580 5640 5640 5700 5700 5700 5760 5760 5820 5880 5880 University of Wyoming −40 −35 −35 −35 −30 −30 −25 −20 −20 −20 −15 −15 −15−15 −15−15 −15 −15 −10 −10 −10 −10 −5 −5 −5 −5 −5 −5 −5 −5 University of Wyoming −20 543 4 03953 1 BIKF −37 522 2 BGBW −11 574 34 −29 544 11 32618 −34 533 16 PABR −30 530 5 PAOT −32 528 5 PAOM −36 516 13 PABE −32 522 5 PAMC −30 527 3 PAFA −36 518 17 PANC −36 516 25 PASN −38 522 9 PACD −39 517 5 PADQ −38 525 15 PAYA −29 542 8 PANT −26 550 32 PASY −27 533 0 YVQ −40 510 9 YUX −24 554 18 YZT −25 552 30 WSE −14 576 10 WSA −15 566 12 YQI −18 565 3 WMW −17 569 2 AYT −16 560 9 YZV −16 566 21 YJT −24 546 27 YYR −27 547 26 YAH −23 552 31 YMO−26 553 18 WPL −22 552 3 YQD −34 532 11 YVP −37 536 3 WPH −28 550 18 ZXS −42 506 8 YFB −29 544 7 YYQ −40 510 4 YRB −33 526 12 YCB −31 533 23 YBK −25 536 2 YSM −26 541 21 YYE −14 498 10 YEV −36 527 9 YXY −11 584 33 KEY −13 583 26 MFL −13 582 29 JAX −14 580 27 CHS −12 584 25 TBW −11 583 30 TLH −13 582 30 FFC −14 583 22 BMX −11 585 24 LIX −13 584 9 JAN −12 584 2 LCH −12 582 3 SHV −13 579 2 FWD −10 587 4 BRO −10 585 5 CRP −11 580 24 DRT −16 573 19 MAF −20 568 18 TUS −15 572 18 NKX −15 576 15 MHX −15 578 9 GSO −16 577 17 RNK −16 580 4 BNA −13 580 3 LZK −15 574 2 OUN −14 572 2 AMA −17 571 29 EPZ −17 570 3 ABQ −19 571 4 FGZ −17 573 10 VEF−11 580 14 VBG −17 573 10 WAL −16 574 16 IAD−17 575 5 ILN −14 576 3 SGF −14 573 3 DDC −14 574 1 TOP −16 572 8 DNR −17 573 7 GJT −12 582 11 REV −11 584 17 OAK −17 567 5 OKX −17 567 5 ALB−17 570 4 BUF −16 572 30 OAX −16 572 17 LBF −15 574 45 SLC −12 578 13 LKN −14 581 3 MFR −16 566 11 DTX −15 561 1 APX−17 563 14 GRB −18 567 20 MPX −17 569 21 ABR −16 571 45 RAP−15 573 42 RIW −14 577 3 BOI −15 574 2 SLE −16 566 16 CAR −21 560 16 INL −16 569 18 BIS −14 567 4 GGW −15 570 2 TFX −16 569 2 OTX −16 567 15 UIL −18 559 15 CWVK −19 569 12 1Y7 −20 561 3 GYX −15 568 10 DVN −19 559 5 CHH −15 573 19 ILX −12 574 2 LMN −9 586 9 ADN −9 584 5 76458 −9 587 13 76526 −7 588 17 76612 −7 589 16 76679 −13 578 13 TXKF −13 582 25 MYNN −7 586 0 MWCR −8 584 23 MKJP −9 586 8 MDSD −9 584 23 TJSJ −6 587 17 MZBZ −4 587 34 MROC −5 584 32 TFFR −4 588 39 TTPP −6 586 8 TNCC −3 588 35 SKSP −4 589 36 SKBQ −7 588 0 SKBG −5 588 −6 589 2 SBBV −8 590 35 PHLI −7 589 22 PHTO 12Z 23 Apr 2019 500 hPa
- 11. Near-global aqua-planet (NG-Aqua) simulation generated by the System for Atmospheric Modeling (Δ" = 4km) 11 https://vimeo.com/299128849
- 12. Coarse-grain data to 160 km boxes 12 Training regionTesting region Coarse-graining A B C NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 13. 13 Parameterization as function approximation !(#; %) Yanai, et. al. (1973) 64 inputs + SST, SOLIN 68 outputs
- 14. Training Approach 1 1. Use finite differences to compute residual tendencies 2. Train neural network : Q2 ⇡ ¯qv(t + 3 h) ¯qv(t) 3 h gLS(t), gLS(t) = ¯v · r¯qv q, s, SHF, LHF, TOA Q1, Q2 Neural Network (1 layer of 256 nodes ≅ 70,000 parameters)
- 15. Neural networks can diagnose Q1 and Q2 NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 15 Neural Network ~ 100,000 parameters Millions of samples Heating (K/day) (finite diff.) !" ≈ .50 Similar to Krasnopolsky et. al. (2013)
- 16. But its coupled! NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 1 6 Neural Network Dry Dynamics
- 17. Test with single column model 1 7 Neural Network Single column model
- 18. Single Column Dynamics x = [qv, s] y = [SHF,LHF, TOASW,#] dx dt = f(x, y(t)) + gLS(t) Prognostic variables Auxiliary variables
- 19. Uh oh…temperature = 1035 K after 1 day Time (d) p (hPa)
- 20. Is fitting Q1 and Q2 the right approach? Assumes that model dynamics are continuous in time • But they are not (Donahue and Caldwell, 2018) Assumes moist physics tendencies are available and accurate • Not true for DYAMOND outputs • Not true for observations Does not ensure good predictions over many time steps
- 21. Fitting the approximate Q1 and Q2 is equivalent to minimizing one-step error 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 50 100 150 200 Error Truth Prediction Hours
- 22. …but that does not ensure longer term performance 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 50 100 150 200 Error Truth Prediction Hours
- 23. Stable and accurate single column model simulations NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 23 Community Atmosphere Model Version 5 (CAM5) Single Column Mode (default physics, no chemistry) Humidity Anomaly (from true zonal mean) (g/kg) days Brenowitz and Bretherton (2018)
- 24. Will it work in a GCM? NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 2 4 Neural Network Dry Dynamical Core Brenowitz and Bretherton (2019)
- 25. Coupled simulations with this model blow-up! 25 • 160 km resolution • System for Atmos. Modeling (SAM) • Fortran calling python 0 5 10 15 20 x (1000 km ) 0.0 2.5 5.0 7.5 10.0 y(1000km) 0 15 30 45 60 50 75 100125 NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 26. The neural network finds spurious correlations! NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 26 Humidity at 200 mb and precipitation
- 27. Synchronization is hard for machine learning NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 27 Pendulums Synchronize People clapping Distinct sub-systems (clappers, pendulums) are perfectly correlated Timme, Max Planck Institute for Dynamics and Self-organization Domain Knowledge Domain knowledge
- 28. Ignoring upper atmospheric humidity stabilizes coupled simulation NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 28 Precipitable water after 5 days Brenowitz and Bretherton, 2019
- 29. Improved coupled forecast accuracy (RMSE of humidity) NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 30. Longer-term drifts in climate 30 NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 31. Biases in circulation 31 Zonal+time mean over days 105-110
- 32. Peering into The Black Box 32
- 33. Lower Tropospheric Stability (LTS) • T(600 mb) – T(surface) • Strength of inversion Mid tropospheric humidity • ∫"## $% &' ()/+ • Strongly controls amount of precipitation Systematically Vary Inputs NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 33
- 34. 34 Insert conditionally averaged input variables into the neural network NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 35. Increasing LTS increases the depth of convection NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 35
- 36. Increasing moisture strengthens convection NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 36
- 37. 37 Linearize the Neural Network NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
- 38. Linearized Response Function 38 NOAH D. BRENOWITZ - NOAHB@VULCAN.COM !(#$, #&) !((, )) Similar to Kuang (2018) Sensitivity of SGS moistening at 600 mb to 1 g/kg of additional moisture at 750 mb.
- 39. Linearized Response Function 39 NOAH D. BRENOWITZ - NOAHB@VULCAN.COM !(#$, #&) !((, )) Similar to Kuang (2018)
- 40. The neural network finds spurious correlations! NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 40 Humidity at 200 mb and precipitation
- 41. Couple Linearized Response Functions to Gravity Waves !′# + ̅!&'( = * + , -./ -! !((1) + -./ -3 3( 1 41 5 3′# + 63&'′ = * + , -.7 -! !((1) + -.7 -3 3( 1 41 5 '# ( = − - -9 7 :;/<( − 4'( :'′ = = =& / >? = =& @+'′ and <( = A B( ̅B NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 41Similar to Kuang (2018)
- 42. Eigenmodes of this Linear System NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 42 With upper atmospheric input (unstable in GCM) Without upper atmospheric input (stable in GCM) Phase speed (m/s) Phase speed (m/s) Growthrate(1/day) Wavenumber (1/km) Wavenumber (1/km)
- 43. A scary eigenmode (w/ unstable NN) NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 43
- 44. A moisture-mode-like standing instability (w/ stable NN) NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 44
- 45. Conclusions and Future Directions • Coupling with GCM makes ML parameterization hard • Synchronization due to time-scale separation pollutes our training data • Ignoring synchronized inputs stabilizes spatially extended simulations • Accurate for ~2 days for precipitation, ~5 days for humidity and temperature • Longer term drifts in the climate • The NN is physically plausible: • Increase humidity controls the strength of the predicted precipitation • Increasing tropospheric stability controls the height of the predictions • Linearized Response Functions can be coupled to waves • Plot wave-speed/growth rate curves and wave structures. NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 45
- 46. References Brenowitz, N. D., & Bretherton, C. S. Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse-graining. JAMES (2019). Brenowitz, N. D. & Bretherton, C. S. Prognostic Validation of a Neural Network Unified Physics Parameterization. Geophys. Res. Lett. 17, 2493 (2018). S. Rasp, M. S. Pritchard, P. Gentine, Deep learning to represent subgrid processes in climate models. Proc. Natl. Acad. Sci. U. S. A. 115, 9684–9689 (2018). NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 46
- 47. ©2018 Vulcan Inc. All rights reserved. The information herein is for informational purposes only and represents the current view of Vulcan Inc. as of the date of this presentation. VULCAN INC MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. 47
- 48. Conditionally Averaged-data (|Lat| < 23 deg) NOAH D. BRENOWITZ - NOAHB@VULCAN.COM 48 Total Population Precipitation – Evaporation Net heating