Chris Tubbs: Quantifying uncertainty in an operational environment

1,737 views

Published on

Risk SIG conference: 24th October 2013

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,737
On SlideShare
0
From Embeds
0
Number of Embeds
1,032
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • <number>
  • <number>
    Our current suite of operational NWP models are shown here along with future configurations. The numbers refer to typical horizontal grid spacings in mid-latitudes. This grid spacing is often referred to as resolution.
  • Results for the Carlisle flood are similar, despite the rather different mechanisms involved.
    Since the radar has to be corrected statistically over high ground, it is not entirely surprising that the model can provide forecasts of better accuracy than the radar “observation”
  • When I joined the Met Office we had a computer with just half a megabyte of memory, which was the fastest computer in Europe. We could not afford to run a global model, but were limited to the northern extra-tropics, with a spacing between the computation locations of 300km. Now we have a global model with a 40km grid spacing and a UK model with a 4km spacing. Instead of describing the atmosphere in ten layers, we now use fifty. And instead of forecasting up to 3 days ahead, we forecast up to 6 days ahead. In addition we receive forecasts from the shared European Weather Centre which go up to 10 days ahead.
  • <number>
    Data assimilation is key to producing a good forecast. The old maxim “Garbage in - garbage out” has never been truer than for a numerical model of an essentially chaotic system - the atmosphere.
    One of the newest forms of data assimilation, which is being adopted by a number of major forecast centres around the world, is 4 dimensional variational assimilation. It’s a quite complex and very computationally demanding technique, but it provides the best way of getting an accurate starting point to run a forecast from. I’ll try and explain the basic concepts, and leave out the complexities, which I don’t understand, and would take up a whole lecture series in themselves!
    We start with a background field - which has been generated from a previous run of the forecast. We run the forecast from the place for 6 hours. We then measure the difference between the value of the forecast fields and the observations we have. Then, the clever bit (this is 4D VAR) is to change the background field in such a way that the difference between the observations and the forecast are reduced. So, using the modified background field, we run forwards 6 hours again. And repeat this process, until we are satisfied that the difference between observations and forecast have been reduced to a satisfactory level. Only then, can we run the full forecast forwards to the required time.
  • In many ways this graph summarises much of what the development of computers has enabled numerical weather prediction to achieve. The improved performance has been achieved through the implementation of research in the areas of finer resolution, larger domains, longer forecasts, better use of observations, better representation of atmospheric processes, and so on… Increasing computer speed and memory has enabled that research to be implemented.
    In 1954, it was written in the anniversary issue of Met Mag that there had been little improvement in the manually produced 24 hour forecast over the previous 10 years, and that there was little skill at longer ranges. We may take it that, for this measure, the 1967 NWP score was similar for T+24. This level of skill was extended to 2 days by NWP in 1985 and to 3 days in 1996.
  • Poor mans ensemble…
    Synoptic elements may include (not exhaustive!): Strong jet, confluent upper pattern, strong surface baroclinicity, high WBPTs etc.
  • For forecasting beyond a day or two ahead, there is still significant uncertainty in the result. However, research has shown that the uncertainty varies, and that we can predict it. So we actually run many slightly different forecasts and then use the differences in the results to estimate how significant the uncertainties are.
  • Poor mans ensemble…
    Synoptic elements may include (not exhaustive!): Strong jet, confluent upper pattern, strong surface baroclinicity, high WBPTs etc.
  • Poor mans ensemble…
    Synoptic elements may include (not exhaustive!): Strong jet, confluent upper pattern, strong surface baroclinicity, high WBPTs etc.
  • Poor mans ensemble…
    Synoptic elements may include (not exhaustive!): Strong jet, confluent upper pattern, strong surface baroclinicity, high WBPTs etc.
  • Poor mans ensemble…
    Synoptic elements may include (not exhaustive!): Strong jet, confluent upper pattern, strong surface baroclinicity, high WBPTs etc.
  • Poor mans ensemble…
    Synoptic elements may include (not exhaustive!): Strong jet, confluent upper pattern, strong surface baroclinicity, high WBPTs etc.
  • Chris Tubbs: Quantifying uncertainty in an operational environment

    1. 1. Quantifying Uncertainty in an Operational Environment – Risky weather forecasts Chris Tubbs Association of Project Management © Crown copyright Met Thursday 24th October 2013
    2. 2. Talk structure • Introduction • Compiling a forecast – 2-3 days • Introducing uncertainty – 4-5 days • Uncertain forecasts; 6-15 day, 16-30 day (monthly) and seasonal forecasting • Converting uncertainty into risks for customers • Looking to the future and questions © Crown copyright Met
    3. 3. Compiling a forecastForecasting Process Observations 4-D winds, rainfall, temperatures……. © Crown copyright Met
    4. 4. Creating weather services 80km high 70 levels 25km Observations du = ∂p – fv dt ∂x dv = ∂p + fu dt ∂y p = RT ρ Knowledge © Crown copyright Met Forecast Model Interpretation, Risk Analysis & Communication
    5. 5. Numerical modelling Weather and Climate Models are huge computer codes based on fundamental mathematical equations of motion, thermodynamics and radiative transfer These govern:  Flow of air and water - winds in the atmosphere, currents in the ocean.  Exchange of heat between the atmosphere and the earth’s surface / ocean  Release of latent heat by condensation during the formation of clouds and raindrops  Absorption of sunshine and emission of thermal (infra-red) radiation Numerical methods must conserve mass, energy, momentum, water and tracers © Crown copyright Met
    6. 6. Operational Forecasting Models: October 2013 Global 25km 70L 2.5 day forecast twice/day 6 day forecast twice/day +24 member EPS at 60km twice/day N.Atlantic/European (NAE) bec Euro4 12km bec 4km 70L 2.5 bec 5 day f’cast 4 times per day +24 member EPS at 18km twice/day UK-V 1.5km 70L 1.5 day forecast 8 times per day 2012: +24 member EPS at 2.2km Met Met © Crown copyright Office Global Regional Ensemble Prediction System = MOGREPS
    7. 7. Resolution: Observed & Forecast Accumulations for the Carlisle Flood Hand analysis of gauges and radar 12 km 12 km 1 km 4 km Model Orography © Crown copyright Met 1 km
    8. 8. Resolution: fog prediction Visibility (m) © Crown copyright Met
    9. 9. The growth of computer power IBM P7 10T 1T 100G ETA 10 10G 1G CYBER 205 100M 10M IBM 360/195 1M 100K KDF 9 10K MERCURY 1K 100 LEO 1 10 © Crown copyright Met IBM P6 NEC SX-8 NEC SX-6 CRAY T3E CRAY C90 CRAY YMP8
    10. 10. Data Assimilation First guess Observations T-3 T-2 T-1 T+0 T+1 T+2 T+3 T+144 • The challenge: • To compute the model state from which the resulting forecast best matches the available observations © Crown copyright Met
    11. 11. Global performance by lead time RMS surface pressure error over the NE Atlantic 4-day f/c 1-day f/c 1982 © Crown copyright Met 2012© Crown copyright Met Office
    12. 12. 1-2 days ahead • Additional benefits from our high resolution (1km and 4km) models • Automated warning products from MOGREPS ensemble system • Weather system can be monitored before it reaches UK © Crown copyright Met
    13. 13. 3 to 5 days ahead • Deterministic global model outputs and ensembles And forecaster interpretation…..’added value’ © Crown copyright Met
    14. 14. 3 to 5 days ahead… Within the current NSWWS timescales • So we have all of this weather information to enable us to give a ‘most likely’ scenario and potential what if’s • Increased confidence • Higher risk can be identified • Details will still be subject to change and should be treated as best estimates • Worth realising that the weather system causing the event will often not even have formed yet! © Crown copyright Met
    15. 15. 3 to 5 days ahead…what action can be taken? • We present these pieces of the jigsaw to other interested parties and partners • Whole ethos of Hazard Centre • NSWWS is impact based • Typical example of Heavy Rain • Flood Forecast Centre (FFC) can use rain forecast and add more info in terms of assessing an impact • Construct a communication plan with key messaging © Crown copyright Met
    16. 16. Quantifying uncertainty with ensembles Deterministic Forecast Forecast uncertainty Initial Condition Uncertainty X CHAOS Analysis Climatology time © Crown copyright Met
    17. 17. Probability of precipitation >5mm in 12 hours © Crown copyright Met
    18. 18. Pictorial representation of uncertainty © Crown copyright Met
    19. 19. Spaghetti plot of fronts © Crown copyright Met
    20. 20. Dalmatian plot of lows and depths © Crown copyright Met
    21. 21. 6 to 15 days ahead Can we detect anything at all? • Ensemble outputs are main source of information There is often a lot of uncertainty © Crown copyright Met
    22. 22. Increased reach - 6 to 15 days ahead • Forecasters are good at: • - linking atmospheric patterns to potential severe weather • - identifying trends (e.g. mobile to block, cold to warm) Sometimes there are hints of the following: Prolonged rain Windy conditions Snow Prolonged heat or cold © Crown copyright Met
    23. 23. Meteogram for 15 days, cloud, precipitation, winds (speed and direction) and temperatures © Crown copyright Met
    24. 24. MSLP ensemble mean every 12 hours to T+240 (10 days) © Crown copyright Met
    25. 25. Shannon entropy, a measure of spread © Crown copyright Met
    26. 26. 6 to 15 days ahead…what action can be taken? • We wouldn’t be able to supply details for any particular region this far out BUT  Heads up information can be useful to open up discussions (e.g. PWS-Advisors, Internal Comms)  These things can be couched in terms of risk, albeit low  Awareness © Crown copyright Met
    27. 27. Monthly and seasonal forecasts • Knowing 6-15 day trends, especially if there is high confidence, helps to extend to monthly • Sometimes ideas around continuation of a spell/climatology can help eg anticyclonic October = warm start, cold end • Seasonal forecasts rely on Global drivers, eg North Atlantic Oscillation (+ve = mild), El Nino, SST anomalies, Arctic Sea Ice, Tropical Storms • Most only have skill in winter half of year in Europe. Better in Tropics. © Crown copyright Met
    28. 28. Our key role as a forecaster is… Identifying the potential for high impact weather…. • 6 to 15 day lead time • 3 to 5 day lead time 0 to 2 day lead time as we approach the ‘event’ © Crown copyright Met
    29. 29. Converting uncertainty into risks for customers © Crown copyright Met
    30. 30. Weather Impact Matrix © Crown copyright Met
    31. 31. 6 to 15 days ahead…what action can be taken? • We wouldn’t be able to supply details for any particular region this far out BUT  Heads up information can be useful to open up discussions (e.g. PWS-Advisors, Internal Comms)  These things can be couched in terms of risk, albeit low  Awareness © Crown copyright Met
    32. 32. 3 to 5 days ahead… Within the current NSWWS timescales • So we have all of this weather information to enable us to give a ‘most likely’ scenario and potential what if’s • Increased confidence • Higher risk can be identified • Details will still be subject to change and should be treated as best estimates • Worth realising that the weather system causing the event will often not even have formed yet! © Crown copyright Met
    33. 33. 3 to 5 days ahead…what action can be taken? • We present these pieces of the jigsaw to other interested parties and partners • Whole ethos of Hazard Centre • NSWWS is impact based • Typical example of Heavy Rain • Flood Forecast Centre (FFC) can use rain forecast and add more info in terms of assessing an impact • Construct a communication plan with key messaging © Crown copyright Met
    34. 34. Covering risk due to lingering uncertainties © Crown copyright Met
    35. 35. Example of uncertainty to risks for Met Office customers • Open Road forecasts for Highways Agencies (HA) & local authorities since mid 1980’s to determine need for overnight winter road gritting • Tendency of customers and Met Office to be risk averse, why is that? • Main reason is 10 to one risk ratio, ie cost of service (£20k) is a tenth of average claim (£200k), and 100 to one ration for each gritting run (£2k) • Therefore whenever the risk of a frost occurring rises above 10% it is cost effective to grit • More useful to EA when deciding on flood prevention measures © Crown copyright Met
    36. 36. Looking to the Future • Better quantification of uncertainty – the UK 2.2km ensemble • Better nowcasts – an hourly UK NWP cycle • New and improved specialist models, eg Weymouth Bay 500m model • Better forecasts for global sites & longer ranges – a higher resolution global model • Longer forecasts – addition of monthly and seasonal forecast information © Crown copyright Met
    37. 37. 1200 Z Harbour Wall: 300 (245V332) 09 KT (backed 60 degrees between 1120 and 1200 Z) Buoy: 340 07 KT Isle of Portland: 270 07 KT © Crown copyright Met
    38. 38. 28/09Z UK300 VT 29/0900 Z Extreme gusts associated with line convection © Crown copyright Met
    39. 39. So to conclude…. • Improvements in science have given us opportunities to provide useful advice at longer lead times • We do need to manage expectations (because weather forecasting is not an exact science) • Communication is key (knowing how/when/what) and also the risks involved for our customers © Crown copyright Met
    40. 40. Any questions? © Crown copyright Met

    ×