Download to read offline
Uploaded byRoope Tervo
Possibilities of Open Source Code
The Finnish Meteorological Institute (FMI) has embraced open data and open-source software, transitioning from a closed ecosystem to one that supports collaboration and innovation in weather services. Since 2013, FMI has opened its data and is actively developing open-source software to enhance weather forecasting and emergency responses, aiming to support both national and international meteorological services. Various software tools like SmartMet are being utilized for data sharing, visualization, and operational activities, significantly improving forecasting capabilities across different regions.
More Related Content
Similar to Possibilities of Open Source Code
Possibilities of Open Source Code
- 1. Open Data andOpen Source Software @Finnish Meteorological Institute WMO EC SE, 11.5.2017 Harri Pietarila, Head of Expert Services Roope Tervo, Development Manager
- 2. Motivation • Ecosystem ischanging • NMHS will not control the whole value chain anymore • Large companies like IBM and Google are becoming more and more relevant players 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 2 Charge for data Charge for Service Closed ecosystem Open ecosystem
- 3. Motivation • A roleof NMHS is also changing • Challenges: • Ensure authoritative voice in warnings • More efficiency in development and operations • Ensure the impact of produced information 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 3
- 4. Finnish Meteorological Institute openedits data 5/12/2017 4 FMI Open Source Software https://en.ilmatieteenlaitos.fi/open-data FMI starts to open it’s software 2013 2016 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 5. • Finnish MeteorologicalInstitute opened its data in 2013. • Basically everything that FMI has property rights was opened. • Both (near) real-time and historical and climatological data. • Data is provided in freely in machine readable format. 5/12/2017 5 FMI Open Data https://en.ilmatieteenlaitos.fi/open-data Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 6. Data set DescriptionTime Interval Estimated publish date Weather Observations Temperature, Wind, Humidity, Ground Temperature… 10 min Open, older data to be added Weather forecast model HIRLAM RCR Point forecasts and grid data Latest model run (4 times a day) 0…54 h Open Aviation Observations METAR 30 min open Weather Radars Precipitation Rate, Precipitation Amount… 5 min Open, older data to be added Lightning Thunder Strikes in Finland 5 min Open Some Example Data Sets 5/12/2017 6 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Visa
- 7. Finnish Meteorological Institute openedits data 5/12/2017 7 FMI Open Source Software https://en.ilmatieteenlaitos.fi/open-data FMI starts to open it’s software 2013 2016 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 8. Motivation Boost Product Developmentof the Software • NMHSs are well connected concerning NWP development and GTS/WIS • Why not extend collaboration to upstream as well? 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 8 NWP Post- processing Interpretation Visualization Applic ations ApplicationsObservations Collaboration and OSS Proprietary software
- 9. Motivation Streamline collaboration withpartners • Open Source Software can boost the development • A very powerful way to support developing countries 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 9 NWP Post- processing Interpretation Visualization Applic ations ApplicationsObservations Collaboration and OSS Proprietary software
- 10. Motivation Increase appropriability of weatherand climate data • NMHSs are successful when weather and climate don’t cause unanticipated unwanted impact to society • Data is valuable only when it reach relevant audience 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 10
- 11. Motivation Increase appropriability of weatherand climate data • Maximal coverage requires several different channels and services • One organization can’t handle them all 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 11
- 12. Motivation Support Research • OpenSource Software empowers research as well • Easy and open methods to access and analyze the data • Methods are repeatable when anyone can access the tools 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 13
- 13. Motivation Public-private-partnership • Open SourceSoftware is a great tool to ease public-private partnership • OSS provides seamless access to weather and climate information from multiple NMHS providers 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 14
- 14. Motivation • Boost andstreamline collaboration between NMHSs • Streamline collaboration with partners • Support developing countries • Boost product development of the software • Support research activities • Increase appropriability of weather and climate data • Support companies 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 15
- 15. Potential Users • OtherNational Meteorological Services • International agencies (such as ECMWF, EUMETSAT…) • Other environmental agencies • Research Institutes • Environmental companies • Private developers 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 16
- 16. License • MIT licenseis recommended for all FMI OSS… • …but every license is considered separately. • For example, other members of the project and/or used third party projects may require some specific license. Commercial Use Distribution Modification Private Use License and Copyright Notice required 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 17
- 17. What • About 30different software under review • Open source is the baseline… • … but every software is considered separately • How easy the software is to isolate from production system? • Is the project vital? Is the software still developed and maintained? • Does the software potentially benefit someone? 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 18
- 18. What Examples • SmartMet Server(open) High-capacity and high-availability data and product server for MetOcean data. • SmartMet Workstation (2018) Tool for visualizing and editing meteorological data • SmartMet Alert (2019) Tool for creating and disseminating (supports CAP) weather warnings • HIMAN (open) Post-processing suite for meteorological data • PyTroll (open) Set of tools for the reading, interpretation, and writing of weather satellite data. • SILAM (open) Airquality and dispersion model 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 19
- 19. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 20 FMI Open Data Portal & FMI INSPIRE Data Services Backend for clients’ web services Integration to clients’ systemsBackend for mobile applications Backend for FMI Client Services Backend for FMI public pages • Basis of FMI product generation SmartMet Server
- 20. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 21 • High-performing high-availability C++ data and product server for all kind of MetOcean data • FMI installation responses to over 30 million requests per day • Over 99,95 % availability • Also used at ECMWF Copernicus Climate Data Store SmartMet Server
- 21. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 22 Web pages with warning maps and texts Integration to clients’ systems Automatic text messaging Mobile phone application • SmartMet Alert provides • A user interface for the forecasters to input warning information • Automatic warning texts • Several possible (automatic) dissemination channels • CAP output SmartMet Alert CAP TV RSS FeedsPrint media
- 22. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 23 • Used in operational warning production in Finland since February 2015 • Has been installed and trained in three Caribbean countries (Cuba, Antigua and Barbuda, Jamaica) • April-November 2016 has been installed and trained also in 6 Pacific countries: Samoa, Vanuatu, Fiji, Tonga, PNG and Solomon Islands • Is going to be installed and trained in Bahamas in 2017 SmartMet Alert
- 23. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 24 • Tool for visualizing and editing meteorological data • Support for all kind of meteorological data • NWP, Satellites, Weather Radars, Observations, Lightning Detection, Soundings… SmartMet Workstation
- 24. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 25 • Post-processing and grid manipulation suite • Runs automatic post-processing scripts like statistical correlations or derivative parameters • Support GRIB1, GRIB2, CSV and QueryData formats • Utilize GPU HIMAN
- 25. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 26 • Free and open source python modules for the reading, interpretation, and writing of weather satellite data. • Project started by SMHI (Sweden) and DMI (Denmark) • FMI is collaborating with many others • User base is global PyTROLL Adding Value for Satellite Users
- 26. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 27 • Operational Emergency and Air Quality Modelling System • a global-to-beta-meso-scale (down to 1 km resolution) dispersion model • applied for wide range of atmospheric composition, air quality, and emergency decision support applications, as well as for inverse dispersion problem solution • Applications for nuclear accidents, wildland fires and volcanic ash as well as for regulatory air quality assessments and forecasting SILAM
- 27. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 28 • Data format conversion tools • Product generation tools • Observation network software and quality control • TAC IWXXM converters • Radar composite and analyzing software • … Many Others Some Open --- Some Under Investigation
- 28. Policies and Guidelines •Source code and documentation in GitHub: https://github.com/fmidev 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 29
- 29. Examples 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 30
- 30. Project Ongoing Projectin Preparation Completed cooperation or consulting project in the country Scientific Collaboration International co-operation & expert services Pacific SIDS Caribbean SIDS Central Asia Andean region S-E Europe 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 31. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 32 SmartMet installed SmartMet – installations
- 32. • Used inoperational warning production in Finland since February 2015 • Has been installed and trained in three Caribbean and six Pacific countries: • Cuba • Antigua and Barbuda • Jamaica • Samoa • Papua New Guinea • Fiji • Tonga • Vanuatu • Solomon Islands SmartMet Alert installations 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 33
- 33. 5/12/2017 Open Source Software@ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 34 Andean region • SmartMet installations: Efficient data display and editing tool for the forecaster & automated weather production for web and mobile - Peru SENAMHI - Ecuador INAMHI - Bolivia SENAMHI - Columbia IDEAM • ICI projects with Peru Ecuador, Bolivia and Columbia • Regional project PRASDES
- 34. SmartMet products example SEMANHI-Bolivia http://www.senamhi.gob.bo/ NewAndroid APP SENAMHI Bolivia the alerts and forecast in Aymara language new color coded Alert protocol of SENAMHI 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 35
- 35. SmartMet in SIDSCaribbean • FMI cooperation in SIDS Caribbean from 2000 • Establishment of Caribbean wide SmartMet (forecaster workstation and forecast production system) network (10 countries) • Improvement in forecast quality • Improvement in forecast production and end-user products 5/12/2017 36 SmartMet installed Dominican Republic Cuba Jamaica Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 36. SmartMet Alert inSIDS Caribbean • Development of Early Warning services • Installation of SmartAlert software in Jamaica, Cuba and Antigua & Barbuda • Improvement in warning delivery and launching of modern end-user products such as maps etc… • Systems in pilot phase (goal: operational after COPS project) 5/12/2017 37 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 37. Google map visualizationworks also as a mobile phone app - Example from Cuba. SmartMet Alert Warning map visualization – google map 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 38
- 38. • FINPAC -Finnish-Pacific Project to Reduce Vulnerability of the Pacific Island Countries' Livelihoods to the Effects of Climate Change • Project covering the entire value chain from improved observations to services and benefits to communities • Funding of 3.8M€ to SPREP with Technical Assistance from FMI • Partnership with SOPAC, Red Cross, USP and WMO • Beneficiaries are the NMHSs from 14 Pacific Island Countries • Kick-off on 1 July 2013 at the Pacific Meteorology Council meeting in Fiji 5/12/2017 39 SmartMet in SIDS Pacific FINPAC (13-16) MFA FMI Project Coordinat or Project Manager Steering Committe e SPREP Manageme nt Team Financial reporting Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 39. 1.4: Implement weatherforecasting tools and production systems at selected NMSs • SmartMet workstation for editing and viewing meteorological data was installed in six countries (Fiji, PNG, Samoa, Solomon Islands, Tonga and Vanuatu). 5/12/2017 40 SmartMet in SIDS Pacific Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 40. • Examples: 5/12/2017 41 SmartMetin SIDS Pacific PNG met service web page Tonga met service web page Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 41. SmartAlert in SIDSPacific forecaster interface to issue warnings Forecaster view - example from Samoa 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 42
- 42. • Examples (googlemap visualization of the warnings): 5/12/2017 43 SmartMet in SIDS Pacific PNG met service web page Solomon Islands met service web page Samoa met service web page Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala
- 43. Social media integration– Facebook (Papua New Guinea Weather Service) • Forecast data produced with SmartMet • Facebook messenger inquiry to receive text format local weather forecast 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 44
- 44. Social media integration– Twitter (Tonga Meteorological Service) • Forecast data produced with SmartMet • Automatized tweeting of forecast information in text format 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 45
- 45. Training to ensuresustainability Tonga Vietnam Papua New Guinea Cuba Peru 5/12/2017 Open Source Software @ Finnish Meteorological Institute | Roope Tervo, Harri Pietarila, Mikko Rauhala 46
- 46.
Editor's Notes
- #3 - Large players are large enough to operate without NMSs
- #9 But there’s a long way from model output to end user information
- #10 Using Open Source Software prevents from vendor locks Collaboration is open and easy No long and burden negotiations – just evaluate, use, develop If you need any changes or modifications, you can do them yourself or order them from 3rd party Even just finding users to the software boost development
- #11 Maximal coverage requires several different channels and services One organization can’t handle them all
- #12 Open data helps, but.. Weather and climate data is complicated to handle Huge volumes, complex formats, complicated domain Providing tools to handle and analyze the data empowers 3rd party users to correctly utilize it Proper tools ensures consistency between information regardless of channel and service providers
- #15 - Even just finding users to the software boost development
- #21 High-performing high-availability C++ data and product server for all kind of MetOcean data FMI installation responses to over 30 million requests per day Over 99,95 % availability Used at ECMWF in Copernicus Climate Data Store
- #30 Small contributions with pull requests In larger contributions, implementation plan is recommended (in GitHub wiki) CLA (Contributor Licence Agreement) will be required