Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20161028 strahlendorff fmi experience in openness

136 views

Published on

Presentation for University of Helsinki event
Openness in Education and Science: http://avointiede.fi/osaajakoulutuksen-ohjelma#avoimettoimintatavat

Published in: Science
  • Be the first to comment

20161028 strahlendorff fmi experience in openness

  1. 1. Openness in research Experiences at Finnish Meteorological Institute 28.10.2016 Finnish Meteorological Institute Mikko Strahlendorff Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  2. 2. Background for my experience Mostly employed by Finnish Meteorological Institute  Weatherforecasting 1995-1997  Public service development 1998--99  Commercial service development and head of it 2000-2003  Head of Informations system services 2004- 2007  Seconded Expert to European Commission Earth Observation program 2008-2010  Ministry of Transport and Communications Ministerial adviser 2011-2012  EU research policy and international organisation 2012-  Helping to build European Research Infrastructure Consortias  ICOS, EuroARGO, SIOS, COPAL  Executive commiittee member in Group on Earth Observation 28.10.2016 Openness experience at FMI – Mikko Strahlendorff 2
  3. 3. FOSS revolution and the internet • Researcher Linus Torvalds at Helsinki university releases Linux as Free and Open-Source Software • 25 years anniversary now and • 96% of all web-servers run Linux • 69% of all mobile devices run Android (developed from Linux) • Sharing ones work with a huge network of like-minded ensures high quality results faster than anything • Motivation comes from the common utility of the result 28.10.2016 Openness experience at FMI – Mikko Strahlendorff 3
  4. 4. • Openness requires extra efforts • Free and open doesn’t generate revenue • How to motivate financing? How to share expenses fairly? • Sharing efforts is making the list of contributors long and single persons are less visible • How to divide the fame? Solutions: • Application of the common to many challenges in many places • Analysis, understanding and education for sale • Fame to the whole system: • Data production is publishing! • Each use of data should be rewarded! • FOSS tools and internet technologies reduce costs Challenges with Openness – rewarding effort 28.10.2016 4Openness experience at FMI – Mikko Strahlendorff
  5. 5. Data is a great common motivation o Analysis needs data and it gets better with more data o Selling scientific data is rarely lucrative and data lying in databases unused is only expenses - no gain o Data is good to be collected in similar ways for similar applications – common interest naturally networks collectors o Quality control and technical solutions are perfect tasks for sharing o Data dissemination has costs that are usually not planned in research o Tracking usage is good for development and for finance motivation 28.10.2016 5
  6. 6. Finnish Meteorological Institute opened its data in 2013 after government decision for all the Finnish public sector. Basically everything that FMI has property rights on was opened. Data is provided freely in machine readable format over the internet. 28.10.2016 Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari 6 FMI Open Data https://en.ilmatieteenlaitos.fi/open-data
  7. 7. Data set Description Time Interval Estimated publish date Weather Observations Temperature, Wind, Humidity, Ground Temperature… 10 min Open, older data to be added Sun Radiation UV, Short and Long Term Radiation… 1 min Open Marine Observations Waves, Sea Temperature, Sea Level… 1 h Open Weather Radars Precipitation Rate, Precipitation Amount… 5 min Open, older data to be added Lightning Thunder Strikes in Finland 5 min Open Example Data Sets 28.10.2016 7Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  8. 8. Example of Data Sets Data set Description Time Interval Estimated publish date Real Time Observations Real Time Observations from specific location(s) AWS 2010 – Soundings 1959 – Flashes 1998 – Sea Level 1971 – Waves 2005 – Open older data will be added Climatological Observations Dayly and monthly temperature mean and extreme values from weather stations 1959 - Open Climatological Observations Monthly temperature and precipitation rate mean values interpolated to grid 1961 - Open Climatological Reference Climatological Reference. Temperature, humidity, pressure, precipitation amount and snow depth. Reference seasons: 1971-2000 1981- 2010 Open 28.10.2016 8Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  9. 9. Example Data Sets Data set Description Time Interval Estimated publish date Weather forecast model HIRLAM RCR Point forecasts and grid data Latest model run (4 times a day) 0…54 h Open Sea forecast models Sea level point forecasts, Wave (WAM) and current (HBM) as grid data Latest model run (4 times a day) 0...54 h Open Environmental Monitoring Facilities Weather observation stations, radars… 2015 Aviation Observations METAR 30 min open Ground & mast observations Special observations from ground and masts 2016 /Open 28.10.2016 9Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  10. 10. Example Data Sets Data set Description Time Interval Estimated publish date Air Quality Observations Air Quality Observations 1h open Silam Model Dispersion Model for Air Quality, Forest Fire and Pollen Latest model run (once a day) 0…96h 2015 HELMI Ice Model Ice forecast model Latest model run (4 times a day) 0...54 h open Soundings Temperature, Humidity, Pressure, Wind from ground to 25 km height 2 times a day open Road Weather Observations (LIVI) Road Weather Observations 10 min open 28.10.2016 10Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  11. 11. FMI Open Data Portal follows INSPIRE requirements. FMI Open Data Portal Meta data Data Models Services The very same data portal works as Open Data and INSPIRE portal. 28.10.2016 11Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  12. 12. Registration o Registration is required to use View and Download Services o Working email address is the only mandatory information o After registration the user gets an API key which have to be added into all requests o GET parameter fmi-apikey=…& o Header fmi-apikey; … o Part of url http://wms.fmi.fi/fmi-apikey/…/wms? o One can create several API keys with one email 28.10.2016 12Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  13. 13. Usage Limits With one API key it’s allowed to o do at most 20 000 requests per day to Download Service o do at most 10 000 requests per day to View Service o do at most 600 requests per 5 minutes to both services o If all observations from one time step is calculated to as one, little over 17 000 new data sets are published daily o So, with one API key it’s allowed to load everything once o View service can be used for testing but can not be used as a back end for popular clients 28.10.2016 13Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  14. 14. And a little over 450 000 data downloads per day (5.2 req/s) At the moment about 11 700 registered users 3 year experience 28.10.2016 14Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  15. 15. Industry is happy to use standardized services Amateur and freelancer coders would prefer simple JSON API Some Experiences 28.10.2016 15Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  16. 16. MetoLib o Open source JavaScript library produced by Finnish Meteorological Institute o Helps users to load and use the data o Supports multi point coverage data format o Python version is on the list Easy requests Cache Parse the data to as JSON 28.10.2016 16Meteorological and Aviation Weather Open Data implementation OGC standards Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari
  17. 17. SmartMet Server • High capacity & availability Data server for MetOcean data • Data is extracted and products generated always on-demand • INSPIRE Compliant • Operational use since 2008 • FMI client services (since 2008) • Finnish Meteorological Institute (FMI) Open Data Portal (since 2013) • Several input sources • GRIB-, NetCDF-, etc. files via internal data format querydata • PostGIS database (vectors) • SQL database (point observations) • Several output interfaces and formats • WMS, WFS 2.0 • JSON, XML, ASCII, HTML, SERIAL • GRIB1, GRIB2, NetCDF 10/28/2016 Open Meteorological Data with OGC and INSPIRE / Roope Tervo 17
  18. 18. SmartMet Server goes Open Source • Going to be published as Open Source in Q4/2016 • MIT license • https://github.com/fmidev 10/28/2016 Open Meteorological Data with OGC and INSPIRE / Roope Tervo 18
  19. 19. Open Science vision Science WIKIPEDIA o Data is collected in standard ways and published immediately o Papers are written collaboratively o Science is mostly about writing scripts/models to analyze data for answers to questions o Analysis scripts are published with the answers so readers can reproduce results and evaluate conclusions themselves o Donors get to ask questions and use the knowledge o Sharing the fame is visible thru linking all the scientists, from every data set and every model/script 28.10.2016 19Openness experience at FMI – Mikko Strahlendorff
  20. 20. • Governments: What data do we need for public interests? • How do private entities support data production? • Science interest • Commercial interest • Research funding should be two-fold: • longer time plans and grants for data production and modeling • short term grants for answers to societal questions for scripts and analysis • How does a new Science approach conquer the world like Linux? • Thanks and please comment! What do we need to do for this vision? 28.10.2016 20Openness experience at FMI – Mikko Strahlendorff
  21. 21. www.fmi.fi http://www.slideshare.net/MikkoStrahlendorff https://en.ilmatieteenlaitos.fi/open-data

×