The Finnish Meteorological Institute (FMI) opened its data in 2013, making basically all of its data with property rights publicly available in machine-readable formats. This includes near real-time and historical weather and climate data. The data is distributed through FMI's Open Data Portal, which follows INSPIRE requirements, as well as on Amazon Web Services (AWS) cloud platform through Amazon S3 buckets as part of a two-year pilot project to increase access and use of weather data. The AWS buckets contain HIRLAM surface and pressure level weather model data for Europe.

1. Open data distributed on Amazon’s cloud service
Roope Tervo, Mikko Visa
Finnish Meteorological Institute

2. • Finnish Meteorological Institute
opened its data in 2013.
• Basically everything that FMI has
property rights was opened.
• Both (near) real-time and historical
and climatological data.
• Data is provided in freely in machine
readable format.
9/12/2017 2
FMI Open Data
https://en.ilmatieteenlaitos.fi/open-data
Open data distributed on Amazon’s cloud service

3. FMI Open Data Portal follows INSPIRE requirements.
FMI Open Data
Data Portal
Meta data
Services
The very same data portal works as Open Data and
INSPIRE portal.
9/12/2017 4
ISO19115 WFS WMS
CSW
Grid Series
Observations
Time Series
Observations
Data
Models O&M
Simple
Feature
GRIB
NetCDF GeoTiff
Open data distributed on Amazon’s cloud service

4. FMI Open Data
Registration
o Registration is required to use View and Download
Services
o Working email address is the only mandatory
information
o After registration the user gets an API key which have to
be added into all requests
o GET parameter fmi-apikey=…&
o Header fmi-apikey; …
o Part of url http://wms.fmi.fi/fmi-apikey/…/wms?
o One can create several API keys with one email
9/12/2017 5Open data distributed on Amazon’s cloud service

5. FMI Open Data
Usage Limits
With one API key it’s allowed to
o do at most 20 000 requests per day to Download Service
o do at most 10 000 requests per day to View Service
o do at most 600 requests per 5 minutes to both services
o If all observations from one time step is calculated to as one,
little over 17 000 new data sets are published daily
o So, with one API key it’s allowed load everything once
o View service can be used for testing but can not be used as a
back end for popular clients
9/12/2017 6Open data distributed on Amazon’s cloud service

6. And a little over
830 000 data
downloads
per day
(9,6 req/s)
At the moment
about 11 700
registered users
Some Experiences
9/12/2017 7Open data distributed on Amazon’s cloud service

7. Motivation
Increase appropriability of
weather and climate data
• Data is valuable only when it
reach relevant audience
• Maximal coverage requires
several different channels
and services
 FMI joins Amazon’s Public
Datasets
9/12/2017
Open Source Software @ Finnish Meteorological
Institute | Roope Tervo, Harri Pietarila, Mikko
Rauhala
8

8. FMI OpenData on AWS
• FMI OpenData is also distributed on
Amazon Web Services (AWS) Cloud platform
• 2-years pilot
• Started in May 2017
• Hirlam surface and pressure levels in the first stage
• The objective is to
• increase the utility and effective use of weather and climate data
• support public-private-partnership
• Specially convenient for users who need the whole model data
• i.e for post-processing or generating map visualizations
• Licence: CC BY 4.0
9/12/2017 Open data distributed on Amazon’s cloud service 9

9. FMI OpenData on AWS
• Content
• Hirlam Surface
• Hirlam Pressure Levels
• Coverage: Europe
• Grid resolution: 7,5 km
• Updates: 4 times a day
• Time range: 54 hours (from model
run start)
• Time step: 1 hour
• Archive kept during the pilot
• Parameters:
http://en.ilmatieteenlaitos.fi/open-
data-on-aws-s3
9/12/2017 Open data distributed on Amazon’s cloud service 10

10. FMI OpenData on AWS
• Access through buckets:
• Surface data: fmi-opendata-rcrhirlam-surface-grib
• Pressure level data: fmi-opendata-rcrhirlam-pressure-grib
• Browse bucket content:
• http://fmi-opendata-rcrhirlam-surface-grib.s3-website-eu-west-
1.amazonaws.com/
• http://fmi-opendata-rcrhirlam-pressure-grib.s3-website-eu-west-
1.amazonaws.com/
• Public Amazon SNS topics are available for every new object added to
the Amazon S3:
• arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-surface-grib
• arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-pressure-
grib
9/12/2017 Open data distributed on Amazon’s cloud service 11

11. FMI OpenData on AWS
See documentation:
http://en.ilmatieteenlaitos.fi/open-data-on-aws-s3
9/12/2017 Open data distributed on Amazon’s cloud service 12

12. Requests by Data Format
in FMI Open Data Portal
9/12/2017 Open data distributed on Amazon’s cloud service 13
Requests and
ip-addresses in
July 2017

13. Hirlam Requests in binary
format by Channel
9/12/2017 Open data distributed on Amazon’s cloud service 14
Hirlam requests
in July 2017

14. Point requests are
still by far the most
popular type
For binary data
downloaders S3 is
already the most
popular channel
(in three months!)
Some Experiences
9/12/2017 16Open data distributed on Amazon’s cloud service

15. 9/12/2017 Open data distributed on Amazon’s cloud service 17
“Use cases of cloud computing and requirements towards data distribution within a private meteorological service provider”
https://meetingorganizer.copernicus.org/EMS2017/Orals/25520/210111

16. www.fmi.fi
https://github.com/fmidev
https://en.ilmatieteenlaitos.fi/open-data
http://en.ilmatieteenlaitos.fi/open-data-on-aws-s3
http://roopetervo.com
http://www.slideshare.net/tervo