SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
Assessing the quality of volunteered weather observations to provide high-resolution weather maps
Assessing the quality of volunteered weather observations to provide high-resolution weather maps
1.
Irene Garcia-Marti
Gerard van der Schrier
Jan-Willem Noteboom
7th November 2019
Assessing the quality
of volunteered
weather observations
to provide
high-resolution
weather maps
2.
› Weather observations are
crucial!
› Spatial sparsity is a
challenge for high-res
weather forecasts
› Increasing number of
weather-related citizen
science projects
– WOW, Wunderground,
Netatmo, Meteoclimatic
Motivation
6 november 2019
Koninklijk Nederlands Meteorologisch Instituut 2
3.
1st
September 2019: 1,400 million observations and 17K stations worldwide
4.
› 2015: KNMI partner of WOW
› Contributors: 400+ CWS
› Data NL+BE: 3.7M obs/month
› Devices: semi-professional
– Manufacturers: Davis, Oregon
scientific, Ventus, Alecto…
– Expected “reasonable” quality of
the observations
WOW-NL
4
5.
› Quality not only related to
device:
– Good with respect to what? What
variables are (not) properly
monitored?
– Local processes: radiation,
shadowing, siting
› Classical challenges of citizen
science data:
– Gaps in data
– Noisy observations
WOW-NL
5
6.
WOW-NL
6
› Quality not only related to
device:
– Good with respect to what? What
variables are (not) properly
monitored?
– Local processes: radiation,
shadowing, siting
› Classical challenges of citizen
science data:
– Discretization
– Scale of the phenomena
10.
6 november 2019
Koninklijk Nederlands Meteorologisch Instituut 10
11.
Quality control
11
› Narrowing down variables
– Not reinventing the wheel:
▪ Temperature:
• (Napoly, 2018) 🡪
• (Meier, 2017)
• (Lussana, Titan project)
– QC based on (Napoly, 2018)
▪ Feasible to implement on WOW-NL
▪ Levels have been compacted
12.
Each of the 11.6M observations go through this workflow and is labeled with a quality level
13.
Overview of the quality of WOW-NL
(Each square represents 10K observations)
M0: incorrect metadata
(not shown)
M1: insufficient Z-score
(presence outliers)
M2: insufficient day/mon
coverage
M3: insufficient (Pearson)
correlation
M4: OK
14.
› Setting up a baseline:
– Kriging (or GP)
▪ Ordinary: no {external drift, trend}
▪ Temporal resolution: hourly
Interpolation
14
Radiation
bias?
16.
› If good enough:
– Open the door for new research:
▪ Fine-grained interpolated layers
▪ Nowcasting / hi-res weather
▪ Gridded weather variables
▪ Study local phenomena
– Governance level:
▪ Lower cost for administration
▪ Better weather forecast for
underrepresented areas
▪ Bottom-up initiatives might work in
developing regions
Why the quality of citizen
science weather data is
important?
17
17.
› Met offices have enough
knowledge to develop QC’s
– Does it make sense?
– Collaboration: “the more, the merrier”
› Re-use pieces of software:
– Corrections: radiation, air pressure
– QC’s for other variables:
precipitation, wind
› Ideally: work together to
build a pipeline with mechanistic,
corrective, and statistical filters
Not reinventing
the wheel
18
18.
Challenges and
opportunities ahead
19
Big data problem!
We are here
Imperfect data, but volume is difficult to ignore
› Big Data problem
– Paradigm shift: code 🡪 data
– Adoption of cloud technologies
› Data analysis:
– Machine learning for QC: Outlier
detection, add external drifts
– Coupling volunteered data with
weather models
– Study local patterns: urban wind
dynamics, urban rainfall, effect of
SVF on outliers
– Assess other quality control
schemas
19.
Questions? ☺
Thanks!
6 november 2019
Koninklijk Nederlands Meteorologisch Instituut 20
0 likes
Be the first to like this
Views
Total views
67
On SlideShare
0
From Embeds
0
Number of Embeds
0
You have now unlocked unlimited access to 20M+ documents!
Unlimited Reading
Learn faster and smarter from top experts
Unlimited Downloading
Download to take your learnings offline and on the go
You also get free access to Scribd!
Instant access to millions of ebooks, audiobooks, magazines, podcasts and more.
Read and listen offline with any device.
Free access to premium services like Tuneln, Mubi and more.