B E G I N N E R S G U I D E T O
W E AT H E R A N D C L I M AT E D ATA
D R M A R G R I E T G R O E N E N D I J K
I B M WAT S O N D ATA P L AT F O R M
@ M A R G R I E T G R
2 9 A P R I L 2 0 1 7 - D ATA S C I E N C E F E S T I VA L - L O N D O N
B E G I N N E R S G U I D E T O
W E AT H E R A N D C L I M AT E D ATA
S L I D E S
H T T P S : / / W W W. S L I D E S H A R E . N E T /
M A R G R I E T G R O E N E N D I J K / P R E S E N TAT I O N S
W E AT H E R
F O R E C A S T
W E AT H E R
F O R E C A S T I N A
N O T E B O O K
D E M O
T H E W E AT H E R
C O M PA N Y A P I
D ATA
Get access here:
https://console.ng.bluemix.net/
W E AT H E R
F O R E C A S T I N A
N O T E B O O K
Try it out here:
datascience.ibm.com
Notebook:
https://github.com/MargrietGroenendijk/Data-Science-Festival
P I X I E D U S T
O P E N S O U R C E
https://ibm-cds-labs.github.io/pixiedust/
!pip install --upgrade pixiedust
O B S E R VAT I O N S + M O D E L S
O B S E R VAT I O N S
• Temperature
• Humidity
• Windspeed and direction
• Air pressure
• Rainfall
• Radiation
http://www.metoffice.gov.uk/public/
weather/climate-network/#?
tab=climateNetwork
H I S T O R I C W E AT H E R
• http://
www.metoffice.gov.uk/
datapoint/
• https://
business.weather.com/
products/the-weather-
company-data-packages
• https://climexp.knmi.nl
• http://www.ecmwf.int/en/
forecasts/datasets
I WA N T A M A P…
B U T
P O I N T S T O
G R I D
T H E P R O B L E M
from scipy.interpolate import griddata
# grid of latitude and longitude values
x = np.linspace(49.0,59.0,100)
y = np.linspace(-6,2,100)
X, Y = np.meshgrid(x,y)
px = points['lat'].as_matrix()
py = points['lon'].as_matrix()
pz = points['temp'].as_matrix()
fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(18, 8))
for i, method in enumerate(('nearest', 'linear', 'cubic')):
Ti = griddata((px, py), pz, (X, Y), method=method)
ax[i].contourf(X, Y, Ti)
ax[i].set_title('method = {}'.format(method))
ax[i].scatter(px, py, c='k', marker='o')
H O W C A N I S T O R E T H I S
D ATA ?
C O O L , B U T …
N E T C D F
B I N A RY F I L E S
W H AT A B O U T F O R E C A S T S
A N D P R E D I C T I O N S ?
T H I S D ATA I S A L L B A S E D O N M E A S U R E M E N T S …
C L I M AT E
M O D E L S
C L I M AT E W I T H D I F F E R E N T S C E N A R I O S
M O D E L E X P E R I M E N T S
G L O B A L
T E M P E R AT U R E
E X P L A I N E D
https://www.bloomberg.com/graphics/2015-whats-warming-the-world/
W H AT C A N I U S E
W E AT H E R D ATA F O R ?
in	vehicle	hail	damage
claims	every	year
increase	in	temperature
means	$24M	more	in
electricity	spending	per	day
drop	in	sales	for	areas
with	more	than	a	10%	drop
in	temperature
I N S U R A N C E
E N E R G Y
R E TA I L
A P P L I C AT I O N S
W E AT H E R A N D
M A C H I N E
L E A R N I N G
https://www.pexels.com/photo/buildings-silhouette-rain-window-27472/
W E AT H E R A N D
M A C H I N E
L E A R N I N G
https://www.pexels.com/photo/green-palm-tree-near-body-of-
water-149671/
W E AT H E R A N D
T R A F F I C
C O L L I S I O N S
E X A M P L E
https://www.pexels.com/photo/blur-cars-dew-drops-125510/
N Y P D T R A F F I C
C O L L I S I O N S
E X A M P L E
https://data.cityofnewyork.us/
Public-Safety/NYPD-Motor-
Vehicle-Collisions/h9gi-nx95
8 1 2 , 5 2 6
T R A F F I C
C O L L I S I O N S
S I N C E A P R I L 2 0 1 4
N Y P D T R A F F I C C O L L I S I O N S
https://apsportal.ibm.com/exchange/public/entry/view/5a7051906b8fe9cc1ba126b53edd948e
T E M P E R AT U R E F O R T H E 5 B O R O U G H S
T E M P E R AT U R E F O R T H E 5 B O R O U G H S
S U M M A RY
• Weather data
• json
• timeseries
• maps
• netcdf
• models
• Example of weather and
traffic collisions
R E F E R E N C E S
• IBM Bluemix - https://console.ng.bluemix.net/
• IBM Data Science Experience - datascience.ibm.com
• PixieDust - https://ibm-cds-labs.github.io/pixiedust/
• Slides - https://www.slideshare.net/MargrietGroenendijk/presentations
• Notebooks - https://github.com/MargrietGroenendijk/Data-Science-
Festival
• Me - mgroenen@uk.ibm.com - @MargrietGr
Hackathon
June 10-11, 2017
Galvanize, San Francisco
Code Challenge
June/July 2017
The SETI Institute
Register Now!
seti.org/ML4SETI

Data Science Festival - Beginners Guide to Weather and Climate Data

  • 1.
    B E GI N N E R S G U I D E T O W E AT H E R A N D C L I M AT E D ATA D R M A R G R I E T G R O E N E N D I J K I B M WAT S O N D ATA P L AT F O R M @ M A R G R I E T G R 2 9 A P R I L 2 0 1 7 - D ATA S C I E N C E F E S T I VA L - L O N D O N
  • 2.
    B E GI N N E R S G U I D E T O W E AT H E R A N D C L I M AT E D ATA S L I D E S H T T P S : / / W W W. S L I D E S H A R E . N E T / M A R G R I E T G R O E N E N D I J K / P R E S E N TAT I O N S
  • 3.
    W E ATH E R F O R E C A S T
  • 4.
    W E ATH E R F O R E C A S T I N A N O T E B O O K D E M O
  • 5.
    T H EW E AT H E R C O M PA N Y A P I D ATA Get access here: https://console.ng.bluemix.net/
  • 6.
    W E ATH E R F O R E C A S T I N A N O T E B O O K Try it out here: datascience.ibm.com Notebook: https://github.com/MargrietGroenendijk/Data-Science-Festival
  • 7.
    P I XI E D U S T O P E N S O U R C E https://ibm-cds-labs.github.io/pixiedust/ !pip install --upgrade pixiedust
  • 8.
    O B SE R VAT I O N S + M O D E L S
  • 9.
    O B SE R VAT I O N S • Temperature • Humidity • Windspeed and direction • Air pressure • Rainfall • Radiation http://www.metoffice.gov.uk/public/ weather/climate-network/#? tab=climateNetwork
  • 10.
    H I ST O R I C W E AT H E R • http:// www.metoffice.gov.uk/ datapoint/ • https:// business.weather.com/ products/the-weather- company-data-packages • https://climexp.knmi.nl • http://www.ecmwf.int/en/ forecasts/datasets
  • 11.
    I WA NT A M A P… B U T
  • 12.
    P O IN T S T O G R I D T H E P R O B L E M
  • 13.
    from scipy.interpolate importgriddata # grid of latitude and longitude values x = np.linspace(49.0,59.0,100) y = np.linspace(-6,2,100) X, Y = np.meshgrid(x,y) px = points['lat'].as_matrix() py = points['lon'].as_matrix() pz = points['temp'].as_matrix() fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(18, 8)) for i, method in enumerate(('nearest', 'linear', 'cubic')): Ti = griddata((px, py), pz, (X, Y), method=method) ax[i].contourf(X, Y, Ti) ax[i].set_title('method = {}'.format(method)) ax[i].scatter(px, py, c='k', marker='o')
  • 15.
    H O WC A N I S T O R E T H I S D ATA ? C O O L , B U T …
  • 16.
    N E TC D F B I N A RY F I L E S
  • 18.
    W H ATA B O U T F O R E C A S T S A N D P R E D I C T I O N S ? T H I S D ATA I S A L L B A S E D O N M E A S U R E M E N T S …
  • 19.
    C L IM AT E M O D E L S
  • 20.
    C L IM AT E W I T H D I F F E R E N T S C E N A R I O S M O D E L E X P E R I M E N T S
  • 21.
    G L OB A L T E M P E R AT U R E E X P L A I N E D https://www.bloomberg.com/graphics/2015-whats-warming-the-world/
  • 24.
    W H ATC A N I U S E W E AT H E R D ATA F O R ?
  • 25.
  • 26.
    W E ATH E R A N D M A C H I N E L E A R N I N G https://www.pexels.com/photo/buildings-silhouette-rain-window-27472/
  • 27.
    W E ATH E R A N D M A C H I N E L E A R N I N G https://www.pexels.com/photo/green-palm-tree-near-body-of- water-149671/
  • 28.
    W E ATH E R A N D T R A F F I C C O L L I S I O N S E X A M P L E https://www.pexels.com/photo/blur-cars-dew-drops-125510/
  • 29.
    N Y PD T R A F F I C C O L L I S I O N S E X A M P L E https://data.cityofnewyork.us/ Public-Safety/NYPD-Motor- Vehicle-Collisions/h9gi-nx95
  • 30.
    8 1 2, 5 2 6 T R A F F I C C O L L I S I O N S S I N C E A P R I L 2 0 1 4
  • 31.
    N Y PD T R A F F I C C O L L I S I O N S https://apsportal.ibm.com/exchange/public/entry/view/5a7051906b8fe9cc1ba126b53edd948e
  • 32.
    T E MP E R AT U R E F O R T H E 5 B O R O U G H S
  • 33.
    T E MP E R AT U R E F O R T H E 5 B O R O U G H S
  • 34.
    S U MM A RY • Weather data • json • timeseries • maps • netcdf • models • Example of weather and traffic collisions
  • 35.
    R E FE R E N C E S • IBM Bluemix - https://console.ng.bluemix.net/ • IBM Data Science Experience - datascience.ibm.com • PixieDust - https://ibm-cds-labs.github.io/pixiedust/ • Slides - https://www.slideshare.net/MargrietGroenendijk/presentations • Notebooks - https://github.com/MargrietGroenendijk/Data-Science- Festival • Me - mgroenen@uk.ibm.com - @MargrietGr
  • 36.
    Hackathon June 10-11, 2017 Galvanize,San Francisco Code Challenge June/July 2017 The SETI Institute Register Now! seti.org/ML4SETI