Data pipeline for NOAA wind data

1. Windpipe
Sophie Diehl
Insight Data Engineering Fellow

2. What is the wind speed
at turbine height –
right now,
and in the past?

3. Dataset
● Wind speed data: NOAA GFS 4
● Global, every 0.5 degrees latitude and longitude
● Forecasts for coming week in 3-hour intervals
● Released daily at midnight

4. Queries
● What is the forecast today?
● Past averages: single days, months

6. HBase Storage
rowkey day1 day2 day3 day4 ...
lat-long1 3.56 2.63 2.56 3.91 ...
lat-long2 3.48 3.29 3.11 4.54
vs.
rowkey value
day1_lat-long1 3.56
day1_lat-long2 3.48
day2_lat-long1 2.63
day2_lat-long2 3.29
... ...
...
Page load for continental USA: 25s vs. 11s

7. Sophie Diehl
MIT B.S. 2014
Electrical Engineering