These were my opening remarks at the panel on Big Data and the NOAA CRADA held at the American Meteorological Society Annual Meeting in New Orleans in January 2016.

1. © 2015 The Climate Corporation All Rights Reserved
Big Data, Cloud and the NOAA
CRADA at the Climate Corporation
1

2. © 2015 The Climate Corporation All Rights Reserved
The Climate Corporation (TCC) provides
decision-making tools for farmers
Weather is both a feature that we provide to our users, and an input to our agronomic models
http://www.climate.com/

3. © 2015 The Climate Corporation All Rights Reserved
Big Data refers to computational limitations of
the analysis that you are performing
http://goo.gl/NB5hDO

4. © 2015 The Climate Corporation All Rights Reserved
The Cloud is remote computational
infrastructure, typically something you rent
4
https://cloud.google.com/
https://aws.amazon.com/
Elastic
Compute Cloud
(EC2)
Simple Storage
Service (S3)

5. © 2015 The Climate Corporation All Rights Reserved
The old way to do large-scale analysis at TCC
NOAA NCDC
Radar Archive
Simple Storage
Service (S3)3.Download
Elastic Compute
Cloud (EC2)
1. Request
2.Launch
6. Launch
Elastic Compute
Cloud (EC2)
5. Validate
TCC S3 bucket
4. Upload
7. Compute

6. © 2015 The Climate Corporation All Rights Reserved
The NOAA CRADA puts NEXRAD data on S3
Simple Storage
Service (S3)
1. Launch
Elastic Compute
Cloud (EC2)
Amazon public bucket
2. Compute

7. © 2015 The Climate Corporation All Rights Reserved
Everyone wins
TCC projects are several weeks shorter
TCC evaluations of new methods happen on larger datasets
We don’t pay Amazon for the S3 bucket to store NEXRAD data
Instead, we pay Amazon for the EC2 instances to process the larger dataset
NOAA data is used more widely, but without overwhelming NCDC
TCC/AWS found long-standing problem in NOAA archive, improving data quality
7
Opportunity around the other two components of the cloud