Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
AWS for
mega data
Cambridge AWS
Meetup, Stirling House, Waterbeach, UK
17/09/2015
@azolnai
www.zolnai.ca
AWS stack
• EC2 m1.small instance
• eu-west-1c zone
• one std. 20Gb volume
• launch-wizard1 security group
• Public elasti...
GeoCloud2 Stack
• www.mapcentia.com
• @mhoegh
• PostGreSQL, PostGIS, MapServer,
TileCache, GDAL, WFS-F, WMS,
Elasticsearch...
NOAA GSHHG data
• Global Self-consistent, Hierarchical, High-resolution Geography
Database
– Original data and my blogpost
Hi-res. coastline
1/4 million vectors
CLIWOC
• CLImatological Database for the World's OCeans
– Global sailings from captains ships logs
– EU project on maritim...
Location only by nationality
Location only by timespan
Wind speed direction by timespan
Wind speed direction all timespans
Not just eye candy
• HC Darby: East Anglia geo-history
– UK Parishes constant framework
– added attributes from 1087 - 187...
BGS and Harvard Geospatial Library
Medieval to Industrial Revolution
Climate history takeaways
• Very rich dataset from 160 - 350
years ago
• Lots of data strung out along ship
tracks
• Navie...
AWS takeaways
• Supercomputer @ your fingertips
– Costs under £1 / day
– Faster on line than on the desktop
– No maintenan...
Upcoming SlideShare
Loading in …5
×

AWS for mega(geo)data

993 views

Published on

Cambridge(UK) AWS Meetup, geo-data application shown

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

AWS for mega(geo)data

  1. 1. AWS for mega data Cambridge AWS Meetup, Stirling House, Waterbeach, UK 17/09/2015 @azolnai www.zolnai.ca
  2. 2. AWS stack • EC2 m1.small instance • eu-west-1c zone • one std. 20Gb volume • launch-wizard1 security group • Public elastic ip 54.194.60.93 • IOW stock instance • Supercomputer for £20 / mo • See demo
  3. 3. GeoCloud2 Stack • www.mapcentia.com • @mhoegh • PostGreSQL, PostGIS, MapServer, TileCache, GDAL, WFS-F, WMS, Elasticsearch, Leaflet and OpenLayers • Superfast rendering
  4. 4. NOAA GSHHG data • Global Self-consistent, Hierarchical, High-resolution Geography Database – Original data and my blogpost
  5. 5. Hi-res. coastline 1/4 million vectors
  6. 6. CLIWOC • CLImatological Database for the World's OCeans – Global sailings from captains ships logs – EU project on maritime records 1750 - 1855 – 1/4M readings, 140 attributes each • Original blogpost and more on this – geohipster.com calendar – ship tracks only • Wind direction + force data – Doubled # points thru ES, FR, NL & UK lookups – almost 1/2 M points... faster online than desktop!
  7. 7. Location only by nationality
  8. 8. Location only by timespan
  9. 9. Wind speed direction by timespan
  10. 10. Wind speed direction all timespans
  11. 11. Not just eye candy • HC Darby: East Anglia geo-history – UK Parishes constant framework – added attributes from 1087 - 1877 • Current BGS geological WMS backdrop • Harvard 1610 Cambridgeshire map overlay • This lead to data validation loop w O|S – 1SpatialOnline online validation – Sent back to O|S for inclusion – Errors dropped from 27 to 1 / 1900 vector polygons
  12. 12. BGS and Harvard Geospatial Library
  13. 13. Medieval to Industrial Revolution
  14. 14. Climate history takeaways • Very rich dataset from 160 - 350 years ago • Lots of data strung out along ship tracks • Navies had standards • But issues w data recording and capture • Compare Old Weather NOAA ICOADS, &c. • Many visualizations • Few share &/or re-use • Foster online collaboration • Further climate research
  15. 15. AWS takeaways • Supercomputer @ your fingertips – Costs under £1 / day – Faster on line than on the desktop – No maintenance hassles • Q: Manage satellite imagery? A: Landsat8 AWS / Esri

×