Downloaded 18 times
Uploaded byAndrew Turner
Geospatial Analysis in the Cloud
This document discusses using cloud computing for scalable geospatial data processing and access. Key points include: - Leveraging cloud platforms like Amazon Web Services allows scaling computational resources elastically on demand. - Data can be stored redundantly across multiple availability zones for reliability. - A modular architecture with APIs allows components like geocoding to scale independently as needed.
More Related Content
![[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介](https://cdn.slidesharecdn.com/ss_thumbnails/20140219redshiftupdatesv1tokyo-140224010117-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Geospatial Analysis in the Cloud
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Geospatial Analysis in the Cloud
- 1. Use of CloudComputing for scalable geospatial data processing and access Andrew Turner CTO, FortiusOne andrew@fortiusone.com Partner: U.S. Federal Geographic Data Committee
- 2. What is GeoCommons? A Brief History
- 3. Vulnerability Identification Chicago Denver Atlanta Fiber Density Route 2 Los Angeles Route 1 Electric Transmission Line Density
- 6. Columbus Circle Holland Tunnel Baseline connectivity of a fiber WTC network provider in NYC. This particular provider is a good proxy for the structure of the entire island of Manhattan since they have about 80% of the right of ways on the island and a large number of egress points off the island. The higher the peak in the map the more frequently used the path is as a possible routing path.
- 7. Lastly a scenariois run where just 10,000 sq ft. of damage is done to the Holland Tunnel and the impact calculated. The result is a 8.6% loss of network connectivity, 134 times the impact of the WTC simulation. The dramatic impact is seen in the image from the loss as well as the stress put on the GW Bridge route out of the city.
- 8.
- 10.
- 11. Find Map a interesting data relevant area
- 12. Find Map a Visualize to interesting data relevant area find meaning
- 13. Find Map a Visualize to interesting data relevant area find meaning Layer, Modify, and Analyze
- 14. Find Map a Visualize to interesting data relevant area find meaning Collaborate Layer, Modify, with others and Analyze
- 15. Find Map a Visualize to interesting data relevant area find meaning Publish and Collaborate Layer, Modify, share results with others and Analyze
- 19.
- 20.
- 21.
- 22. Modularize Application Programming Interface Finder Maker RESTful Core Interfaces
- 23. Relational Databases Don’tScale Well
- 24. Datasets as Databases KML Shapefile CSV (Excel) GeoRSS Documents Finder Maker Core
- 25. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Finder Maker Core
- 26. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Finder Maker Parse & Store Core
- 27. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Finder Maker Parse & Store Core
- 28. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Finder Maker Parse & Store Core
- 29. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Finder Maker Parse & Store Core
- 30. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Download Finder Maker Parse & Store Core
- 31. Datasets as Databases Upload KML Shapefile CSV (Excel) GeoRSS Documents Download Finder Maker Parse & Store Analyze Core
- 32. Datasets as Databases Upload KML Shapefile Visualize CSV (Excel) GeoRSS Documents Download Finder Maker Parse & Store Analyze Core
- 33. Geospatial Catalog andServer
- 34.
- 35. Appliances • Sun 4150 •RAID Array
- 36. Web Scaled Racks •3 Appliances • Network File Storage • Load Balancer • Monitoring and Tunnels • Production & Staging racks • Racks in office for development
- 37. Limits in Limits in Scaling Development
- 38. Limits in Limits in Scaling Development People
- 39. Limits in Limits in Scaling Development People Power
- 40. Limits in Limits in Scaling Development People Power Size
- 41. Limits in Limits in Scaling Development People Power Size Cost
- 42. Limits in Limits in Scaling Development People Power Size Cost Time
- 43. Limits in Limits in Scaling Development People Power Size Cost Time
- 44. Limits in Limits in Scaling Development People Testing on “clean” machines Power Size Cost Time
- 45. Limits in Limits in Scaling Development People Testing on “clean” machines Power Size Deployment testing of Cost upgrades Time
- 46. Limits in Limits in Scaling Development People Testing on “clean” machines Power Size Deployment testing of Cost upgrades Time Controlled Environments
- 47. Leveraging the Cloud http://www.flickr.com/photos/kky/704056791 url
- 48.
- 49.
- 50.
- 51.
- 52. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand CentOS AMI
- 53. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand CentOS AMI build
- 54. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand register CentOS AMI bundle build
- 55. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand register CentOS AMI bundle instantiate build
- 56. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand register CentOS AMI bundle instantiate build
- 57. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand register CentOS AMI bundle instantiate build
- 58. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand register CentOS AMI bundle instantiate build
- 59. Elastic Computing Cluster- EC2 • Virtual Servers • Machine Images (AMI) • On-Demand register CentOS AMI bundle instantiate build
- 60. Elastic Block Store- EBS Create EBS 100 GB
- 61. Elastic Block Store- EBS Create EBS attach 100 GB
- 62. Elastic Block Store- EBS Create EBS attach 100 GB snapshot
- 63. Elastic Block Store- EBS Create EBS attach 100 GB snapshot S3 Diff v1
- 64. Elastic Block Store- EBS Create EBS attach 100 GB snapshot S3 Diff v1 Diff v2
- 65. Elastic Block Store- EBS Create EBS attach 100 GB snapshot Create & Attach S3 Diff v1 Diff v2
- 66. Elastic Block Store- EBS Create EBS attach 100 GB snapshot Create & Attach S3 Diff v1 Diff v2
- 67. Elastic Block Store- EBS Create EBS attach 100 GB snapshot Create & Attach S3 Diff v1 Diff v2
- 68. Elastic Block Store- EBS Create EBS attach 100 GB snapshot Create & Attach S3 Diff v1 Diff v2
- 69.
- 70. Additional Benefits • Federation •Tile generation • Content-delivery System • Simple Queue System (SQS) tiles/openstreetmap/9/74/97.png tiles/openstreetmap/9/74/98.png tiles/bluemarble/9/74/97.png S3 Storage tiles/bluemarble/9/74/98.png
- 71. Cloud Architecture • EC2image of current system architecture • EBS image stored to S3 of default database • Current application release in S3 • Start an EC2, attach data, attach code, startup v1.4.3 Default Datasets
- 72. Cloud Architecture • EC2image of current system architecture • EBS image stored to S3 of default database • Current application release in S3 • Start an EC2, attach data, attach code, startup create instance v1.4.3 Default Datasets
- 73. Cloud Architecture • EC2image of current system architecture • EBS image stored to S3 of default database • Current application release in S3 • Start an EC2, attach data, attach code, startup create instance v1.4.3 Default Datasets
- 74. Cloud Architecture • EC2image of current system architecture • EBS image stored to S3 of default database • Current application release in S3 • Start an EC2, attach data, attach code, startup create instance v1.4.3 attach data Default Datasets
- 75. Cloud Architecture • EC2image of current system architecture • EBS image stored to S3 of default database • Current application release in S3 • Start an EC2, attach data, attach code, startup create instance v1.4.3 Snapshot attach data Default Datasets Backup Backup Backup
- 76. Cloud Architecture • EC2image of current system architecture • EBS image stored to S3 of default database • Current application release in S3 • Start an EC2, attach data, attach code, startup create instance Cache S3 Downloads v1.4.3 Snapshot attach data Default Datasets Backup Backup Backup
- 77. Scaling • RESTful architecture •Caching for speed, and CDN support • Amazon Web Services • CloudWatch • Elastic Scaling • Load Balancer
- 78.
- 79. First Users: Meedan,Media
- 80.
- 81.
- 82. Data Federation community
- 83.
- 84.
- 85. Geocoding - Scaleas Required Upload CSV Cache Geocode Results API Geocoding Engine TIGER/Line SQLite
- 86. Geocoding - Scaleas Required Upload CSV Cache Geocode Results API Geocoding Engine TIGER/Line SQLite
- 87. Best Practices Appliedto the Government • Built using open, established tools • Full choice - Linux, Windows • Full Control • Repeatable processes • Continual backup • Scaling dynamic and large datasets • Synchronous and Asynchronous analysis
- 88. Level of Maturity •Widely adopted • Broad support and ecosystem • Full stack support
- 89. Perceived Impediments toAdoption • Single Vendor (open-source alternatives arising) • Maintenance and Location • Data Security
- 90. Thank you Andrew Turner andrew@fortiusone.com http://highearthorbit.com/presentations