The State of Big Data for Geo - ESRI Big Data Meetup

  • 2,278 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,278
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
81
Comments
0
Likes
5

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • The biggest issue with map-reduce and geostatistics is that map process assumes no inter-record dependencies. Which means you need flow control and multiple MR jobs (why cascading makes so much sense).
  • Meet customer demands for handling larger data, faster data and more data types

Transcript

  • 1. The State of Big Datafor Geo @ajturner CTO DC Dev Center @seangorman Strategist DC Dev Center
  • 2. Trends
  • 3. Of the 318 million mobile handsetsshipped in 2011 79.9% were GPSenabled iSuppli 2010
  • 4. Jess3 2011
  • 5. Meeker 2011
  • 6. In the United States 72.2% of usersaccessed social media sites andblogs through their mobile devices –up 37% from 2010 ComScore 2011
  • 7. What is Big Data?
  • 8. Volume
  • 9. MrGeo
  • 10. MapReduce Geo (MrGeo)• DIA project initiated by Terry Busch to extend geoprocessing to very large data sets• Built by SPADAC -> GeoEye -> Digital Globe• Uses HDFS and MapReduce to store, process, and index geospatial imagery and vector data• Interoperable with: – ArcGIS Desktop – COMET – Google Earth – WMS clients – Adobe Flex and Silverlight environments• Listed on the Hadoop Apache page to be open sourced
  • 11. Brian Levy 2010
  • 12. Velocity
  • 13. Esper
  • 14. GCEP• Geospatial complex event processing• Extends the to include the ability to use Geospatial constructs in the rules for filtering events• The ability to utilize the OGC Geospatial Functions within Espers Event Processing Language (EPL). –Contains, within, disjoint, intersects, overlaps, crosses, intersection, touches, buffer, relate, union, convex hull
  • 15. Variety
  • 16. Neo4j Spatial• Utilities for importing from ESRI Shapefile as well as Open Street Map files• Support for all the common geometry types• An RTree index for fast searches on geometries• Support for topology operations during the search (contains, within, intersects, covers, disjoint, etc.)• The possibility to enable spatial operations on any graph of data, regardless of the way the spatial data is stored, as long as an adapter is provided to map from the graph to the geometries.• Ability to split a single layer or dataset into multiple sub- layers or views with pre-configured filters
  • 17. Peter Neubauer 2011
  • 18. Peter Neubauer 2011
  • 19. ESRI and Big Data
  • 20. Why Big Data?
  • 21. 1. Connecting Big Data to GIS
  • 22. 2. Scaling Geoprocessing
  • 23. Case Study:NYC Marathon
  • 24. Case Study:Colorado Wildfires
  • 25. 3. Evolving GeoAnalysis
  • 26. Are we doing itbackwards?
  • 27. May 2nd OBL Tweets6,454 Tweets with GPS6,000,000 total Tweets
  • 28. Why run our analysiswith six thousandinstead of six million?
  • 29. Tracking the firstOBL Tweet
  • 30. Extract Analyze orAnalyze Extract
  • 31. Statistical Mechanics
  • 32. Community Detection
  • 33. The Future
  • 34. Real Time PatternAnalysis and Alerting