Your SlideShare is downloading. ×
0
The State of Big Datafor Geo                          @ajturner                  CTO DC Dev Center                     @se...
Trends
Of the 318 million mobile handsetsshipped in 2011 79.9% were GPSenabled                            iSuppli 2010
Jess3 2011
Meeker 2011
In the United States 72.2% of usersaccessed social media sites andblogs through their mobile devices –up 37% from 2010    ...
What is Big Data?
Volume
MrGeo
MapReduce Geo (MrGeo)• DIA project initiated by Terry Busch to extend geoprocessing to  very large data sets• Built by SPA...
Brian Levy 2010
Velocity
Esper
GCEP• Geospatial complex event processing• Extends the to include the ability to use Geospatial  constructs in the rules f...
Variety
Neo4j Spatial• Utilities for importing from ESRI Shapefile as well as  Open Street Map files• Support for all the common g...
Peter Neubauer 2011
Peter Neubauer 2011
ESRI and Big Data
Why Big Data?
1. Connecting Big   Data to GIS
2. Scaling   Geoprocessing
Case Study:NYC Marathon
Case Study:Colorado Wildfires
3. Evolving   GeoAnalysis
Are we doing itbackwards?
May 2nd OBL Tweets6,454 Tweets with GPS6,000,000 total Tweets
Why run our analysiswith six thousandinstead of six million?
Tracking the firstOBL Tweet
Extract        Analyze          orAnalyze        Extract
Statistical Mechanics
Community Detection
The Future
Real Time PatternAnalysis and Alerting
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
The State of Big Data for Geo - ESRI Big Data Meetup
Upcoming SlideShare
Loading in...5
×

The State of Big Data for Geo - ESRI Big Data Meetup

2,591

Published on

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,591
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
96
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide
  • The biggest issue with map-reduce and geostatistics is that map process assumes no inter-record dependencies. Which means you need flow control and multiple MR jobs (why cascading makes so much sense).
  • Meet customer demands for handling larger data, faster data and more data types
  • Transcript of "The State of Big Data for Geo - ESRI Big Data Meetup"

    1. 1. The State of Big Datafor Geo @ajturner CTO DC Dev Center @seangorman Strategist DC Dev Center
    2. 2. Trends
    3. 3. Of the 318 million mobile handsetsshipped in 2011 79.9% were GPSenabled iSuppli 2010
    4. 4. Jess3 2011
    5. 5. Meeker 2011
    6. 6. In the United States 72.2% of usersaccessed social media sites andblogs through their mobile devices –up 37% from 2010 ComScore 2011
    7. 7. What is Big Data?
    8. 8. Volume
    9. 9. MrGeo
    10. 10. MapReduce Geo (MrGeo)• DIA project initiated by Terry Busch to extend geoprocessing to very large data sets• Built by SPADAC -> GeoEye -> Digital Globe• Uses HDFS and MapReduce to store, process, and index geospatial imagery and vector data• Interoperable with: – ArcGIS Desktop – COMET – Google Earth – WMS clients – Adobe Flex and Silverlight environments• Listed on the Hadoop Apache page to be open sourced
    11. 11. Brian Levy 2010
    12. 12. Velocity
    13. 13. Esper
    14. 14. GCEP• Geospatial complex event processing• Extends the to include the ability to use Geospatial constructs in the rules for filtering events• The ability to utilize the OGC Geospatial Functions within Espers Event Processing Language (EPL). –Contains, within, disjoint, intersects, overlaps, crosses, intersection, touches, buffer, relate, union, convex hull
    15. 15. Variety
    16. 16. Neo4j Spatial• Utilities for importing from ESRI Shapefile as well as Open Street Map files• Support for all the common geometry types• An RTree index for fast searches on geometries• Support for topology operations during the search (contains, within, intersects, covers, disjoint, etc.)• The possibility to enable spatial operations on any graph of data, regardless of the way the spatial data is stored, as long as an adapter is provided to map from the graph to the geometries.• Ability to split a single layer or dataset into multiple sub- layers or views with pre-configured filters
    17. 17. Peter Neubauer 2011
    18. 18. Peter Neubauer 2011
    19. 19. ESRI and Big Data
    20. 20. Why Big Data?
    21. 21. 1. Connecting Big Data to GIS
    22. 22. 2. Scaling Geoprocessing
    23. 23. Case Study:NYC Marathon
    24. 24. Case Study:Colorado Wildfires
    25. 25. 3. Evolving GeoAnalysis
    26. 26. Are we doing itbackwards?
    27. 27. May 2nd OBL Tweets6,454 Tweets with GPS6,000,000 total Tweets
    28. 28. Why run our analysiswith six thousandinstead of six million?
    29. 29. Tracking the firstOBL Tweet
    30. 30. Extract Analyze orAnalyze Extract
    31. 31. Statistical Mechanics
    32. 32. Community Detection
    33. 33. The Future
    34. 34. Real Time PatternAnalysis and Alerting
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×