Unlocking the Potential of the Cloud for IBM Power Systems
Big Data (Dev Meet Up II)
1.
2. What is it?
What is Big Data?
“...high-volume, [high] velocity and
[high] variety information assets that
demand cost-effective, innovative forms of
information processing for enhanced insight
and decision making.”
-2012 Gartner report
Hadoop / MrGeoSPADAC is now part of DigitalGlobe and their site has no mention of MrGeo…DIA project initiated by Terry Busch to extend geoprocessing to very large data setsBuilt by SPADAC ->GeoEye-> Digital GlobeUses HDFS and MapReduce to store, process, and index geospatial imagery and vector dataInteroperable with:ArcGIS DesktopCOMETGoogle EarthWMS clientsAdobe Flex and Silverlight environmentsListed on the Hadoop Apache page to be open sourcedAlso see Mansour Raad’s blog for some notes on doing distributed geoprocessing on Hadoop, e.g.http://thunderheadxpler.blogspot.com.au/2012/08/big-dataspatial-pigthreaded.html and http://thunderheadxpler.blogspot.com.au/2012/09/processing-big-data-with-apache-hive.htmlNeo4j SpatialUtilities for importing from ESRI Shapefile as well as Open Street Map filesSupport for all the common geometry typesAn RTree index for fast searches on geometriesSupport for topology operations during the search (contains, within, intersects, covers, disjoint, etc.)The possibility to enable spatial operations on any graph of data, regardless of the way the spatial data is stored, as long as an adapter is provided to map from the graph to the geometries.Ability to split a single layer or dataset into multiple sub-layers or views with pre-configured filtersMongoDBUtilities for importing from ESRI Shapefile as well as Open Street Map filesSupport for all the common geometry typesAn RTree index for fast searches on geometriesSupport for topology operations during the search (contains, within, intersects, covers, disjoint, etc.)The possibility to enable spatial operations on any graph of data, regardless of the way the spatial data is stored, as long as an adapter is provided to map from the graph to the geometries.Ability to split a single layer or dataset into multiple sub-layers or views with pre-configured filtersGeoTrellisDistributed raster & vector processing. Written in Scala. Open source. By Azavea. Quite new. Actively developed. Relatively good documentation. Quite promising. (My favourite)EsperJava event stream processing engine. Actually sounds pretty good. Not sure how active it is or how supported it is (open source basic version with enterprise, supported options). Has a spatial component.Geospatial complex event processingExtends the to include the ability to use Geospatial constructs in the rules for filtering events The ability to utilize the OGC Geospatial Functions within Esper's Event Processing Language (EPL).Contains, within, disjoint, intersects, overlaps, crosses, intersection, touches, buffer, relate, union, convex hull