Geohash: Integration of Disparate Geospatial Data

1,446 views
1,261 views

Published on

"Geohash: Integration of Disparate Geospatial Data," Abe Usher, Chief Innovation Officer, HumanGeo Group

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,446
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Geohash: Integration of Disparate Geospatial Data

  1. 1. Heatmaps & Geohash: Integration of Multi-Source Geospatial DataAbe Usher, CIOabe@thehumangeo.com703.955.1540@abeusher AKA Heatmaps are the HeatINFORMATION INTO INSIGHT
  2. 2. Our Menu of SubtopicsA LITTLE HISTORY WHY HEATMAPS? GEOHASH? DATA GONE WILD Big data requires new approaches. A new organizing construct forWhy geospatial data? information analysis. KITCHEN MODEL SPECIFIC EXAMPLES EASTER EGG New ways to combine internal data Concrete take-aways. Treats for making it through another with external new media for presentation. maximum insight. 2
  3. 3. What’s in it for you? 3
  4. 4. What’s in it for you? Make custom heatmaps Three iron laws of geospatial data integration and analysis 4
  5. 5. Whoami?People Data Beverages 5
  6. 6. GeohashGeohash is a coordinatetransformationthat facilitates combining twovariables (latitude and longitude)into a single(text) variablethat represents a bounding-boxcontaining the point of interest. 6
  7. 7. Heatmap“Discrete & continuous methods of kernel densityestimation” 7
  8. 8. A rose by any other name“Discrete & continuous methods of kernel densityestimation”  Gaussian  Quartic  Exponential  Triangular  Uniform  Epanechnikov 8
  9. 9. A rose by any other name“Discrete methods of kernel density estimation” 9
  10. 10. AboutBig Data (Digital) Human Geography  Predictive models of social drift & human behavior  Streaming media analytics  Micro-demographicsWe’re hiring! info@thehumangeo.com 10
  11. 11. Why Heatmaps and Geohash? Too much data 11
  12. 12. Why Heatmaps and Geohash? Too much data Trust in Internet data 12
  13. 13. Why Heatmaps and Geohash? Too much data Trust in Internet data Heatmaps look cool 13
  14. 14. Why Heatmaps and Geohash? Too much data Trust in Internet data Heatmaps look cool Geohash helps quantify data 14
  15. 15. Why Heatmaps and Geohash? Visual summary Too much data Trust in Internet data Heatmaps look cool Geohash helps quantify data Quantitative methods 15
  16. 16. Trust and Internet InformationTracy Morrow aka “Ice T” 16
  17. 17. Trust and Internet Information “Game knows game, baby.”Tracy Morrow aka “Ice T” 17
  18. 18. Trust and Internet Information “If you have expert knowledge, then you are capable of recognizing expert knowledge.” [paraphrased]Tracy Morrow aka “Ice T” 18
  19. 19. Trust and Internet InformationCan we actually trust this Internet stuff? 19
  20. 20. Trust and Internet Information 20
  21. 21. Trust and Internet Information 21
  22. 22. Trust and Internet Information 22
  23. 23. Salami Slicing Salami slicing: series of minor observations, resulting in a larger observation that would be difficult to performhttp://en.wikipedia.org/wiki/Salami_slicing 23
  24. 24. Seven Layer GLT1. OpenStreetMap data2. Flickr3. Panoramio4. Geonames.org5. Twitter6. Wikimapia7. 4Square* Geospatial Lattice of Trusted Data 24
  25. 25. Seven Layer GLT1. OpenStreetMap data2. Flickr3. Panoramio4. Geonames.org5. Twitter6. Wikimapia7. 4SquareSpatial Temporal User Finds From the Field (STUFFF) 25
  26. 26. Rule #1: Think in terms of aggregation Twitter data Panoramio Tourist photos Classified dataTwitter geohash ez420 – coffee shopPanoramio geohash ez420 – StarbucksClassified geohash ez420 - Wifi Trust through aggregation 26
  27. 27. Rule #1: Think in terms of aggregation Twitter dataTwitter geohash ez420 – coffee shopPanoramio geohash ez420 – StarbucksClassified geohash ez420 - Wifi Panoramio Tourist photosGeohash creates simple string variables. Classified dataMatching strings = super easyMatching similar coordinates = impossible Trust through aggregation 27
  28. 28. Rule #1: Think in terms of aggregation Twitter data Panoramio Tourist photosUse geohash to apply collaborativefiltering techniques to develop new Classified datamodels of trust & data confidence. Trust through aggregation 28
  29. 29. Rule #2: Selectively throw away precisionEntity #1 Latitude Longitude 40.998946 28.9232 41.005164 28.973668 41.018765 29.016412 41.062268 29.030145Entity #2 Latitude Longitude 40.999100 28.92111 41.018112 28.973991 41.018880 29.016902 41.062110 29.030122 29
  30. 30. Rule #2: Selectively throw away precisionEntity #1 Latitude Longitude Geohash 40.998946 28.9232 SXK94 41.005164 28.973668 SXK97 41.018765 29.016412 SXK9K 41.062268 29.030145 SXK9S 30
  31. 31. Rule #2: Selectively throw away precisionEntity #1 Latitude Longitude Geohash 40.998946 28.9232 SXK94 41.005164 28.973668 SXK97 41.018765 29.016412 SXK9K 41.062268 29.030145 SXK9SEntity #2 Latitude Longitude Geohash 40.999100 28.92111 SXK94 41.018112 28.973991 SXK97 41.018880 29.016902 SXK9K 41.062110 29.030122 SXK9S 31
  32. 32. Kitchen Model for Spatial Analysis 32
  33. 33. Kitchen Model for Spatial AnalysisChef 33
  34. 34. Kitchen Model for Spatial AnalysisChef Ingredients 34
  35. 35. Kitchen Model for Spatial AnalysisChef Ingredients Utensils 35
  36. 36. Kitchen Model for Spatial AnalysisChef Ingredients Utensils Presentation 36
  37. 37. Kitchen Model for Spatial AnalysisChef Ingredients Utensils Presentation 37
  38. 38. Types of HeatmapsTurnkey GeoCommons SpatialKey MapBox/TileMill ArcGIS Desktop QGISCustom Python R Javascript 38
  39. 39. Types of HeatmapsTurnkey GeoCommons SpatialKey MapBox/TileMill ArcGIS Desktop QGISCustom Python R Javascript 39
  40. 40. Heatmap: Recipe One“OSM Style”Get Python http://python.orgGet the sethoscope libraryhttp://www.sethoscope.net/heatmap/Get datahttp://bit.ly/geotweet_schttps://dev.twitter.com/docs/streaming-api/methods#locationsCommand line:heatmap.py -g portland.gpx -o output.png --height 800 --osm 40
  41. 41. 41
  42. 42. Heatmap: Recipe OneStitch it together in an MP4 movie!Get the CLI app:http://ffmpeg.org/Command line:heatmap.py -g portland.gpx -o output.png --height 800 –osm –affmpeg -i frame-%05d.png OSM_is_awesome.mp4 42
  43. 43. Heatmap: Recipe OneProps toSeth Golub from Googlehttp://www.sethoscope.net/ 43
  44. 44. Rule #3:Beware of population effects http://xkcd.com/1138/ 44
  45. 45. Rule #3: Beware of population effects Absolute valueNormalized value = Population estimate 45
  46. 46. Rule #3:Beware of population effects 34 72 52 22 46
  47. 47. Rule #3: Beware of population effects 34 Absolute value 2,000Normalized value = Population estimate 72 52 16,000 25,000 22 2,000 47
  48. 48. Conclusion1. Think in terms of aggregation2. Selectively throw away precision3. Beware of population effects 48
  49. 49. Contact UsHumanGeo NY HumanGeo DC1221 Avenue of the Americas 2500 Wilson BoulevardSuite 4200 | New York, NY 10020 Suite 310 | Arlington, VA 22201info@thehumangeo.com | 703.955.1540 | www.thehumangeo.com 49
  50. 50. Easter Egghttp://bit.ly/geotweet_sc 50

×