Download to read offline
Uploaded byDon Sheu
Corinne Hutchinson's 7/8/2015 PuPPy Presentation on GeoDjango
This document discusses using location data in web applications. It provides an overview of GeoDjango for building geospatial web apps in Django. GeoDjango allows importing geospatial vector and raster data and provides ORM for geospatial queries. The document discusses setting up models and databases, importing geospatial data, making queries, and scaling applications using techniques like geohashes to cache location lookups. Geohashes represent locations as encoded strings to quickly determine containing polygons and scale point-in-polygon lookups.
SoftwareâŠ
Corinne Hutchinson's 7/8/2015 PuPPy Presentation on GeoDjango
- 1. Itâs All AboutLocation, Location, Location Corinne Hutchinson July 8, 2015 -- PuPPY
- 2. Highlights of TonightâsTalk â Reasons for using location data in a web application â Overview of GeoDjango & setting up a basic web application with geospatial support â Scaling a system reliant on geospatial queries
- 3. Basic GeoSpatial Questions âWhere is A located? (i.e. point-in-polygon, mappability) â What is the distance between A & B? â What is the shortest path between A & B? (i.e. route planning) â Whatâs the elevation change between A & B?
- 4. GeoDjango â Included instandard installations of Django since 1.4 â Allows importing of geospatial data from essentially any vector data source (e.g. KML, shapefiles); raster data (bitmaps, etc) are not supported â Provides familiar ORM interface for geospatial queries â Straightforward to learn: excellent tutorial on main Django site
- 5. Starting a GeospatialApplication: DB â Pick your database and install any needed extensions â per-DB tutorial in GeoDjango docs, supported options are PostgreSQL/Postgis, MySQL, SQLite, or Oracle â PostgreSQL/Postgis and Oracle Spatial generally considered the most mature spatial database options
- 6. Starting a GeospatialApplication: Models â Define geomodels â Add admin interface
- 7. Adding Data? Lotsof Free Geospatial Data Sources â US Census TIGER (Topologically Integrated Geographic Encoding and Referencing): political boundaries e.g. states, counties, metro areas â Natural Earth: natural features â OpenStreetMap: map tiles, land use, etc â NASAâs Socioeconomic Data and Applications Center (SEDAC): data about human-environment interactions e.g. land use, poverty, climate â Open Topography: topo data, most of the world
- 8. Simple Data ImportTools
- 9. Modifying/Updating Polygons â GeoDjangoadmin provides drag-and-drop editing tools
- 10. Using Your App:Making Queries â Geospatial lookups through ORM o distance o point-in-polygon
- 11. This is great!Can we scale it? â Minimize direct database hits â Re-route duplicated database calls to cache (e.g. Redis, MemCache) â Reformat our data for cacheability: geohashes
- 12. Geohashes â Developed byGustavo Niemeyer, entered into public domain in 2008 â Method of sequentially subdividing the globe into spatial buckets â Buckets represented as encoded binary strings (e.g. 0010110101011100011000110001101111000111 -> 5pf666y7) â Allows for very fast point-in-polygon lookups
- 13.
- 14. Adding Geohashes toPoint-in-Polygon Lookups â Choose level of precision (5 or 6 are likely good) â Convert point to a geohash, then extract the center of that geohash â DB lookup to determine containing polygon â Finally, cache the geohash-to-polygon mapping (i.e. set the key âc22zpâ to the value âSeattleâ) â Subsequent lookups; check cache for existing key matching geohash before conducting DB lookup
- 15.
- 16. Take-Aways â GeoDjango issimple to use â Geohashes are a good tool to help scale geospatial lookups
- 17. More Awesome GeoSpatialLibraries â OGR/GDAL: interacting with geospatial data formats, i.e. opening files, etc â PyShp: ESRI shapefile handling in pure Python (https://pypi.python.org/pypi/pyshp) â PySAL: spatial analysis functions (https://github.com/pysal/pysal) â PyQGIS: essentially anything you might want to do with GIS data (http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/intro.html) â geopy: geocoding; integration with OpenStreetMaps, Google Geocoding API, Baidu Maps, and many more (https://github.com/geopy/geopy) â python-geohash: encoding/decoding points to geohashes, looking up geohash neighbors â descartes: plotting geometric objects in matplotlib (https://pypi.python.org/pypi/descartes) â NumPy: data wrangling (http://www.numpy.org/) â pandas: data wrangling (http://pandas.pydata.org/)
Editor's Notes
- #2Â about me
- #4Â Takeaway: these questions are all just basic questions about geometry
- #5Â clearer idea of the questions, whatâre the tools? In this talk I wanted to focus on getting a web application up and running that can do geospatial lookups. There are a LOT of other tools that can be used for geospatial data (not jsut on the web), Iâve listed a number on the last slide of this deck but wonât be talking about those in any greater depth tonight. Tonight Iâm just going to talk about using GeoDjango, which is a pretty fantastic tool. essentially diff is vector data is the data broken down into geometric features, contrasted with raster data which is the data broken down into grid. Remembering back to the questions we outlined before, itâs not terribly surprising that vector data will allow us to ask our questions.
- #6Â Not going to talk at great length about db choice; that could very easily be a whole talk on its own .
- #8Â ESRI (Environmental Systems Research Institute titan of GIS data, CA-based company started off as land-use consultants back in the 60s) shapefiles? Do you want to start off with any areas defined? do you want to, say, know the location of NYC, or do you want to custom-define regions yourself? If so, letâs check out some options on the next slide.
- #12Â er, sounds super, but what exactly does that mean? We donât want to kill our database doing a thousand lookups a second. However, we can easily throw many times that many lookups at a caching layer, and have no problem. Great! So how can we do this? advantages to relational db (structured data store, etc), cache is going to be super fast but thereâs no caching system that currently provides support for complex geospatial data. Caches like Redis use a basic key-value store, so we can cache a key (i.e. a unique string) and know that when we go to pull that out of our cache, weâll get whatever it was that we stored on the other end (a list, a string, etc). But we *canât* do awesome queries like we saw in the Django ORM. So . . . how can we do this? We need to restructure! Enter . . . next slide.
- #13Â Who here has heard of geohashes? Who here has worked with geohashes? Skip this slide if *everyone* raises their hand. System weâve been talking about up until now is pretty intuitive, very similar to how we naturally tend to think of data. A city is a polygon, you can draw it on a map, if you want to know if something is inside of it you just find the dot and find out if the dot is inside that polygon. Super! Geohashes are also pretty intuitive, but allow for a bit of abstraction on the level of precision.
- #15Â doesnât matter the level of precision, really, bigger is less precise of course, but smaller you just have way more to manage. consistency *does* matter, though.