SAVING GAIAw/jQuery Mobile, OpenLayers and GeoDjango
CALVIN CHENG     www.calvinx.comwww.od-eon.com/blogs/calvin
1   Context         2    Technology Stack                    3   Installation
4   Django GeoModel & ORM         5   Geospatial Libraries                    6   Utilities
7   GeoAdmin        8   OpenLayers                 9   jQuery (Mobile)
10   Possibilities          11   Conclusion
1Context
REDDCalculator.com (Beta)         SEC (Closed Beta)                                                          LUC          ...
SPECIES WHA--??Species Extinction Calculator (SEC)
4%                       4"&#()#*%           123#              !"#$$%&()*"+,-./0"            !"#$%
3,000                       2,250   Number of species   1,500                        750                          0       ...
+)!) ) &,-! )    * "#$%$&(            ( )             .             &,-             .                 )             "#$%$&...
!"#$%&()3+(451"67(&/&012(                     !"#$%&()*+(,-.&(&/&012(
0$1,23+4")56)."+4,2+"+*,7$2/+)$4"8$*.51                   !"#$%&""(")*+,--,."+*"/#"+                                      ...
MATRIX-CALIBRATED SPECIES-AREAAccounts for taxon-specific responses to different components of the landscape
EXAMPLE OF AN AREA?Conservation Forests, National Reserves etc    Borneo 743,330 square kilometres
USERS AND USE-CASEConservation scientists and Land owners define areas which areconserved by plotting polygons on a map so...
USERS AND USE-CASEREDDCalculator.com computes the socio-economic trade-offs andresults as a consequence of the Norway-Indo...
2Technology Stack
Layers & Features (‘Vector Objects’) jQuery &OpenLayers                                                    User Interface ...
3Installation
(2.7)
(trunk)
sudo -u postgres psql -d yourdatabase_name -f { ... }/postgis.sqlsudo -u postgres psql -d yourdatabase_name -f { ... }/pos...
sudo -u postgres psql -d yourdatabase_name -f /opt/local/var/macports/software/postgis/1.5.2_1+postgresql90/opt/local/shar...
Portage existsBut here’s a magical way - https://github.com/stefantalpalaru/dotfiles/tree/master/Gentoo/custom%20overlay/p...
# settings.pyINSTALLED_APPS = (  # Core Django Apps  django.contrib.contenttypes,  django.contrib.auth,  django.contrib.se...
# settings.pyDATABASES = {        default: {           ENGINE:             django.contrib.gis.db.backends.postgis,        ...
4 DjangoGeoModel & APIs
from django.contrib.gis.db import models
from django.contrib.gis.db import modelsclass Province(models.Model):   name = models.CharField(max_length=256)   name_lon...
from django.contrib.gis.db import modelsclass Province(models.Model):   name = models.CharField(max_length=256)   name_lon...
What are the geometry fields available to us in django?
GEOS: OGC simple feature implementation PointField MultiPointField LineStringField MultiLineStringField PolygonField Multi...
PointField         (x, y)
MultiPointField((x1, y1), (x2, y2), (x3, y3))
LineStringField((x1, y1), (x2, y2), (x3, y3), (x4, y4))
MultiLineStringField(((x1, y1), (x2, y2), (x3, y3), (x4,     y4)),((x5, y5), (x6, y6)))
PolygonField((x1, y1), (x2, y2), (x3, y3), (x4, y4), (x1, y1))
MultiPolygonField((x1, y1), (x2, y2), (x3, y3), (x4, y4), (x1, y1)),     ((x5, y5), (x6, y6), (x7, y7), (x5, y5)))
GeometryCollectionField((x1, y1), (x2, y2), (x3, y3), (x4, y4), (x1, y1)),          ((x5, y5), (x6, y6), (x7, y7)),       ...
Practical Example #1: Species Extinction Calculator
Practical Example #2: REDDCalculator.com
from django.contrib.gis.gdal import datasource, spatialreferencefrom django.contrib.gis.geos import multipolygon, geosgeom...
from django.contrib.gis.db import modelsclass AreaData(models.Model):   district = models.ForeignKey(District) # District ...
area = AreaData.objects.get(fid = feature.fid)geometry_name = feature.geom.geom_type.namegeometry_object = feature.geom.tran...
5Geospatial Libraries
GEOS   GDAL   PROJ.4  Mapnik   GeoIP
GEOS                Geometry Engine - Open SourceWhat does it do?  Works with django GIS app to give us GeoModels!
GEOS                Geometry Engine - Open SourceOther Details   Required dependency for django GIS app   C++  Implements ...
GDAL                 Geospatial Data Abstraction LibraryWhat does it do?  Reads and writes to a variety of vector file for...
GDAL               Geospatial Data Abstraction LibraryOther Details  C/C++  Implements OGC simple feature specification  e...
PROJ.4             Cartographic Projections LibraryWhat does it do?  Manages spatial reference systems and projections e.g...
PROJ.4               Cartographic Projections LibraryOther Details  Required dependency for PostGIS  C  Ensures that geome...
Mapnik             Toolkit for developing Mapping ApplicationsWhat does it do?  For the control freak in you :-)  Build yo...
Mapnik             Toolkit for developing Mapping ApplicationsOther Details  C++  Uses boost for python interface  No py27...
GeoIP              Locate users by IP addressWhat does it do?  Determines the location of app user via user’s IP address
GeoIP              Locate users by IP addressOther Details  C  No py27-geoip yet. Solution? Create your own local portfile...
6Utilities
LayerMapping./manage.py ogrinspect       ./manage.py ogrinfo
./manage.py ogrinspect         and    LayerMapping
$ ./manage.py ogrinspect ../proj_public/media/files/redd_data/kalimantan_universe.shp Location --srid=4326 --mapping# This...
# models.pyfrom django.contrib.gis.db import modelsclass Location(models.Model):   island = models.CharField(max_length=50...
import osfrom django.contrib.gis.utils import LayerMappingfrom django.contrib.gis.gdal import DataSourcefrom location.mode...
7GeoAdmin
from django.contrib.gis import admin as geoadminfrom location.models import Locationadmin.site.register(Location, geoadmin...
8OpenLayers
“ OpenLayers is a pure JavaScript library for  displaying map data in most modern web browsers,  with no server-side depen...
9 jQuery(mobile)
“ jQuery Mobile is a Touch-Optimized Web  Framework for Smartphones and Tablets. ”
+   +
$(document).ready(function(){  $(#map_delete).hide()  // Define a custom styles which will be used by our polygon layer  v...
// Define our vector layerpolygonLayer = new OpenLayers.Layer.Vector("Polygon Layer", {styleMap: styleMap})polygonLayer.ev...
// controlsdrawControl = new OpenLayers.Control.DrawFeature(polygonLayer, OpenLayers.Handler.Polygon,                     ...
// Define Map Layersvar gphy = new OpenLayers.Layer.Google("Google Physical", {type: google.maps.MapTypeId.TERRAIN});var g...
// Google.v3 uses EPSG:900913 as projection, so we have to transform our coordinates mapProjection = map.getProjectionObje...
// draw the existing features into the polygonLayer  var loaded_features = []  for (var i=0; i<list_of_coords_json.length;...
10Possibilities
1. Offline Maps for Field Usage   2. Remote Controlled UAV for data collection3. Crunch collected data and generate visual...
Offline Maps for Field Usage
Layers & Features (Saved Locally, Syncing)   NativeiOS/Android                                                            ...
The Problem with iOS MapKit FrameworkMKMapView class implements Google Maps API nativelyUnfortunately, Google Maps terms o...
Alternative? https://github.com/route-me/route-meSupports many map sources - OpenStreetMap, MicrosoftVirtualEarth, CloudMa...
A Comparison                          MKMapKit     Route-Me Embed in iOS App           YES          YES     Interaction   ...
The Problem with Android Google Maps FrameworkRenders Google Maps of courseSupport for alternative map sources is important
Alternative? https://github.com/camptocamp/android-gis.git Supports many map sources - OpenStreetMap, Microsoft VirtualEar...
Benefits  Fast  (obviously, since Map Tiles are stored locally)  100% Offline  Stored as blob data for SQLite Client Access
Remote Controlled UAVs?
Collect Real Data                             =>         Effects on Landscape Change on Species       e.g. obtain actual c...
Prediction is great.But we want real world data from primary forests!
http://www.youtube.com/watch?v=xjLXL72Medw&feature=autoshare
Crunch Collected Data           =>  2 possible approaches- Decoupled R and Python- Integrated Approach RPy Option
Decouple R and PythonPython retrieves data from django/postgresqlPython converts data shapefiles, images etcLaunches R giv...
Integrated R & Python with RPy R and Python closely coupled RPy and RPy2 provides an interface to R Programming Language P...
11Conclusion
Getting a GPS fix via Satellite?
Mathematical Modelling, Scenario Analysis      for Environmental Policies
REDDCalculator.com (Live)         SEC (Closed Beta)                                                          LUC          ...
Species Extinction Calculator and REDDCalculator:Mathematical Models and Formulas by Koh et. al are completelytranslated i...
Key Takeaways
Mapnik is useful for rasterizing vector objects when we encounter largedatasets (in our case, 200 MB+ of protected areas i...
geography=True/False in a GeoDjango field2 ways to import data from a given GIS Shape File(1) LayerMapping(2) Direct Assig...
“ Advancing Environmental Sciences: helping people, scientists, NGOs and governments make practical Land Area Usage and Co...
Gotchas?
Make sure Projections/Datum in your shape file source correspondswith your geometry field’s SRID and Map SourceUsing LayerMa...
Credits
+ North-South FundProf. Koh Lian Pin                 Prof. Matthew Hansen             Prof. Holly GibbsETH Zurich, Dept of...
Stefan Talpalaru                  Liviu Bogdan Chiributa   That’s Me! :-)Odeon Consulting Group            Odeon Consultin...
Previous GeoDjango Presentations + my little contribution   www.geodjango.org/presentations   www.od-eon.com/presentations
Questions and Discussion welcome!    calvin.cheng@od-eon.com
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Saving Gaia with GeoDjango
Upcoming SlideShare
Loading in...5
×

Saving Gaia with GeoDjango

1,542

Published on

Various projects with ETH Zurich, Koh et al.

Computing the socio-economic consequences of land use/change decisions made!

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,542
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide
  • * hallo and thank you for joining me here to learn more about geodjango, jquery mobile and openlayers.\n* This is my first time at europython and I am glad to be here, learning so much from you, my python peers.\n
  • * I am Calvin and I work as a user-interface/python developer in a company incorporated in Singapore.\n* We are a very small and relatively young software development team (3-4 years old), but our company is unusual because we work with customers from all over the world.\n* We are very fortunate to be able to work on some really fascinating projects and amazing customers for the past 2 to 3 years.\n* We have built all sorts of application including game and social network ones.\n* But over the past couple of years, we have discovered that we are especially passionate about building software related to science.* Specifically, in our various projects with ETH Zurich, I am learning a lot about GIS (Geographical Information System) in relation to Conservation Biology and Environmental Science.\n* In a separate and unrelated project with the University of Southern California, our team has the opportunity of helping neuroscience researches archive and understand neural circuitry - a fascinating and ever-evolving domain as scientists learn more about different parts of the mammalian brain. \n
  • * Today, I will be talking about the context in which jQuery Mobile, OpenLayers and GeoDjango are applied to help scientists, NGOs and governments make practical land use and conservation decisions.\n* I will illustrate the technology stack that we have used and talk briefly about the installation process for various GeoDjango dependencies on Mac OS X and Gentoo. \n* We usually use Gentoo on our servers. Yes I know, it&amp;#x2019;s nuts but that&amp;#x2019;s what Stefan, my colleague and CTO in Odeon, is proficient with, with over 11 years of experience.\n* My weapon of choice for my interface design and python work is my Macbook.\n
  • * We will of course, have some code examples showing how we set up the Models in our environmental science apps and how we use the GeoDjango ORM to update our database with information from the user or from existing shapefile datasource.\n* In order for us to work effectively, it also helps to understand a little about what goes on behind the scenes and what each of the geospatial libraries we have installed do.\n* It also helps for us to understand some of the helper functions, otherwise known as utilities, in GeoDjango helps make life easier when we want to inspect a shape file or write our model knowing what to expect from the shape file.\n\n
  • * Part of what makes django so wonderful is its django admin feature. The good news, of course, is that when geodjango was officially merged into django trunk as the GIS contrib app, they have also made GeoAdmin awesome by automagically showing the geographical feature on a map using openlayers in the admin.\n* Section 8 and 9 actually go hand-in-hand together and I will show you some example javascript code there.\n
  • * In the last part of my presentation, I will explore different ways, possible ways to improve the app in our next development iteration.\n* And finally, summarize the lessons learnt in our first development iteration.\n
  • \n
  • * There are 3 tools under active development for environmental scientists at ETH Zurich.\n* The first app is reddcalculator.com, which is a tool to illustrate socioeconomic consequences, job losses and gains as a result of implementing the Norway-Indonesia REDD Moratorium.\n* REDD stands for &amp;#x201C;Reducing Emissions from Deforestation and Forest Degradation in Developing countries&amp;#x201D;.\n* The idea is to assign a financial value for the carbon stored in forests and offer incentives for developing countries to reduce emissions from forested lands and invest in low-carbon paths to sustainable development.\n* The 2nd tool is species extinction calculator, which is a mathematical model designed by a group of Environmental Scientists, Koh Et Al, to predict species loss as a consequence of change in their habitat or the landscape.\n* The land use calculator is a generalized model, similar to what we have already done in reddcalculator.com and predicts socioeconomic consequences of land use decisions and policies. \n\n
  • * We will begin by talking about the Species Extinction Calculator, an app which is currently under closed beta.\n
  • * Orangutan\n* Clouded Leopard\n* Pygmy Elephants\n* Sumatran Rhino\n* These are the 4 major species that are going extinct in a South East Asian island called Borneo.\n* As our human activities reduce the natural habitats, forests and food available to these species, they are directly affected and begin to disappear from Earth.\n* While I am not a green activist or terrorist, it saddens me that the species diversity in this world are dramatically reduced as a consequence of our actions.\n* Following Ms Ravenscroft admonition to adopt Diversity as a Dependency in the Python community on Monday, I am proposing that we consider our friends in the animal kingdom part of beauty and diversity of the Python community.\n* After all, Python&amp;#x2019;s logo is pretty much represented by 2 very cute blue and yellow snakes.\n* So, on a serious note, this project is very much about using Python code to help decision makers understand the consequence of their actions and help people make better decisions on land use and socio-economic tradeoffs.\n
  • * So, this is where it begins. Species extinction in the old days have been simply correlated to Area loss using the Arrhenius equation.\n\n
  • * Here&amp;#x2019;s a graphical representation of the Arrhenius formula.\n* But of course, this is an outdated model and is wildly inaccurate.\n
  • * Modern conservation biologists and environmental scientists needed a more accurate prediction model\n* And they realise that land-use change on a mainland often creates landscapes that kind of look like islands of forests in an ocean of other land uses\n\n
  • * So this is what modern environmental scientists adopted as a working model.\n* The species-area model is often used by conservation biologists to predict biodiversity loss based upon forest loss\n* S(new) refers to species after an area of interest (also referred to as the &amp;#x201C;Total Landscape Area&amp;#x201D;) has been modified by human activity.\n* S(original) refers to the original number of species residing in the area.\n* Ditto for A\n* Z is an arbitrary number which varies according to various other conditions\n
  • However, there are 2 problems with this approach...\n#1: it assumes that the landscape matrix, the land uses in-between forest fragments, are completely inhospitable to any species, which obviously is not always true\n#2: it assumes that species are not affected by edge effects: deleterious effects on a forest fragment mediated by biophysical interactions near the interface between forest and matrix\nAlthough ecologists know that these assumptions are unrealistic, the species-area model continues to be used in assessments of biodiversity loss and in conservation planning. That was what motivated Koh et. al to develop a better tool \n\n
  • * Koh et al&amp;#x2019;s improved model addresses those shortcomings by accounting for how species respond to changes in composition &amp; quality of the matrix, and it also accounts for effects of forest edges\n* These improvements allow for predictions of extinction risks that are specific to the landscape and the taxon of interest\n* Result? An improved accuracy in predicting how land use changes affect species diversity\n* This new model of predicting species response to land use changes has been accepted in Conservation Biology and Journal of Applied Ecology.\n\n
  • * This gives us a beautiful 3 dimensional graph as above.\n* So how does all these formulas relate to what we do with a GIS (Geographical Information System) application using GeoDjango?\n
  • * Area, of course. Calculating and defining the area is core to producing a reasonably accurate prediction estimate for policy making by governments, NGOs and the united nation.\n* 3rd largest island in the world. (after greenland and new guinea). Ref: http://en.wikipedia.org/wiki/List_of_islands_by_area\n*The island is divided among three countries: Brunei, Indonesia and Malaysia. Approximately 73% of the island is Indonesian territory. The Malaysian states of Sabah and Sarawak in the north occupy about 26% of the island. The sovereign state of Brunei, located on the north coast, comprises about 1% of Borneo&apos;s land area. It has the oldest rainforest in the world ahead of the Amazon rainforest inBrazil.\n
  • * 3rd largest island in the world. (after greenland and new guinea). Ref: http://en.wikipedia.org/wiki/List_of_islands_by_area\n* Specifically Kalimantan\n\n
  • * Loss of species, carbon emissions as a balance against employment and financial losses\n\n
  • * Here&amp;#x2019;s an example screen shot of polygons plotted by land owners in collaboration with environmental scientists.\n* We will go into the code details and what each of these polygons in a while.\n* But first, for those not familiar with terminologies and the basic concepts of Geographical Information Systems (GIS) software, I will provide a simple overview of the technology stack and associated libraries which you will need to install before getting started.\n
  • \n
  • * The user interface\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • * &amp;#x201C;from django.db import models&amp;#x201D; is, of course, the usual way we import the django models class before writing our custom models.\n* For our model to contain GIS capabilities, like storing a geographical feature as an attribute, we will simply subclass from django.contrib.gis.db instead.\n\n
  • * add the geometry field you need. in this example, &amp;#x201C;geom&amp;#x201D; is our geometry field and it is a contains a MultiPolygon object using the datum WGS84 (i.e. 4326)\n* \n\n
  • * an interesting note about the geography attribute in a GeoDjango Field\n* polygons computed are assumed to be based on its 2D area.\n* by stating geography=True, areas computed will based on the ellipsoidal surface area, which gives a more accurate result at the expense of slightly poorer performance\n* in other words, you need to take a design decision on this trade-off.\n
  • * &amp;#x201C;from django.db import models&amp;#x201D; is, of course, the usual way we import the django models class before writing our custom models.\n* For our model to contain GIS capabilities, like storing a geographical feature as an attribute, we will simply subclass from django.contrib.gis.db instead.\n\n
  • \n\n
  • * represented by coordinates, specific to the projection or datum\n\n
  • * represented by coordinates, specific to the projection or datum\n\n
  • * represented by coordinates, specific to the projection or datum\n\n
  • * represented by coordinates, specific to the projection or datum\n\n
  • * represented by coordinates, specific to the projection or datum\n\n
  • * obviously :-) more than 1 or more Polygons stored in this field!\n\n
  • * A collection of different geometries can be stored in the field\n\n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n\n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n\n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n\n
  • * Here&amp;#x2019;s an example showing you how I have set up our AreaData class. The attributes in the AreaData class includes the MultiPolygonField as well as other information I need for Species Extinction computation.\n\n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n\n
  • * ogrinspect is a really useful django management command which comes with django.contrib.gis.\n* It inspects a given .shp file (Shape File) and auto generates a suggested model and LayerMapping dictionary.\n\n
  • * So as suggested by ogrinspect, we should define the location_mapping dictionary if we decide to use the LayerMapping utility to import shapefile data into your postgis database.\n\n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n* The strict parameter, when True, is to stop the execution of model mapping upon the first error encountered. For me, I wanted to know when an error has occurred when I importing the data in the shapefile, hence, I specify it to be True. The default is False, which means that it will attempt to continue even if it encounters an error.\n\n
  • \n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n\n
  • * Using GeoDjango APIs with a model containing a MultiPolygonField!\n\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • * Here, in this OpenLayer javascript, we define custom controls for the user and assign it to the toolbar variable, which is later on assigned to the controls attribute in the next slide.\n
  • * Placing the various map options from Google Map Source, and in the above code, rendering the Google Hybrid code in priority.\n* Google Hybrid. Google Physical. Google Street Maps. Google Satellite.\n* backgroundLayer contains holds the grey border, transparent polygon -&gt; &amp;#x201C;Total Landscape Area&amp;#x201D;, more expressively understood as &amp;#x201C;the area we are interested in&amp;#x201D;\n* polygonLayer contains either the &amp;#x201C;Before&amp;#x201D; we change the landscape polygons, each representing different landscape type (which is referred to by the scientists as a &amp;#x2018;Matrix&amp;#x2019;)\n* Or &amp;#x201C;After&amp;#x201D;.\n* Once we have &amp;#x201C;Total Landscape Polygon&amp;#x201D; defined, &amp;#x201C;Before polygons defined&amp;#x201D;, &amp;#x201C;After polygons&amp;#x201D; defined, we can compute &amp;#x201C;expected species loss&amp;#x201D;\n
  • * Here, you can visualize what I just explained.\n
  • * When we first define our Total Landscape Area polygon, we don&amp;#x2019;t need the backgroundLayer.* The backgroundLayer is intended to display the Total Landscape Area polygon already defined when the User is plotting out his &amp;#x201C;Before&amp;#x201D; or &amp;#x201C;After&amp;#x201D; polygons.\n
  • \n
  • \n
  • \n\n
  • \n\n
  • * The user interface comprising jQuery Mobile and OpenLayers could be completely replaced by native ones so land surveyors can work independent of 3G/Edge Network availability.\n* Sending field data into the server only when 3G/Edge Network are available.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • * MBTiles is a specification for storing tiled map data in SQLite databases for immediate use and for transfer. The files are designed for portability of thousands, hundreds of thousands, or even millions of standard map tile images in a single file.* This makes it easy to manage and share map tiles\n\n
  • \n\n
  • \n\n
  • \n
  • \n\n
  • \n\n
  • \n\n
  • \n\n
  • \n\n
  • \n\n
  • \n\n
  • \n
  • * There are interesting products and satellite services in the market that work with Android and iOS devices.\n* An example is TomTom. This is something I have yet to explore in detail but it is an interesting feature that would empower field work for scientists and land owners where terrestrial 3G network isn&amp;#x2019;t available.\n
  • \n\n
  • * Here&amp;#x2019;s one of the excel file holding all the formulas our scientist collaborators have painstakingly built.\n
  • \n
  • * Translation of excel formulas and vbscript (ack!) to Python\n
  • \n\n
  • \n\n
  • * The geography argument in our GeoDjango field is an important factor when considering the accuracy of the area in your defined Polygon.* The default is False. When we specify it to be True, area computation for the given Polygon or MultiPolygon assumes a surface area on an ellipsoidal earth. \n* This computation is done at the expense of some performance loss.\n* geography=False computes area ased on a flat, 2D polygon area.\n* LayerMapping versus directly assigning to \n\n
  • \n\n
  • \n\n
  • \n
  • \n
  • \n
  • \n
  • * First, I must say that django GIS&amp;#x2019;s documentation is kickass. If my presentation is not clear in anyway, I am pretty sure Django GIS documentation will help you out.\n* I have also listed a very comprehensive list of really good presentations relating to GeoDjango. If you are serious about GIS development, I encourage you to go through every single one of them.\n* I will, of course, upload this presentation, my little contribution to the fascinating world of GeoDjango online. You should be able to find it on the 2nd url by tomorrow.\n
  • * GIS is a really fascinating topic in the world of Python and Django.\n* I am new to the topic and have much to learn!\n* I will be happy to link up with any geodjango developers out there to share our skills and knowledge.\n\n
  • Transcript of "Saving Gaia with GeoDjango"

    1. 1. SAVING GAIAw/jQuery Mobile, OpenLayers and GeoDjango
    2. 2. CALVIN CHENG www.calvinx.comwww.od-eon.com/blogs/calvin
    3. 3. 1 Context 2 Technology Stack 3 Installation
    4. 4. 4 Django GeoModel & ORM 5 Geospatial Libraries 6 Utilities
    5. 5. 7 GeoAdmin 8 OpenLayers 9 jQuery (Mobile)
    6. 6. 10 Possibilities 11 Conclusion
    7. 7. 1Context
    8. 8. REDDCalculator.com (Beta) SEC (Closed Beta) LUC (Coming Soon!) ! ENVIRONMENTAL iTOOLS Spatially Explicit Scenario Analysis: Norway-Indonesia’s Forest Moratorium Species Extinction Calculator (SEC) Land Use Calculator (LUC)
    9. 9. SPECIES WHA--??Species Extinction Calculator (SEC)
    10. 10. 4% 4"&#()#*% 123# !"#$$%&()*"+,-./0" !"#$%
    11. 11. 3,000 2,250 Number of species 1,500 750 0 100 200 300 400 500 600 700 800 Area conserved (km sq) SPECIES-AREAMore habitats conserved (Area) = More fauna diversity (Species)
    12. 12. +)!) ) &,-! ) * "#$%$&( ( ) . &,- . ) "#$%$&( )
    13. 13. !"#$%&()3+(451"67(&/&012( !"#$%&()*+(,-.&(&/&012(
    14. 14. 0$1,23+4")56)."+4,2+"+*,7$2/+)$4"8$*.51 !"#$%&""(")*+,--,."+*"/#"+ N # $ % "i $ ! i i =1! - M ( ) #% "!$ j ) ) ) * &+, &+, j =1! - ) ) "#$%$&( "#$%$&(
    15. 15. MATRIX-CALIBRATED SPECIES-AREAAccounts for taxon-specific responses to different components of the landscape
    16. 16. EXAMPLE OF AN AREA?Conservation Forests, National Reserves etc Borneo 743,330 square kilometres
    17. 17. USERS AND USE-CASEConservation scientists and Land owners define areas which areconserved by plotting polygons on a map source (e.g. Google Maps).Areas which are slated for development are also marked.By quantifying area conservation goals (areas conserved before and afterdevelopment) and its impact on the loss of species, SEC is a tool whichhelps people make better trade-off decisions.
    18. 18. USERS AND USE-CASEREDDCalculator.com computes the socio-economic trade-offs andresults as a consequence of the Norway-Indonesia REDD+ Moratorium.
    19. 19. 2Technology Stack
    20. 20. Layers & Features (‘Vector Objects’) jQuery &OpenLayers User Interface Map SourceKML GML WKT GeoJSON GeoRSS Others Data (serialized) GeoDjango Python Logic C/C++ Libraries w/ Python Bindings & Datastore
    21. 21. 3Installation
    22. 22. (2.7)
    23. 23. (trunk)
    24. 24. sudo -u postgres psql -d yourdatabase_name -f { ... }/postgis.sqlsudo -u postgres psql -d yourdatabase_name -f { ... }/postgis_comments.sqlsudo -u postgres psql -d yourdatabase_name -f { ... }/spatial_ref_sys.sql
    25. 25. sudo -u postgres psql -d yourdatabase_name -f /opt/local/var/macports/software/postgis/1.5.2_1+postgresql90/opt/local/share/postgresql90/contrib/postgis-1.5/postgis.sqlsudo -u postgres psql -d yourdatabase_name -f /opt/local/var/macports/software/postgis/1.5.2_1+postgresql90/opt/local/share/postgresql90/contrib/postgis-1.5/postgis_comments.sqlsudo -u postgres psql -d yourdatabase_name -f /opt/local/var/macports/software/postgis/1.5.2_1+postgresql90/opt/local/share/postgresql90/contrib/postgis-1.5/spatial_ref_sys.sql
    26. 26. Portage existsBut here’s a magical way - https://github.com/stefantalpalaru/dotfiles/tree/master/Gentoo/custom%20overlay/portage/dev-db/postgis -custom postgis ebuild modified by Stefanadd ebuild to local ‘overlay’ (a gentoo terminology)emerge postgisedit configuration file in /etc/conf.d/postgis and add in the postgresqldatabase that you need ‘postgis-enabled’emerge --config postgis
    27. 27. # settings.pyINSTALLED_APPS = ( # Core Django Apps django.contrib.contenttypes, django.contrib.auth, django.contrib.sessions, django.contrib.sites, django.contrib.admin, django.contrib.admindocs, django.contrib.flatpages, django.contrib.humanize, django.contrib.comments, django.contrib.markup, django.contrib.webdesign, django.contrib.sitemaps, django.contrib.gis, # Custom Apps for REDDCalculator.com updatemodel, location, areadata, # Generic and reusable apps for most projects south, django_cherrypy_odeon, misc, cron, notify, log, homepage, userprofile,)
    28. 28. # settings.pyDATABASES = { default: { ENGINE: django.contrib.gis.db.backends.postgis, ‘NAME’: ‘yourdatabase_name’, USER: yourdatabase_user’, PASSWORD: yourdatabase_password, HOST: your_ip_or_domain, }}
    29. 29. 4 DjangoGeoModel & APIs
    30. 30. from django.contrib.gis.db import models
    31. 31. from django.contrib.gis.db import modelsclass Province(models.Model): name = models.CharField(max_length=256) name_long = models.CharField(max_length=256, null=True) island = models.ForeignKey(Location, null=True) geom = models.MultiPolygonField(srid=4326, null=True) objects = models.GeoManager()
    32. 32. from django.contrib.gis.db import modelsclass Province(models.Model): name = models.CharField(max_length=256) name_long = models.CharField(max_length=256,null=True) island = models.ForeignKey(Location, null=True) geom = models.MultiPolygonField(srid=4326, null=True, geography=True) objects = models.GeoManager()
    33. 33. What are the geometry fields available to us in django?
    34. 34. GEOS: OGC simple feature implementation PointField MultiPointField LineStringField MultiLineStringField PolygonField MultiPolygonField GeometryCollectionField
    35. 35. PointField (x, y)
    36. 36. MultiPointField((x1, y1), (x2, y2), (x3, y3))
    37. 37. LineStringField((x1, y1), (x2, y2), (x3, y3), (x4, y4))
    38. 38. MultiLineStringField(((x1, y1), (x2, y2), (x3, y3), (x4, y4)),((x5, y5), (x6, y6)))
    39. 39. PolygonField((x1, y1), (x2, y2), (x3, y3), (x4, y4), (x1, y1))
    40. 40. MultiPolygonField((x1, y1), (x2, y2), (x3, y3), (x4, y4), (x1, y1)), ((x5, y5), (x6, y6), (x7, y7), (x5, y5)))
    41. 41. GeometryCollectionField((x1, y1), (x2, y2), (x3, y3), (x4, y4), (x1, y1)), ((x5, y5), (x6, y6), (x7, y7)), ((x8, y8)))
    42. 42. Practical Example #1: Species Extinction Calculator
    43. 43. Practical Example #2: REDDCalculator.com
    44. 44. from django.contrib.gis.gdal import datasource, spatialreferencefrom django.contrib.gis.geos import multipolygon, geosgeometryshapefile = os.path.abspath(os.path.join(os.path.dirname(__file__),../../../../proj_public/media/files/giveshapefile.shp))ds = DataSource(shapefile)layer = ds[0] # Checked in Python shell that this has only 1 layer
    45. 45. from django.contrib.gis.db import modelsclass AreaData(models.Model): district = models.ForeignKey(District) # District is FKed to Province land_use = models.IntegerField(choices=VEGETATION_TYPES) allocation = models.IntegerField(choices= CHOICES[LAND_ALLOCATION]) intact = models.IntegerField() concession = models.CharField(max_length=255, blank=True) peat = models.CharField(max_length=255, blank=True) area = models.FloatField(help_text=hectares) fid = models.IntegerField() # ... and any other. geom = models.MultiPolygonField(srid=4326, null=True) objects = models.GeoManager()
    46. 46. area = AreaData.objects.get(fid = feature.fid)geometry_name = feature.geom.geom_type.namegeometry_object = feature.geom.transform(SpatialReference(4326),clone=True).geos# update our area object with the geom data from our shape fileif geometry_name == Polygon: # If our geometry object is a Polygon, we cast as MultiPolygon first geometry_object = MultiPolygon(geometry_object)area.geom = geometry_objectarea.save()
    47. 47. 5Geospatial Libraries
    48. 48. GEOS GDAL PROJ.4 Mapnik GeoIP
    49. 49. GEOS Geometry Engine - Open SourceWhat does it do? Works with django GIS app to give us GeoModels!
    50. 50. GEOS Geometry Engine - Open SourceOther Details Required dependency for django GIS app C++ Implements OGC simple feature for SQL specification OGC = Open Geospatial Consortium e.g. sudo port install geos
    51. 51. GDAL Geospatial Data Abstraction LibraryWhat does it do? Reads and writes to a variety of vector file formats, e.g. shp
    52. 52. GDAL Geospatial Data Abstraction LibraryOther Details C/C++ Implements OGC simple feature specification e.g. sudo port install gdal
    53. 53. PROJ.4 Cartographic Projections LibraryWhat does it do? Manages spatial reference systems and projections e.g. the mercator projection commonly used by Google Maps, MapQuest etc
    54. 54. PROJ.4 Cartographic Projections LibraryOther Details Required dependency for PostGIS C Ensures that geometry objects stored in PostGIS use the appropriate projection sudo port install libproj4
    55. 55. Mapnik Toolkit for developing Mapping ApplicationsWhat does it do? For the control freak in you :-) Build your own maps, control and customize every single detail
    56. 56. Mapnik Toolkit for developing Mapping ApplicationsOther Details C++ Uses boost for python interface No py27-mapnik yet. Solution? Create your own local portfile using py26-mapnik sudo port install py27-mapnik
    57. 57. GeoIP Locate users by IP addressWhat does it do? Determines the location of app user via user’s IP address
    58. 58. GeoIP Locate users by IP addressOther Details C No py27-geoip yet. Solution? Create your own local portfile using py26-geoip sudo port install libgeoip py27-libgeoip
    59. 59. 6Utilities
    60. 60. LayerMapping./manage.py ogrinspect ./manage.py ogrinfo
    61. 61. ./manage.py ogrinspect and LayerMapping
    62. 62. $ ./manage.py ogrinspect ../proj_public/media/files/redd_data/kalimantan_universe.shp Location --srid=4326 --mapping# This is an auto-generated Django model module created by ogrinspect.from django.contrib.gis.db import modelsclass Location(models.Model): island = models.CharField(max_length=50) geom = models.MultiPolygonField(srid=4326) objects = models.GeoManager()# Auto-generated `LayerMapping` dictionary for Location modellocation_mapping = { island : island, geom : MULTIPOLYGON,}
    63. 63. # models.pyfrom django.contrib.gis.db import modelsclass Location(models.Model): island = models.CharField(max_length=50) geom = models.MultiPolygonField(srid=4326) objects = models.GeoManager()# Auto-generated `LayerMapping` dictionary for Kalimantan modellocation_mapping = { island : island, geom : MULTIPOLYGON,}
    64. 64. import osfrom django.contrib.gis.utils import LayerMappingfrom django.contrib.gis.gdal import DataSourcefrom location.models import Location, location_mappingshapefile = os.path.abspath(os.path.join(os.path.dirname(__file__),../../../../proj_public/media/files/kalimantan_universe.shp))ds = DataSource(shapefile)layer = ds[0] # Checked in Python shell that this has only 1 layerkalimantan = LayerMapping(Location, shapefile, location_mapping, transform = True, encoding=iso-8859-1)kalimantan.save(strict = True, verbose = True)
    65. 65. 7GeoAdmin
    66. 66. from django.contrib.gis import admin as geoadminfrom location.models import Locationadmin.site.register(Location, geoadmin.GeoModelAdmin)
    67. 67. 8OpenLayers
    68. 68. “ OpenLayers is a pure JavaScript library for displaying map data in most modern web browsers, with no server-side dependencies. ”
    69. 69. 9 jQuery(mobile)
    70. 70. “ jQuery Mobile is a Touch-Optimized Web Framework for Smartphones and Tablets. ”
    71. 71. + +
    72. 72. $(document).ready(function(){ $(#map_delete).hide() // Define a custom styles which will be used by our polygon layer var style = $.extend(true, {}, OpenLayers.Feature.Vector.style[default]) // get a copy of default style style.pointRadius = 15 var styleMap = new OpenLayers.StyleMap({"default": style}) // Define a custom style which will be used by our background layer var backgroundStyle = $.extend(true, {}, OpenLayers.Feature.Vector.style[default]) // get copy backgroundStyle.fillOpacity = "0.1" backgroundStyle.fillColor = "#000000" backgroundStyle.strokeColor = "#999999" var backgroundStyleMap = new OpenLayers.StyleMap({"default": backgroundStyle}) // Define our background layer backgroundLayer = new OpenLayers.Layer.Vector("Background Layer", {styleMap:backgroundStyleMap} )
    73. 73. // Define our vector layerpolygonLayer = new OpenLayers.Layer.Vector("Polygon Layer", {styleMap: styleMap})polygonLayer.events.on({ featureselected: show_delete_button, featureunselected: hide_delete_button})// Define our toolbar, panels and associated controls// pinch and zoom is defined in touch navigationvar touchNavigation = new OpenLayers.Control.TouchNavigation({ dragPanOptions: { interval: 100, enableKinetic: true }})// define our zoom panelvar zoomPanel = new OpenLayers.Control.ZoomPanel()// define our layer switchervar layerSwitcher = new OpenLayers.Control.LayerSwitcher()
    74. 74. // controlsdrawControl = new OpenLayers.Control.DrawFeature(polygonLayer, OpenLayers.Handler.Polygon, { displayClass: olControlDrawFeaturePolygon});selectControl = new OpenLayers.Control.SelectFeature(polygonLayer, { clickout: true });editControl = new OpenLayers.Control.ModifyFeature(polygonLayer, { selectControl: selectControl, vertexRenderIntent: temporary, displayClass: olControlModifyFeature});// define our custom control tool barvar toolbar = new OpenLayers.Control.Panel({displayClass: olControlEditingToolbar});toolbar.addControls([ //this control is just there to be able to deactivate the drawing tools and pan the map drawControl, editControl, new OpenLayers.Control({ displayClass: olControlNavigation })]);
    75. 75. // Define Map Layersvar gphy = new OpenLayers.Layer.Google("Google Physical", {type: google.maps.MapTypeId.TERRAIN});var gmap = new OpenLayers.Layer.Google("Google Streets", {numZoomLevels: 20});var ghyb = new OpenLayers.Layer.Google("Google Hybrid", {type: google.maps.MapTypeId.HYBRID, numZoomLevels: 20});var gsat = new OpenLayers.Layer.Google("Google Satellite", {type: google.maps.MapTypeId.SATELLITE, numZoomLevels: 20});// Feed the controls and layers defined above to our map objectmap = new OpenLayers.Map({ div: map, controls: [zoomPanel, touchNavigation, layerSwitcher, toolbar], layers: [ghyb, gphy, gmap, gsat, backgroundLayer, polygonLayer]})
    76. 76. // Google.v3 uses EPSG:900913 as projection, so we have to transform our coordinates mapProjection = map.getProjectionObject() map.setCenter(new OpenLayers.LonLat(map_center_lon, map_center_lat).transform(defaultProjection,mapProjection), map_zoom) // if theres a Total Landscape Area polygon, we will draw it on the backgroundLayer. if (background_coords_json){ var pointList = [] for (var i=0; i<background_coords_json.length; i++) { var point = new OpenLayers.Geometry.Point(background_coords_json[i][0], background_coords_json[i][1]) pointList.push(point.transform(defaultProjection, mapProjection)); } var linearRing = new OpenLayers.Geometry.LinearRing(pointList) totalAreaPolygon = new OpenLayers.Feature.Vector(new OpenLayers.Geometry.Polygon([linearRing])) backgroundLayer.addFeatures([totalAreaPolygon]) }
    77. 77. // draw the existing features into the polygonLayer var loaded_features = [] for (var i=0; i<list_of_coords_json.length; i++) { var pointList = [] for (var j=0; j<list_of_coords_json[i].length; j++) { var point = new OpenLayers.Geometry.Point(list_of_coords_json[i][j][0], list_of_coords_json[i][j][1]) pointList.push(point.transform(defaultProjection, mapProjection)); } var linearRing = new OpenLayers.Geometry.LinearRing(pointList) var polygonFeature = new OpenLayers.Feature.Vector(newOpenLayers.Geometry.Polygon([linearRing]), {fid: list_of_coords_ids_json[i]}) loaded_features.push(polygonFeature) } polygonLayer.addFeatures(loaded_features) polygonLayer.events.on({ "featuremodified": polygonSave, "featureadded": polygonSave, "vertexmodified": adjustVertex, "sketchstarted": adjustVertex, "sketchmodified": adjustVertex }) // confirmation dialog $(#project_map_delete_polygon_yes).click(delete_polygon)})/* END of $(document).ready() */
    78. 78. 10Possibilities
    79. 79. 1. Offline Maps for Field Usage 2. Remote Controlled UAV for data collection3. Crunch collected data and generate visualizations
    80. 80. Offline Maps for Field Usage
    81. 81. Layers & Features (Saved Locally, Syncing) NativeiOS/Android Native? Map Source (Local Map Tiles)KML GML WKT GeoJSON GeoRSS Others Data Transfer GeoDjango Python Logic C/C++ Libraries w/ Python Bindings & Datastore
    82. 82. The Problem with iOS MapKit FrameworkMKMapView class implements Google Maps API nativelyUnfortunately, Google Maps terms of use prevents us from cachingor storing map tiles locally
    83. 83. Alternative? https://github.com/route-me/route-meSupports many map sources - OpenStreetMap, MicrosoftVirtualEarth, CloudMade, OpenAerialMap, OpenCycleMap,SpatialCloudOpenStreetMap permits map tile caching and local storageSupports 2 offline, database-backed formats DBMap and MBTiles
    84. 84. A Comparison MKMapKit Route-Me Embed in iOS App YES YES Interaction YES YES Geolocalization YES YESDifferent Map Sources NO YES Offline Maps NO YES
    85. 85. The Problem with Android Google Maps FrameworkRenders Google Maps of courseSupport for alternative map sources is important
    86. 86. Alternative? https://github.com/camptocamp/android-gis.git Supports many map sources - OpenStreetMap, Microsoft VirtualEarth, CloudMade, OpenAerialMap, OpenCycleMap, SpatialCloud OpenStreetMap permits map tile caching and local storage Supports 2 offline, database-backed formats DBMap and MBTiles
    87. 87. Benefits Fast (obviously, since Map Tiles are stored locally) 100% Offline Stored as blob data for SQLite Client Access
    88. 88. Remote Controlled UAVs?
    89. 89. Collect Real Data => Effects on Landscape Change on Species e.g. obtain actual change in Forest Boundaries(instead of just predicting on map on current prediction tool)
    90. 90. Prediction is great.But we want real world data from primary forests!
    91. 91. http://www.youtube.com/watch?v=xjLXL72Medw&feature=autoshare
    92. 92. Crunch Collected Data => 2 possible approaches- Decoupled R and Python- Integrated Approach RPy Option
    93. 93. Decouple R and PythonPython retrieves data from django/postgresqlPython converts data shapefiles, images etcLaunches R given a *.r file with Scientist’s parametersR does the analysisPython does post-processing, projecting data etc
    94. 94. Integrated R & Python with RPy R and Python closely coupled RPy and RPy2 provides an interface to R Programming Language Python retrieves and organizes parameters from django/postgresql RPy module imported and R commands are executed within thePython script file
    95. 95. 11Conclusion
    96. 96. Getting a GPS fix via Satellite?
    97. 97. Mathematical Modelling, Scenario Analysis for Environmental Policies
    98. 98. REDDCalculator.com (Live) SEC (Closed Beta) LUC (Coming Soon!) ! ENVIRONMENTAL iTOOLS Spatially Explicit Scenario Analysis: Norway-Indonesia’s Forest Moratorium Species Extinction Calculator (SEC) Land Use Calculator (LUC)
    99. 99. Species Extinction Calculator and REDDCalculator:Mathematical Models and Formulas by Koh et. al are completelytranslated into PythonLand Use Calculator:Ditto. Coming Soon!
    100. 100. Key Takeaways
    101. 101. Mapnik is useful for rasterizing vector objects when we encounter largedatasets (in our case, 200 MB+ of protected areas in Kalimantan).(jQuery Mobile) Web App can benefit from optimization through djangocaching. To be done right after EuroPython! And besides jQuery MobileBeta 1 has just been released on Monday (20 June 2011)! :-)Usability and App Responsiveness can possibly be improved, Out-Field/Offline Benefits attainable, if we move interface elements and Map Tilesinto a native app (Android or iOS).
    102. 102. geography=True/False in a GeoDjango field2 ways to import data from a given GIS Shape File(1) LayerMapping(2) Direct Assignment to obj.geom attribute as mentioned.
    103. 103. “ Advancing Environmental Sciences: helping people, scientists, NGOs and governments make practical Land Area Usage and Conservation Decisions using jQuery (Mobile), OpenLayers and GeoDjango! ”
    104. 104. Gotchas?
    105. 105. Make sure Projections/Datum in your shape file source correspondswith your geometry field’s SRID and Map SourceUsing LayerMapping?transform = TrueDirectly creating or saving into an object’s attribute?geometry_object = feature.geom.transform(SpatialReference(4326),clone=True).geosarea.geom = geometry_objectarea.save()
    106. 106. Credits
    107. 107. + North-South FundProf. Koh Lian Pin Prof. Matthew Hansen Prof. Holly GibbsETH Zurich, Dept of Env Sciences South Dakota State University Stanford Dept of EnvironmentalNUS, Dept of Biological Sciences GIS Center of Excellence Earth System Sciences
    108. 108. Stefan Talpalaru Liviu Bogdan Chiributa That’s Me! :-)Odeon Consulting Group Odeon Consulting Group Odeon Consulting Group + ........
    109. 109. Previous GeoDjango Presentations + my little contribution www.geodjango.org/presentations www.od-eon.com/presentations
    110. 110. Questions and Discussion welcome! calvin.cheng@od-eon.com

    ×