0
The Centre for Australian Weather and Climate Research
A partnership between CSIRO and the Bureau of Meteorology
Dynamic v...
The Centre for Australian
Slides & code
github.com/pfctdayelise/dapbook
The Centre for Australian
What we have...
The Centre for Australian
What we have...
The Centre for Australian
The problem
Can't zoom
Can't pan
Can't layer
IPython Notebook makes it easy to explore datasets ...
The Centre for Australian
...what we want
The Centre for Australian
What is this data?
NetCDF file – common format for gridded data (could easily have been HDF5)
Co...
The Centre for Australian
MetEye
The Centre for Australian
OpenLayers
PyDAP +
WMS response
How to get there?
NetCDF
The Centre for Australian
IPython notebook
“a web-based interactive computational environment where you can
combine code e...
The Centre for Australian
IPython notebook
pip install ipython
pip install tornado pyzmq # needed for notebook, not ipytho...
The Centre for Australian
IPython notebook
Rich display – essentially more types of __repr__
HTML
JSON
PNG, JPEG, SVG
LaTe...
The Centre for Australian
OpenLayers
PyDAP +
WMS response
How to get there?
NetCDF
The Centre for Australian
JavaScript mapping libraries
Leaflet and OpenLayers:
Specify layers as “tile layers” or “WMS ti...
The Centre for Australian
WMS
The Centre for Australian
Web Map Service
Supports 2+ request types – GetMap, GetCapabilities
http://test.pydap.org/coads....
The Centre for Australian
OpenLayers
PyDAP +
WMS response
How to get there?
NetCDF
The Centre for Australian
PyDAP
2 parts: OPeNDAP server, OPeNDAP client
Client
• “Lazy loading” of data from any OPeNDAP s...
The Centre for Australian
PyDAP
The Centre for Australian
PyDAP
pip install pydap pydap.handlers.netcdf pydap.responses.wms
paster create -t pydap myserve...
The Centre for Australian
PyDAP
The Centre for Australian
PyDAP
The Centre for Australian
PyDAP
The Centre for Australian
Projections 101
Mapping libraries can reproject points and vectors, but not map tiles...
Base la...
The Centre for Australian
Projections 101
CRS EPSG:4326 EPSG:3857
(aka EPSG:900913)
Example of Geographic coordinate
syste...
The Centre for Australian
Option 1: Render overlays in EPSG:3857
Base layer
Overlays
PyDAP
WMS
4326
OSM
Google
Yahoo
Bing
...
The Centre for Australian
Option 2: Pick base layer in EPSG:4326
Base layer
Overlays
MetaCarta
PyDAP
WMS
4326
4326
Cascadi...
The Centre for Australian
Cascading WMS
• MapServer
• C
• MIT license
• GeoServer + GeoWebCache
• Java
• GPL, LGPL license...
The Centre for Australian
Putting it all together
The Centre for Australian
The Centre for Australian
template.html
The Centre for Australian
Back to the notebook
The Centre for Australian
Other options – GeoDjango?
First impression -
seems to be a lot more about vector layers than gr...
The Centre for Australian
The future – HTML5 backend for matplotlib?
The Centre for Australian
Credits
With thanks to:
Nathan Faggian, for the original leavis, and willingness to share his id...
The Centre for Australian Weather and Climate Research
A partnership between CSIRO and the Bureau of Meteorology
Brianna L...
Upcoming SlideShare
Loading in...5
×

Dynamic viz in the IPython Notebook

1,703

Published on

Slides for a talk at PyCon AU 2013. Integrating PyDAP + WMS + OpenLayers + IPython Notebook.
Video: http://www.youtube.com/watch?v=YJqBGi48RAM
The IPython Notebook is a powerful web app for exploring ideas and data sets with Python. It has excellent integration with Matplotlib, giving the user highly customisable static plots with ease. But for larger data sets, a static plot may not be ideal - the ability to pan, zoom, choose dynamic layers and sample the data at particular points would be nice. This talk will demonstrate just how easy it is to integrate a Web Map Service/client such as Pydap/Leaflet.js into the IPython Notebook.

Published in: Technology, Art & Photos
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,703
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Dynamic viz in the IPython Notebook"

  1. 1. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Dynamic visualisation in the IPython Notebook Brianna Laugher Hobart, Tasmania July 2013, PyCon AU www.cawcr.gov.au
  2. 2. The Centre for Australian Slides & code github.com/pfctdayelise/dapbook
  3. 3. The Centre for Australian What we have...
  4. 4. The Centre for Australian What we have...
  5. 5. The Centre for Australian The problem Can't zoom Can't pan Can't layer IPython Notebook makes it easy to explore datasets and plot them with matplotlib... ...but for gridded datasets, it really would be nice to have them on a map!
  6. 6. The Centre for Australian ...what we want
  7. 7. The Centre for Australian What is this data? NetCDF file – common format for gridded data (could easily have been HDF5) Commonly used by earth science communities Sample TAS MinT grid from the Australian Digital Forecast Database produced by BoM, see http://www.bom.gov.au/catalogue/data-feeds.shtml for details and to download sample grids ADFD grids feed into MetEye - http://www.bom.gov.au/australia/meteye/
  8. 8. The Centre for Australian MetEye
  9. 9. The Centre for Australian OpenLayers PyDAP + WMS response How to get there? NetCDF
  10. 10. The Centre for Australian IPython notebook “a web-based interactive computational environment where you can combine code execution, text, mathematics, plots and rich media into a single document” • Inspired by Mathematica, SAGE notebooks • Built-in support for numpy, matplotlib • Half-interpreter, half-script • Great way to “show your work” • Useful for tuning fiddly APIs (ahem matplotlib) • Perfect for tutorials! Run a local server, and/or Publish your notebook as .ipynb and use http://nbviewer.ipython.org/
  11. 11. The Centre for Australian IPython notebook pip install ipython pip install tornado pyzmq # needed for notebook, not ipython shell ipython notebook --pylab inline
  12. 12. The Centre for Australian IPython notebook Rich display – essentially more types of __repr__ HTML JSON PNG, JPEG, SVG LaTeX Expect this API to change (improve) in future versions, esp. for JavaScript Make a class “Notebook compatible” by adding html, png etc properties: from IPython.display import Image class Foo(object): …. @property def png(self): Return Image(...)
  13. 13. The Centre for Australian OpenLayers PyDAP + WMS response How to get there? NetCDF
  14. 14. The Centre for Australian JavaScript mapping libraries Leaflet and OpenLayers: Specify layers as “tile layers” or “WMS tile layers” (different APIs) Can set projection as required Can reproject points and vectors (but not map tiles) on-the-fly
  15. 15. The Centre for Australian WMS
  16. 16. The Centre for Australian Web Map Service Supports 2+ request types – GetMap, GetCapabilities http://test.pydap.org/coads.nc.wms?SST[0]&LAYERS=SST&TRANSPARENT=true&FORMAT=image %2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG %3A900913&BBOX=-180,-90,1252164.27125,1252254.27125&WIDTH=256&HEIGHT=256 REQUEST=GetMap SERVICE=WMS VERSION=1.1.1 LAYERS=MinT_SFC SRS=EPSG:900913 BBOX=-180,-10018844.17,10018574.17,-90 FORMAT=image/png TRANSPARENT=true HEIGHT=256 WIDTH=256
  17. 17. The Centre for Australian OpenLayers PyDAP + WMS response How to get there? NetCDF
  18. 18. The Centre for Australian PyDAP 2 parts: OPeNDAP server, OPeNDAP client Client • “Lazy loading” of data from any OPeNDAP server into numpy (incl. slices, subsets) • Don't have to care about file format of original data Server • WSGI app • Handlers (input): NetCDF, HDF5, SQL, CSV, remote!, ... • Responses (output): DAS/DDS/DODS, HTML, ASCII, WMS, KML, XLS, ...
  19. 19. The Centre for Australian PyDAP
  20. 20. The Centre for Australian PyDAP pip install pydap pydap.handlers.netcdf pydap.responses.wms paster create -t pydap myserver cp mydata.nc myserver/data/ # starts a server like http://test.pydap.org/ # run server with 4 workers, better for serving up map tiles gunicorn_paster -w 4 -b 127.0.0.1:8001 myserver/server.ini
  21. 21. The Centre for Australian PyDAP
  22. 22. The Centre for Australian PyDAP
  23. 23. The Centre for Australian PyDAP
  24. 24. The Centre for Australian Projections 101 Mapping libraries can reproject points and vectors, but not map tiles... Base layers and overlays need to be requested in the same projection!
  25. 25. The Centre for Australian Projections 101 CRS EPSG:4326 EPSG:3857 (aka EPSG:900913) Example of Geographic coordinate system (represents a globe/part of a globe) Projected coordinate system, “Spherical Mercator” (represents a map) Used by “GIS enthusiasts”, Metacarta “almost all free and commercial tile providers” (Google, OSM, Yahoo, Bing) Coords as Lat-lons in decimal degrees “Metres” PyDAP WMS can serve map tiles in √ OpenLayers supports by default √ √ Leaflet supports by default √ But... issue #1207 “EPSG 4326 Support Broken for TileLayers” and... CloudMade doesn't provide WMS. √
  26. 26. The Centre for Australian Option 1: Render overlays in EPSG:3857 Base layer Overlays PyDAP WMS 4326 OSM Google Yahoo Bing 3857 Cascading WMS 3857
  27. 27. The Centre for Australian Option 2: Pick base layer in EPSG:4326 Base layer Overlays MetaCarta PyDAP WMS 4326 4326 Cascading WMS 4326 OSM Google Yahoo Bing 3857
  28. 28. The Centre for Australian Cascading WMS • MapServer • C • MIT license • GeoServer + GeoWebCache • Java • GPL, LGPL licenses • http://maps.opengeo.org/geowebcache/demo • Includes OpenStreetMap, NASA “Blue Marble”
  29. 29. The Centre for Australian Putting it all together
  30. 30. The Centre for Australian
  31. 31. The Centre for Australian template.html
  32. 32. The Centre for Australian Back to the notebook
  33. 33. The Centre for Australian Other options – GeoDjango? First impression - seems to be a lot more about vector layers than gridded data No particular mention of netcdf, gridded data, WMS
  34. 34. The Centre for Australian The future – HTML5 backend for matplotlib?
  35. 35. The Centre for Australian Credits With thanks to: Nathan Faggian, for the original leavis, and willingness to share his ideas Leafvis – a WMS-ish app, DIY map tile rendering. https://github.com/nfaggian/leafvis And also: Roberto De Almeida James Sofra Danielle Madeley Roald de Wit Credits Map tiles photo: “Carcassonne” by Tom & Katrien, licensed CC-BY-SA. http://www.flickr.com/photos/inferis/283379928/ PyDAP slide: by PyCon 2007 talk by Roberto De Almeida. http://www.scribd.com/doc/2864/PyCon-2007
  36. 36. The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Brianna Laugher Email: b.laugher@bom.gov.au Web: www.cawcr.gov.au Slides & code: https://github.com/pfctdayelise/dapbook Thank you www.cawcr.gov.au
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×