Successfully reported this slideshow.
Your SlideShare is downloading. ×

PyWPS-4.0.0

PyWPS-4.0.0

Download to read offline

Talk about PyWPS-4.0.0 at ISPRS 2016, Prague, related paper is published in conference proceedings.

Talk about PyWPS-4.0.0 at ISPRS 2016, Prague, related paper is published in conference proceedings.

Advertisement
Advertisement

More Related Content

Advertisement
Advertisement

PyWPS-4.0.0

  1. 1. PyWPS 4.0.0
  2. 2. Introduction
  3. 3. What is PyWPS • PyWPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium • PyWPS is written in Python • Started in the Spring of 2006 • Supports all available tools in Python for geospatial operations • http://pywps.org
  4. 4. What PyWPS is NOT • complicated • a client • a GUI • a server with pre-installed processes
  5. 5. OGC Web Processing Service
  6. 6. The OGC Web Processing Service • OGC open web standard for remote geo-spatial processing. • Integrated with web data services: WFS, WCS. • Three basic requests: • GetCapabilities • DescribeProcess • Execute • Three basic input/output classes: • Literal • Complex - for geo-spatial data and services • BouningBox - for geo-spatial data extent
  7. 7. The OGC Web Processing Service http://www.slideshare.net/TheodorFoerster/restful-web-processing-service
  8. 8. Essential PyWPS Functionality • Communication bridge with WPS. • Fetches input data referenced in Execute requests. • Creates a container for the process instance. • Manage processes: communication, reporting, logging. • Data storage for final data outputs. • Client notification, status reporting
  9. 9. History
  10. 10. 2006 • Jachym Cepicky starts working on a project to link GRASS with UMN Mapserver. • Scholarship funded by the German Foundation for Environment. • September: PyWPS is presented at FOSS4G 2006. • October: Project homepage hosted by Intevation GmbH. • November: Version 1.0.0 is released; other developers join the project.
  11. 11. 2007 - 2010 • Version 2.0.0 released in October of 2007 • Introduced the Process class; • WPS plugin for OpenLayers; • Improved stability. • Version 3.0.0 released in November of 2008 • New implementation of the WPS 1.0.0 standard. • JavaScript WPS client introduced in 2009 with PyWPS 3.1.0.
  12. 12. 2011 • The WPS Cookbook is published. • A deliverable of the NETMAR project. • Introduces the WPS standard. • Presents PyWPS as implementation. • Still today a valuable introduction to WPS. • Practical examples of SOAP envelopes and Taverna Workflow management system • Staple GIS software in Industry and Academia. • PyWPS featuring regularly in scientific publications.
  13. 13. 2012-2014 • 2013 FOSS4G-CEE: • New implementation starts being discussed; • First lines of code are written. • 2014 WPS Workshop organised by the Luxembourg Institute of Science and Technology (LIST): • Broad architecture for PyWPS-4 devised; • Move to a new Licence discussed (MIT); • Application to OSGeo
  14. 14. 2015 • Development picks ups throughout 2015: • Student at LIST; • Google Summer of Code (GSoC). • PyWPS enters the OSGeo incubation process in April of 2015.
  15. 15. 2016 • Big push from OSGeo mentorship (Tom Kralidis) • Consolidation of documentation and websites. • PyWPS.org goes live. • More successful GSoC applications. • Official release of PyWPS 4.0.0 some time this Summer ...
  16. 16. PyWPS-4
  17. 17. Motivation for PyWPS-4 • PyWPS is now a decade old. • Python version 3 taking over. • New bindings for other libraries (e.g PyGRASS). • New data formats (e.g. GeoJSON, KML, TopoJSON). • Version 2.0 of the WPS standard. • Less restrictive licence (MIT).
  18. 18. New Technologies • lxml • Flask / Werkzeug • WSGI • Multiprocessing • PyPy • Jython • OWSLib
  19. 19. Data Validation • A four level approach: • No validation - always considered valid. • Simple validation - uses just the mimeType. • Strict validation - attempts to open Complex inputs with GDAL, comparing with mimeType. • Very strict validation - uses an XML schema. • Plus custom validation, already including: ESRI Shapefile, GeoJSON, GML, GeoTIFF.
  20. 20. Process Containerising • WPS server should run Execute requests concurrently. • but in totally insulated fashion; • no shared resources or data. • PyWPS-3: • a temporary folder is created for each Execute; • resulting data is moved to publishing folder; • temporary folder is deleted at execution end. • PyWPS-4 aims at a safer approach (e.g Docker, vagrant)
  21. 21. Asynchronous Execution • PyWPS can process various Execute requests in parallel: • configurable number; • plus a queue of waiting requests. • PyWPS-4 now uses the Multiprocessing module: • os.fork() abandoned; • PyWPS should now run on Windows too. • Process metadata now stored in a database (SQLite, PostgreSQL - logging, request queueing, etc). - GSoC project • Local database to support WPS 2.0.
  22. 22. Future Work • New requests in WPS 2.0: Pause, Release, Delete. • Improved security with process containerising. • Administrative web interface. • External services to publish outputs. • REST API • Support for other languages beyond English.
  23. 23. Administration interface and REST API
  24. 24. OSGeo Incubation
  25. 25. Application to OSGeo • Motion to enter incubation approved in April of 2015. • Process mentored by Tom Kralidis. • PyWPS-3 to feature in the next OSGeoLive release.
  26. 26. Governance • PyWPS now has a Project Steering Committee (PSC) • currently composed by 5 members • PSC sets the roadmap, technical standards, release schedule, code review, ... • Monthly meetings (IRC)
  27. 27. Community • Mailing list https://lists.osgeo.org/mailman/listinfo/pywps-dev • 35 members • IRC #geopython at irc.freenode.net server • Gitter https://gitter.im/PyWPS • Attending code sprints, conferences, ...
  28. 28. Example
  29. 29. Installing # Checkout *pywps* git clone git@github.com:geopython/pywps.git pywps # Checkout *pywps-demo* git clone git@github.com:geopython/pywps-demo.git pywps-demo # Install dependencies (*requirements.txt*) cd pywps && pip install -r requirements-dev.txt # Install pywps sudo python setup.py install # Run the demo server cd ../pywps-demo && python demo.py
  30. 30. Folder structure pywps pywps-demo +-- demo.py +-- LICENCE.txt +-- processes <- working folder | +-- area.py | +-- bboxinout.py | `-- ... +-- pywps.cfg <- configurations +-- requirements.txt +-- server.py <- run server +-- setup.py +-- static +-- templates `-- tests
  31. 31. Process structure from pywps import Process, ComplexInput, LiteralOutput, ... class Area(Process): def __init__(self): inputs = [ComplexInput('layer', 'Layer', ... outputs = [LiteralOutput('area', 'Area', ... super(Area, self).__init__( self._handler, identifier='area', title='Process Area', inputs=inputs, outputs=outputs, store_supported=True, status_supported=True) def _handler(self, request, response):
  32. 32. The handler method def _handler(self, request, response): from shapely.geometry import shape with temp_dir() as tmp: input_gml = request.inputs['layer'][0].file input_geojson = os.path.join(tmp, 'input.geojson') subprocess.check_call(['ogr2ogr', '-f', 'geojson', str(input_geojson), input_gml]) with open(input_geojson, 'rb') as f: data = json.loads(f.read()) for feature in data['features']: geom = shape(feature['geometry']) feature['area'] = geom.area response.outputs['area'].data = [feature['area'] ... return response
  33. 33. The PyWPS Philosophy
  34. 34. Bikes Jachym's Luís'
  35. 35. Bikes
  36. 36. Bikes
  37. 37. Bikes
  38. 38. The PyWPS Philosophy • Versatility: PyWPS can be whatever you want. • Ease of use: you only need to known how to pedal.
  39. 39. Thank you ! Homepage: http://pywps.org/ Source code, download: https://github.com/geopython/pywps Twitter: @PyWPS
  40. 40. Licence • Presenstation licence CC-Attribution-ShareAlike • http://creativecommons.org/licenses/by-sa/2.5/

×