SlideShare a Scribd company logo
1 of 40
Download to read offline
PyWPS 4.0.0
Introduction
What is PyWPS
• PyWPS is an implementation of the Web Processing Service
standard from the Open Geospatial Consortium
• PyWPS is written in Python
• Started in the Spring of 2006
• Supports all available tools in Python for geospatial operations
• http://pywps.org
What PyWPS is NOT
• complicated
• a client
• a GUI
• a server with pre-installed processes
OGC Web Processing Service
The OGC Web Processing Service
• OGC open web standard for remote geo-spatial processing.
• Integrated with web data services: WFS, WCS.
• Three basic requests:
• GetCapabilities
• DescribeProcess
• Execute
• Three basic input/output classes:
• Literal
• Complex - for geo-spatial data and services
• BouningBox - for geo-spatial data extent
The OGC Web Processing Service
http://www.slideshare.net/TheodorFoerster/restful-web-processing-service
Essential PyWPS Functionality
• Communication bridge with WPS.
• Fetches input data referenced in Execute requests.
• Creates a container for the process instance.
• Manage processes: communication, reporting, logging.
• Data storage for final data outputs.
• Client notification, status reporting
History
2006
• Jachym Cepicky starts working on a project to link GRASS with UMN
Mapserver.
• Scholarship funded by the German Foundation for Environment.
• September: PyWPS is presented at FOSS4G 2006.
• October: Project homepage hosted by Intevation GmbH.
• November: Version 1.0.0 is released; other developers join the
project.
2007 - 2010
• Version 2.0.0 released in October of 2007
• Introduced the Process class;
• WPS plugin for OpenLayers;
• Improved stability.
• Version 3.0.0 released in November of 2008
• New implementation of the WPS 1.0.0 standard.
• JavaScript WPS client introduced in 2009 with PyWPS 3.1.0.
2011
• The WPS Cookbook is published.
• A deliverable of the NETMAR project.
• Introduces the WPS standard.
• Presents PyWPS as implementation.
• Still today a valuable introduction to WPS.
• Practical examples of SOAP envelopes and Taverna
Workflow management system
• Staple GIS software in Industry and Academia.
• PyWPS featuring regularly in scientific publications.
2012-2014
• 2013 FOSS4G-CEE:
• New implementation starts being discussed;
• First lines of code are written.
• 2014 WPS Workshop organised by the Luxembourg Institute of
Science and Technology (LIST):
• Broad architecture for PyWPS-4 devised;
• Move to a new Licence discussed (MIT);
• Application to OSGeo
2015
• Development picks ups throughout 2015:
• Student at LIST;
• Google Summer of Code (GSoC).
• PyWPS enters the OSGeo incubation process in April of 2015.
2016
• Big push from OSGeo mentorship (Tom Kralidis)
• Consolidation of documentation and websites.
• PyWPS.org goes live.
• More successful GSoC applications.
• Official release of PyWPS 4.0.0 some time this Summer ...
PyWPS-4
Motivation for PyWPS-4
• PyWPS is now a decade old.
• Python version 3 taking over.
• New bindings for other libraries (e.g PyGRASS).
• New data formats (e.g. GeoJSON, KML, TopoJSON).
• Version 2.0 of the WPS standard.
• Less restrictive licence (MIT).
New Technologies
• lxml
• Flask / Werkzeug
• WSGI
• Multiprocessing
• PyPy
• Jython
• OWSLib
Data Validation
• A four level approach:
• No validation - always considered valid.
• Simple validation - uses just the mimeType.
• Strict validation - attempts to open Complex inputs with
GDAL, comparing with mimeType.
• Very strict validation - uses an XML schema.
• Plus custom validation, already including: ESRI Shapefile, GeoJSON,
GML, GeoTIFF.
Process Containerising
• WPS server should run Execute requests concurrently.
• but in totally insulated fashion;
• no shared resources or data.
• PyWPS-3:
• a temporary folder is created for each Execute;
• resulting data is moved to publishing folder;
• temporary folder is deleted at execution end.
• PyWPS-4 aims at a safer approach (e.g Docker, vagrant)
Asynchronous Execution
• PyWPS can process various Execute requests in parallel:
• configurable number;
• plus a queue of waiting requests.
• PyWPS-4 now uses the Multiprocessing module:
• os.fork() abandoned;
• PyWPS should now run on Windows too.
• Process metadata now stored in a database (SQLite, PostgreSQL -
logging, request queueing, etc). - GSoC project
• Local database to support WPS 2.0.
Future Work
• New requests in WPS 2.0: Pause, Release, Delete.
• Improved security with process containerising.
• Administrative web interface.
• External services to publish outputs.
• REST API
• Support for other languages beyond English.
Administration interface and REST API
OSGeo Incubation
Application to OSGeo
• Motion to enter incubation approved in April of 2015.
• Process mentored by Tom Kralidis.
• PyWPS-3 to feature in the next OSGeoLive release.
Governance
• PyWPS now has a Project Steering Committee (PSC)
• currently composed by 5 members
• PSC sets the roadmap, technical standards, release schedule, code
review, ...
• Monthly meetings (IRC)
Community
• Mailing list https://lists.osgeo.org/mailman/listinfo/pywps-dev
• 35 members
• IRC #geopython at irc.freenode.net server
• Gitter https://gitter.im/PyWPS
• Attending code sprints, conferences, ...
Example
Installing
# Checkout *pywps*
git clone git@github.com:geopython/pywps.git pywps
# Checkout *pywps-demo*
git clone git@github.com:geopython/pywps-demo.git pywps-demo
# Install dependencies (*requirements.txt*)
cd pywps && pip install -r requirements-dev.txt
# Install pywps
sudo python setup.py install
# Run the demo server
cd ../pywps-demo && python demo.py
Folder structure
pywps
pywps-demo
+-- demo.py
+-- LICENCE.txt
+-- processes <- working folder
| +-- area.py
| +-- bboxinout.py
| `-- ...
+-- pywps.cfg <- configurations
+-- requirements.txt
+-- server.py <- run server
+-- setup.py
+-- static
+-- templates
`-- tests
Process structure
from pywps import Process, ComplexInput, LiteralOutput, ...
class Area(Process):
def __init__(self):
inputs = [ComplexInput('layer', 'Layer', ...
outputs = [LiteralOutput('area', 'Area', ...
super(Area, self).__init__(
self._handler,
identifier='area',
title='Process Area',
inputs=inputs,
outputs=outputs,
store_supported=True,
status_supported=True)
def _handler(self, request, response):
The handler method
def _handler(self, request, response):
from shapely.geometry import shape
with temp_dir() as tmp:
input_gml = request.inputs['layer'][0].file
input_geojson = os.path.join(tmp, 'input.geojson')
subprocess.check_call(['ogr2ogr', '-f', 'geojson',
str(input_geojson), input_gml])
with open(input_geojson, 'rb') as f:
data = json.loads(f.read())
for feature in data['features']:
geom = shape(feature['geometry'])
feature['area'] = geom.area
response.outputs['area'].data = [feature['area'] ...
return response
The PyWPS Philosophy
Bikes
Jachym's Luís'
Bikes
Bikes
Bikes
The PyWPS Philosophy
• Versatility: PyWPS can be whatever you want.
• Ease of use: you only need to known how to pedal.
Thank you !
Homepage: http://pywps.org/
Source code, download: https://github.com/geopython/pywps
Twitter: @PyWPS
Licence
• Presenstation licence CC-Attribution-ShareAlike
• http://creativecommons.org/licenses/by-sa/2.5/

More Related Content

What's hot

Staying on Topic - Invoke OpenFaaS functions with Kafka
Staying on Topic - Invoke OpenFaaS functions with KafkaStaying on Topic - Invoke OpenFaaS functions with Kafka
Staying on Topic - Invoke OpenFaaS functions with KafkaRichard Gee
 
The ELK Stack - Get to Know Logs
The ELK Stack - Get to Know LogsThe ELK Stack - Get to Know Logs
The ELK Stack - Get to Know LogsGlobalLogic Ukraine
 
DoxLon | Life with kube, containers and microservices
DoxLon | Life with kube, containers and microservicesDoxLon | Life with kube, containers and microservices
DoxLon | Life with kube, containers and microservicesRichard Marshall
 
Intro to Ratpack (CDJDN 2015-01-22)
Intro to Ratpack (CDJDN 2015-01-22)Intro to Ratpack (CDJDN 2015-01-22)
Intro to Ratpack (CDJDN 2015-01-22)David Carr
 
Vladimir Ulogov - Beyond the Loadable Module
Vladimir Ulogov - Beyond the Loadable ModuleVladimir Ulogov - Beyond the Loadable Module
Vladimir Ulogov - Beyond the Loadable ModuleZabbix
 
Reporting Large Environment Zabbix Database
Reporting Large Environment Zabbix DatabaseReporting Large Environment Zabbix Database
Reporting Large Environment Zabbix DatabaseAlain Ganuchaud
 
Dockerize magento 2 24.02.2016
Dockerize magento 2   24.02.2016Dockerize magento 2   24.02.2016
Dockerize magento 2 24.02.2016Andreas Pointner
 
Tips For Maintaining OSS Projects
Tips For Maintaining OSS ProjectsTips For Maintaining OSS Projects
Tips For Maintaining OSS ProjectsTaro L. Saito
 
Freifunk Munich - How to scale Jitsi
Freifunk Munich - How to scale JitsiFreifunk Munich - How to scale Jitsi
Freifunk Munich - How to scale JitsiAnnika Wickert
 
The Road to Kubernetes
The Road to KubernetesThe Road to Kubernetes
The Road to KubernetesDeniz Zoeteman
 
Taking the Long View: How the Oslo Program Reduces Technical Debt
Taking the Long View: How the Oslo Program Reduces Technical DebtTaking the Long View: How the Oslo Program Reduces Technical Debt
Taking the Long View: How the Oslo Program Reduces Technical Debtdoughellmann
 
Tuenti Release Workflow
Tuenti Release WorkflowTuenti Release Workflow
Tuenti Release WorkflowTuenti
 
A Cassandra driver from and for the Lua community
A Cassandra driver from and for the Lua communityA Cassandra driver from and for the Lua community
A Cassandra driver from and for the Lua communityThibault Charbonnier
 
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web Scale
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web ScaleSaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web Scale
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web ScaleSaltStack
 
Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016
Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016
Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016Zabbix
 
Opinionated containers and the future of game servers by Brendan Fosberry
Opinionated containers and the future of game servers by Brendan FosberryOpinionated containers and the future of game servers by Brendan Fosberry
Opinionated containers and the future of game servers by Brendan FosberryDocker, Inc.
 
Saltconf 2016: Salt stack transport and concurrency
Saltconf 2016: Salt stack transport and concurrencySaltconf 2016: Salt stack transport and concurrency
Saltconf 2016: Salt stack transport and concurrencyThomas Jackson
 
CFWheels - Pragmatic, Beautiful Code
CFWheels - Pragmatic, Beautiful CodeCFWheels - Pragmatic, Beautiful Code
CFWheels - Pragmatic, Beautiful Codeindiver
 
Add observability to your django application - PyCon FR 2019
Add observability to your django application - PyCon FR 2019Add observability to your django application - PyCon FR 2019
Add observability to your django application - PyCon FR 2019Bleemeo
 
Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014
Hadoop Demystified + Automation Smackdown!  Austin JUG June 24 2014Hadoop Demystified + Automation Smackdown!  Austin JUG June 24 2014
Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014datafundamentals
 

What's hot (20)

Staying on Topic - Invoke OpenFaaS functions with Kafka
Staying on Topic - Invoke OpenFaaS functions with KafkaStaying on Topic - Invoke OpenFaaS functions with Kafka
Staying on Topic - Invoke OpenFaaS functions with Kafka
 
The ELK Stack - Get to Know Logs
The ELK Stack - Get to Know LogsThe ELK Stack - Get to Know Logs
The ELK Stack - Get to Know Logs
 
DoxLon | Life with kube, containers and microservices
DoxLon | Life with kube, containers and microservicesDoxLon | Life with kube, containers and microservices
DoxLon | Life with kube, containers and microservices
 
Intro to Ratpack (CDJDN 2015-01-22)
Intro to Ratpack (CDJDN 2015-01-22)Intro to Ratpack (CDJDN 2015-01-22)
Intro to Ratpack (CDJDN 2015-01-22)
 
Vladimir Ulogov - Beyond the Loadable Module
Vladimir Ulogov - Beyond the Loadable ModuleVladimir Ulogov - Beyond the Loadable Module
Vladimir Ulogov - Beyond the Loadable Module
 
Reporting Large Environment Zabbix Database
Reporting Large Environment Zabbix DatabaseReporting Large Environment Zabbix Database
Reporting Large Environment Zabbix Database
 
Dockerize magento 2 24.02.2016
Dockerize magento 2   24.02.2016Dockerize magento 2   24.02.2016
Dockerize magento 2 24.02.2016
 
Tips For Maintaining OSS Projects
Tips For Maintaining OSS ProjectsTips For Maintaining OSS Projects
Tips For Maintaining OSS Projects
 
Freifunk Munich - How to scale Jitsi
Freifunk Munich - How to scale JitsiFreifunk Munich - How to scale Jitsi
Freifunk Munich - How to scale Jitsi
 
The Road to Kubernetes
The Road to KubernetesThe Road to Kubernetes
The Road to Kubernetes
 
Taking the Long View: How the Oslo Program Reduces Technical Debt
Taking the Long View: How the Oslo Program Reduces Technical DebtTaking the Long View: How the Oslo Program Reduces Technical Debt
Taking the Long View: How the Oslo Program Reduces Technical Debt
 
Tuenti Release Workflow
Tuenti Release WorkflowTuenti Release Workflow
Tuenti Release Workflow
 
A Cassandra driver from and for the Lua community
A Cassandra driver from and for the Lua communityA Cassandra driver from and for the Lua community
A Cassandra driver from and for the Lua community
 
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web Scale
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web ScaleSaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web Scale
SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web Scale
 
Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016
Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016
Lukáš Malý - Log management ELISA controlled by Zabbix | ZabConf2016
 
Opinionated containers and the future of game servers by Brendan Fosberry
Opinionated containers and the future of game servers by Brendan FosberryOpinionated containers and the future of game servers by Brendan Fosberry
Opinionated containers and the future of game servers by Brendan Fosberry
 
Saltconf 2016: Salt stack transport and concurrency
Saltconf 2016: Salt stack transport and concurrencySaltconf 2016: Salt stack transport and concurrency
Saltconf 2016: Salt stack transport and concurrency
 
CFWheels - Pragmatic, Beautiful Code
CFWheels - Pragmatic, Beautiful CodeCFWheels - Pragmatic, Beautiful Code
CFWheels - Pragmatic, Beautiful Code
 
Add observability to your django application - PyCon FR 2019
Add observability to your django application - PyCon FR 2019Add observability to your django application - PyCon FR 2019
Add observability to your django application - PyCon FR 2019
 
Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014
Hadoop Demystified + Automation Smackdown!  Austin JUG June 24 2014Hadoop Demystified + Automation Smackdown!  Austin JUG June 24 2014
Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014
 

Similar to PyWPS-4.0.0

GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...
GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...
GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...Neo4j
 
Nginx conference 2015
Nginx conference 2015Nginx conference 2015
Nginx conference 2015ING-IT
 
DevOps of Python applications using OpenShift (Italian version)
DevOps of Python applications using OpenShift (Italian version)DevOps of Python applications using OpenShift (Italian version)
DevOps of Python applications using OpenShift (Italian version)Francesco Fiore
 
Twelve Factor App @ PHPCon 2015
Twelve Factor App @ PHPCon 2015Twelve Factor App @ PHPCon 2015
Twelve Factor App @ PHPCon 2015Tomasz Skręt
 
Fluo CICD OpenStack Summit
Fluo CICD OpenStack SummitFluo CICD OpenStack Summit
Fluo CICD OpenStack SummitMiguel Zuniga
 
Developing Cross-Platform Web Apps with ASP.NET Core1.0
Developing Cross-Platform Web Apps with ASP.NET Core1.0Developing Cross-Platform Web Apps with ASP.NET Core1.0
Developing Cross-Platform Web Apps with ASP.NET Core1.0EastBanc Tachnologies
 
12 Factor App Methodology
12 Factor App Methodology12 Factor App Methodology
12 Factor App Methodologylaeshin park
 
Building OPNFV as a Platform
Building OPNFV as a PlatformBuilding OPNFV as a Platform
Building OPNFV as a PlatformOPNFV
 
Creating a commercial PaaS offer based on Fiware
Creating a commercial PaaS offer based on Fiware Creating a commercial PaaS offer based on Fiware
Creating a commercial PaaS offer based on Fiware Giovanni Coppa
 
CI/CD with Azure DevOps and Azure Databricks
CI/CD with Azure DevOps and Azure DatabricksCI/CD with Azure DevOps and Azure Databricks
CI/CD with Azure DevOps and Azure DatabricksGoDataDriven
 
Introducing ASP.NET vNext - A tour of the new ASP.NET platform
Introducing ASP.NET vNext - A tour of the new ASP.NET platformIntroducing ASP.NET vNext - A tour of the new ASP.NET platform
Introducing ASP.NET vNext - A tour of the new ASP.NET platformJeffrey T. Fritz
 
State of NuPIC
State of NuPICState of NuPIC
State of NuPICNumenta
 
321 codeincontainer brewbox
321 codeincontainer brewbox321 codeincontainer brewbox
321 codeincontainer brewboxLino Telera
 
Tech 2 Tech IPv6 presentation
Tech 2 Tech IPv6 presentationTech 2 Tech IPv6 presentation
Tech 2 Tech IPv6 presentationJisc
 
How we realized SOA by Python at PyCon JP 2015
How we realized SOA by Python at PyCon JP 2015How we realized SOA by Python at PyCon JP 2015
How we realized SOA by Python at PyCon JP 2015hirokiky
 
PyCon India 2012: Celery Talk
PyCon India 2012: Celery TalkPyCon India 2012: Celery Talk
PyCon India 2012: Celery TalkPiyush Kumar
 
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TShapeBlue
 
PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux Neotys
 

Similar to PyWPS-4.0.0 (20)

GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...
GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...
GraphDay Paris - Intégrer des flux de données dans Neo4j avec l'ETL Open Sour...
 
Nginx conference 2015
Nginx conference 2015Nginx conference 2015
Nginx conference 2015
 
DevOps of Python applications using OpenShift (Italian version)
DevOps of Python applications using OpenShift (Italian version)DevOps of Python applications using OpenShift (Italian version)
DevOps of Python applications using OpenShift (Italian version)
 
Twelve Factor App @ PHPCon 2015
Twelve Factor App @ PHPCon 2015Twelve Factor App @ PHPCon 2015
Twelve Factor App @ PHPCon 2015
 
Fluo CICD OpenStack Summit
Fluo CICD OpenStack SummitFluo CICD OpenStack Summit
Fluo CICD OpenStack Summit
 
Developing Cross-Platform Web Apps with ASP.NET Core1.0
Developing Cross-Platform Web Apps with ASP.NET Core1.0Developing Cross-Platform Web Apps with ASP.NET Core1.0
Developing Cross-Platform Web Apps with ASP.NET Core1.0
 
12 Factor App Methodology
12 Factor App Methodology12 Factor App Methodology
12 Factor App Methodology
 
Intro to CakePHP
Intro to CakePHPIntro to CakePHP
Intro to CakePHP
 
Building OPNFV as a Platform
Building OPNFV as a PlatformBuilding OPNFV as a Platform
Building OPNFV as a Platform
 
Sas 2015 event_driven
Sas 2015 event_drivenSas 2015 event_driven
Sas 2015 event_driven
 
Creating a commercial PaaS offer based on Fiware
Creating a commercial PaaS offer based on Fiware Creating a commercial PaaS offer based on Fiware
Creating a commercial PaaS offer based on Fiware
 
CI/CD with Azure DevOps and Azure Databricks
CI/CD with Azure DevOps and Azure DatabricksCI/CD with Azure DevOps and Azure Databricks
CI/CD with Azure DevOps and Azure Databricks
 
Introducing ASP.NET vNext - A tour of the new ASP.NET platform
Introducing ASP.NET vNext - A tour of the new ASP.NET platformIntroducing ASP.NET vNext - A tour of the new ASP.NET platform
Introducing ASP.NET vNext - A tour of the new ASP.NET platform
 
State of NuPIC
State of NuPICState of NuPIC
State of NuPIC
 
321 codeincontainer brewbox
321 codeincontainer brewbox321 codeincontainer brewbox
321 codeincontainer brewbox
 
Tech 2 Tech IPv6 presentation
Tech 2 Tech IPv6 presentationTech 2 Tech IPv6 presentation
Tech 2 Tech IPv6 presentation
 
How we realized SOA by Python at PyCon JP 2015
How we realized SOA by Python at PyCon JP 2015How we realized SOA by Python at PyCon JP 2015
How we realized SOA by Python at PyCon JP 2015
 
PyCon India 2012: Celery Talk
PyCon India 2012: Celery TalkPyCon India 2012: Celery Talk
PyCon India 2012: Celery Talk
 
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
 
PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux
 

More from Jachym Cepicky

Python testing-frameworks overview
Python testing-frameworks overviewPython testing-frameworks overview
Python testing-frameworks overviewJachym Cepicky
 
What is the price of open source
What is the price of open sourceWhat is the price of open source
What is the price of open sourceJachym Cepicky
 
Testing web application with Python
Testing web application with PythonTesting web application with Python
Testing web application with PythonJachym Cepicky
 
Danube hack 2015 - Open (-data, -communities)
Danube hack 2015 - Open (-data, -communities)Danube hack 2015 - Open (-data, -communities)
Danube hack 2015 - Open (-data, -communities)Jachym Cepicky
 
Push it through the wire
Push it through the wirePush it through the wire
Push it through the wireJachym Cepicky
 
How Prague is opening data
How Prague is opening dataHow Prague is opening data
How Prague is opening dataJachym Cepicky
 
Webgis, Cloud computing, OGC OWS
Webgis, Cloud computing, OGC OWSWebgis, Cloud computing, OGC OWS
Webgis, Cloud computing, OGC OWSJachym Cepicky
 
Co může udělat vaše firma pro open source
Co může udělat vaše firma pro open sourceCo může udělat vaše firma pro open source
Co může udělat vaše firma pro open sourceJachym Cepicky
 
Otevřené standardy, Otevřená data, Otevřený software, Otevření lidé
Otevřené standardy, Otevřená data, Otevřený software, Otevření lidéOtevřené standardy, Otevřená data, Otevřený software, Otevření lidé
Otevřené standardy, Otevřená data, Otevřený software, Otevření lidéJachym Cepicky
 
Úvod do otevřená geoinfrastruktury
Úvod do otevřená geoinfrastrukturyÚvod do otevřená geoinfrastruktury
Úvod do otevřená geoinfrastrukturyJachym Cepicky
 
Sdílené intelektuální spoluvlastnictví
Sdílené intelektuální spoluvlastnictvíSdílené intelektuální spoluvlastnictví
Sdílené intelektuální spoluvlastnictvíJachym Cepicky
 
Co brání většímu rozšíření open source nástrojů
Co brání většímu rozšíření open source nástrojůCo brání většímu rozšíření open source nástrojů
Co brání většímu rozšíření open source nástrojůJachym Cepicky
 
Open Source JavaScript Mapping Framework
Open Source JavaScript Mapping FrameworkOpen Source JavaScript Mapping Framework
Open Source JavaScript Mapping FrameworkJachym Cepicky
 
PyWPS at COST WPS Workshop
PyWPS at COST WPS WorkshopPyWPS at COST WPS Workshop
PyWPS at COST WPS WorkshopJachym Cepicky
 
Cepicky os-mapping-frameworks
Cepicky os-mapping-frameworksCepicky os-mapping-frameworks
Cepicky os-mapping-frameworksJachym Cepicky
 

More from Jachym Cepicky (20)

Switch from shapefile
Switch from shapefileSwitch from shapefile
Switch from shapefile
 
Python testing-frameworks overview
Python testing-frameworks overviewPython testing-frameworks overview
Python testing-frameworks overview
 
What is the price of open source
What is the price of open sourceWhat is the price of open source
What is the price of open source
 
Testing web application with Python
Testing web application with PythonTesting web application with Python
Testing web application with Python
 
Danube hack 2015 - Open (-data, -communities)
Danube hack 2015 - Open (-data, -communities)Danube hack 2015 - Open (-data, -communities)
Danube hack 2015 - Open (-data, -communities)
 
Push it through the wire
Push it through the wirePush it through the wire
Push it through the wire
 
How Prague is opening data
How Prague is opening dataHow Prague is opening data
How Prague is opening data
 
Webgis, Cloud computing, OGC OWS
Webgis, Cloud computing, OGC OWSWebgis, Cloud computing, OGC OWS
Webgis, Cloud computing, OGC OWS
 
Co může udělat vaše firma pro open source
Co může udělat vaše firma pro open sourceCo může udělat vaše firma pro open source
Co může udělat vaše firma pro open source
 
Otevřené standardy, Otevřená data, Otevřený software, Otevření lidé
Otevřené standardy, Otevřená data, Otevřený software, Otevření lidéOtevřené standardy, Otevřená data, Otevřený software, Otevření lidé
Otevřené standardy, Otevřená data, Otevřený software, Otevření lidé
 
Úvod do otevřená geoinfrastruktury
Úvod do otevřená geoinfrastrukturyÚvod do otevřená geoinfrastruktury
Úvod do otevřená geoinfrastruktury
 
PyWPS Status report
PyWPS Status reportPyWPS Status report
PyWPS Status report
 
Geosense Geoportal
Geosense GeoportalGeosense Geoportal
Geosense Geoportal
 
Cepicky pywps4
Cepicky pywps4Cepicky pywps4
Cepicky pywps4
 
Sdílené intelektuální spoluvlastnictví
Sdílené intelektuální spoluvlastnictvíSdílené intelektuální spoluvlastnictví
Sdílené intelektuální spoluvlastnictví
 
Co brání většímu rozšíření open source nástrojů
Co brání většímu rozšíření open source nástrojůCo brání většímu rozšíření open source nástrojů
Co brání většímu rozšíření open source nástrojů
 
Open Source JavaScript Mapping Framework
Open Source JavaScript Mapping FrameworkOpen Source JavaScript Mapping Framework
Open Source JavaScript Mapping Framework
 
PyWPS at COST WPS Workshop
PyWPS at COST WPS WorkshopPyWPS at COST WPS Workshop
PyWPS at COST WPS Workshop
 
Cepicky osgeocz
Cepicky osgeoczCepicky osgeocz
Cepicky osgeocz
 
Cepicky os-mapping-frameworks
Cepicky os-mapping-frameworksCepicky os-mapping-frameworks
Cepicky os-mapping-frameworks
 

Recently uploaded

Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 

Recently uploaded (20)

Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 

PyWPS-4.0.0

  • 3. What is PyWPS • PyWPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium • PyWPS is written in Python • Started in the Spring of 2006 • Supports all available tools in Python for geospatial operations • http://pywps.org
  • 4. What PyWPS is NOT • complicated • a client • a GUI • a server with pre-installed processes
  • 6. The OGC Web Processing Service • OGC open web standard for remote geo-spatial processing. • Integrated with web data services: WFS, WCS. • Three basic requests: • GetCapabilities • DescribeProcess • Execute • Three basic input/output classes: • Literal • Complex - for geo-spatial data and services • BouningBox - for geo-spatial data extent
  • 7. The OGC Web Processing Service http://www.slideshare.net/TheodorFoerster/restful-web-processing-service
  • 8. Essential PyWPS Functionality • Communication bridge with WPS. • Fetches input data referenced in Execute requests. • Creates a container for the process instance. • Manage processes: communication, reporting, logging. • Data storage for final data outputs. • Client notification, status reporting
  • 10. 2006 • Jachym Cepicky starts working on a project to link GRASS with UMN Mapserver. • Scholarship funded by the German Foundation for Environment. • September: PyWPS is presented at FOSS4G 2006. • October: Project homepage hosted by Intevation GmbH. • November: Version 1.0.0 is released; other developers join the project.
  • 11. 2007 - 2010 • Version 2.0.0 released in October of 2007 • Introduced the Process class; • WPS plugin for OpenLayers; • Improved stability. • Version 3.0.0 released in November of 2008 • New implementation of the WPS 1.0.0 standard. • JavaScript WPS client introduced in 2009 with PyWPS 3.1.0.
  • 12. 2011 • The WPS Cookbook is published. • A deliverable of the NETMAR project. • Introduces the WPS standard. • Presents PyWPS as implementation. • Still today a valuable introduction to WPS. • Practical examples of SOAP envelopes and Taverna Workflow management system • Staple GIS software in Industry and Academia. • PyWPS featuring regularly in scientific publications.
  • 13. 2012-2014 • 2013 FOSS4G-CEE: • New implementation starts being discussed; • First lines of code are written. • 2014 WPS Workshop organised by the Luxembourg Institute of Science and Technology (LIST): • Broad architecture for PyWPS-4 devised; • Move to a new Licence discussed (MIT); • Application to OSGeo
  • 14. 2015 • Development picks ups throughout 2015: • Student at LIST; • Google Summer of Code (GSoC). • PyWPS enters the OSGeo incubation process in April of 2015.
  • 15. 2016 • Big push from OSGeo mentorship (Tom Kralidis) • Consolidation of documentation and websites. • PyWPS.org goes live. • More successful GSoC applications. • Official release of PyWPS 4.0.0 some time this Summer ...
  • 17. Motivation for PyWPS-4 • PyWPS is now a decade old. • Python version 3 taking over. • New bindings for other libraries (e.g PyGRASS). • New data formats (e.g. GeoJSON, KML, TopoJSON). • Version 2.0 of the WPS standard. • Less restrictive licence (MIT).
  • 18. New Technologies • lxml • Flask / Werkzeug • WSGI • Multiprocessing • PyPy • Jython • OWSLib
  • 19. Data Validation • A four level approach: • No validation - always considered valid. • Simple validation - uses just the mimeType. • Strict validation - attempts to open Complex inputs with GDAL, comparing with mimeType. • Very strict validation - uses an XML schema. • Plus custom validation, already including: ESRI Shapefile, GeoJSON, GML, GeoTIFF.
  • 20. Process Containerising • WPS server should run Execute requests concurrently. • but in totally insulated fashion; • no shared resources or data. • PyWPS-3: • a temporary folder is created for each Execute; • resulting data is moved to publishing folder; • temporary folder is deleted at execution end. • PyWPS-4 aims at a safer approach (e.g Docker, vagrant)
  • 21. Asynchronous Execution • PyWPS can process various Execute requests in parallel: • configurable number; • plus a queue of waiting requests. • PyWPS-4 now uses the Multiprocessing module: • os.fork() abandoned; • PyWPS should now run on Windows too. • Process metadata now stored in a database (SQLite, PostgreSQL - logging, request queueing, etc). - GSoC project • Local database to support WPS 2.0.
  • 22. Future Work • New requests in WPS 2.0: Pause, Release, Delete. • Improved security with process containerising. • Administrative web interface. • External services to publish outputs. • REST API • Support for other languages beyond English.
  • 25. Application to OSGeo • Motion to enter incubation approved in April of 2015. • Process mentored by Tom Kralidis. • PyWPS-3 to feature in the next OSGeoLive release.
  • 26. Governance • PyWPS now has a Project Steering Committee (PSC) • currently composed by 5 members • PSC sets the roadmap, technical standards, release schedule, code review, ... • Monthly meetings (IRC)
  • 27. Community • Mailing list https://lists.osgeo.org/mailman/listinfo/pywps-dev • 35 members • IRC #geopython at irc.freenode.net server • Gitter https://gitter.im/PyWPS • Attending code sprints, conferences, ...
  • 29. Installing # Checkout *pywps* git clone git@github.com:geopython/pywps.git pywps # Checkout *pywps-demo* git clone git@github.com:geopython/pywps-demo.git pywps-demo # Install dependencies (*requirements.txt*) cd pywps && pip install -r requirements-dev.txt # Install pywps sudo python setup.py install # Run the demo server cd ../pywps-demo && python demo.py
  • 30. Folder structure pywps pywps-demo +-- demo.py +-- LICENCE.txt +-- processes <- working folder | +-- area.py | +-- bboxinout.py | `-- ... +-- pywps.cfg <- configurations +-- requirements.txt +-- server.py <- run server +-- setup.py +-- static +-- templates `-- tests
  • 31. Process structure from pywps import Process, ComplexInput, LiteralOutput, ... class Area(Process): def __init__(self): inputs = [ComplexInput('layer', 'Layer', ... outputs = [LiteralOutput('area', 'Area', ... super(Area, self).__init__( self._handler, identifier='area', title='Process Area', inputs=inputs, outputs=outputs, store_supported=True, status_supported=True) def _handler(self, request, response):
  • 32. The handler method def _handler(self, request, response): from shapely.geometry import shape with temp_dir() as tmp: input_gml = request.inputs['layer'][0].file input_geojson = os.path.join(tmp, 'input.geojson') subprocess.check_call(['ogr2ogr', '-f', 'geojson', str(input_geojson), input_gml]) with open(input_geojson, 'rb') as f: data = json.loads(f.read()) for feature in data['features']: geom = shape(feature['geometry']) feature['area'] = geom.area response.outputs['area'].data = [feature['area'] ... return response
  • 35. Bikes
  • 36. Bikes
  • 37. Bikes
  • 38. The PyWPS Philosophy • Versatility: PyWPS can be whatever you want. • Ease of use: you only need to known how to pedal.
  • 39. Thank you ! Homepage: http://pywps.org/ Source code, download: https://github.com/geopython/pywps Twitter: @PyWPS
  • 40. Licence • Presenstation licence CC-Attribution-ShareAlike • http://creativecommons.org/licenses/by-sa/2.5/