Toulouse, CNES - 2013.10.01 RTSTechniques de l'Information et de la Communication Synthesis of R&T projects on Cloud Computing and WPS - i.e. OpenStack cloud based solution for processing SRTM data followed by Land Cover classification through WPS

1. Cloud Computing & Web Processing Services
https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services
Jérôme Gasperi
Jerome.Gasperi@cnes.fr
RTS Techniques de l'Information et de la Communication
CNES - Toulouse, France - October 1st, 2013

2. Cloud Computing
Introduction
What we have done
Issues
Web Processing Services
Introduction
What we have done
Issues
What's next ?
WPS on the cloud

3. R&T Cloud Computing (2011)
Use a cloud infrastructure to process Earth Observation data

4. ?

5. host
build
consume
IAAS
PAAS
SAAS

6. host
build
consume
IAAS
PAAS
SAAS

7. Private or public ?

8. process
to
1 €
1 time
store
+
5 €
per year
access
0.5 €
each time
(Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)

9. process
to
1 €
1 time
store
+
5 €
per year
access
0.5 €
each time
(Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)

10. Data security and user privacy cannot
be guaranteed in public clouds
Majority of cloud providers are subject to governmental law (e.g. US patriot act)

11. ...so we choose a private solution

12. What we have
done
Web browser
eNovance (Paris)

13. What we have
done
Data
SRTM tile
Web browser
eNovance (Paris)

14. What we have
done
Data
Processing (Orfeo Toolbox)
SRTM tile
Web browser
eNovance (Paris)

15. What we have
done
Data
Processing (Orfeo Toolbox)
SRTM tile
Hill Shading
Web browser
eNovance (Paris)

16. What we have
done
Data
Processing (Orfeo Toolbox)
SRTM tile
Hill Shading
Projecting
Web browser
eNovance (Paris)

17. What we have
done
Data
Processing (Orfeo Toolbox)
VISUALIZE
SRTM tile
WMS Server
Hill Shading
Projecting
Tiling
Web browser
eNovance (Paris)

18. Conclusions

19. Private cloud technologies are quite simple to implement. They reduce
the cost of operation and maintenance by sharing a common
infrastructure across multiple projects

20. However...

21. However...
Transfering large volumes of data is problematic

22. However...
Transfering large volumes of data is problematic
This is true for upload and download

23. ...so

24. ...so
Data and processes should be colocated

25. ...so
Data and processes should be colocated
Processing results should be downloaded only if needed

26. ...so
Data and processes should be colocated
Processing results should be downloaded only if needed
Better to offer a Web service to visualize/manipulate results

27. ...so
Data and processes should be colocated
Processing results should be downloaded only if needed
Better to offer a Web service to visualize/manipulate results
Standardize processes inputs/outputs description (e.g. WPS)

28. Web Processing Service (2012-2013)
Set WPS on top of Orfeo Toolbox processing chains

29. Web Processing Services (WPS) is an OGC standard which defines
interfaces to publish, describe and execute geospatial processes

30. Web application
WPS
Processing a

31. What we have
done
From pixel...
Remote sensing image

32. What we have
done
From pixel...
Remote sensing image
...to information
Land cover thematic map

33. Orfeo Toolbox
Processing
http://orfeo-toolbox.org
Constellation
WPS
http://www.constellation-sdi.org
mapshup
http://mapshup.info
Web
application

34. Processing
Orfeo Toolbox
Remote sensing image library
Open Source
Developped by the French Space Agency

35. Processing
Orfeo Toolbox
More than 70 high level processing chains
orthorectification
segmentation
classification
etc.

36. Processing
Supervised learning
Based on SVM
(http://en.wikipedia.org/wiki/Support_vector_machine)
(land cover is computed from a set of "well known areas" given by user)
Orfeo Toolbox
More than 70 high level processing chains
orthorectification
segmentation
classification
etc.

37. WPS
Constellation
Geospatial Networking Services
OGC/ISO implementations
Multi platform (JAVA)
OpenSource
WMS
WFS
CSW
WPS

38. Web
application
mapshup
Improves access to geospatial data
HTML + javascript
Based on OpenLayers and jQuery

39. Web
application
mapshup
Supported data sources
WMS
WFS
CSW
WPS
OpenSearch
KML
Google
maps
Bing
maps
WMTS
OpenStreetMap
MapBox
Youtube
Wikipedia
GeoRSS
Flickr
etc.

40. See WPS demo
Web
application
https://vimeo.com/67288678
mapshup
Supported data sources
WMS
WFS
CSW
WPS
OpenSearch
KML
Google
maps
Bing
maps
WMTS
OpenStreetMap
MapBox
Youtube
Wikipedia
GeoRSS
Flickr
etc.

41. What have
we done ?
https://vimeo.com/67288677

42. Conclusions

43. WPS 1.0 lacks a true asynchronous process management...
There is no way to cancel a launched process !

44. However...

45. However...
WPS 2.0 defines a set of process management operations - GetStatus,
Delete, Pause and Resume

46. However...
WPS 2.0 defines a set of process management operations - GetStatus,
Delete, Pause and Resume
This is a must have to deploy asynchronous WPS on the cloud

47. However...
WPS 2.0 defines a set of process management operations - GetStatus,
Delete, Pause and Resume
This is a must have to deploy asynchronous WPS on the cloud
Should be an official OGC standard by the end of 2013

48. What's next
WPS on the cloud - OGC OWS-10 Testbeb

49. Orthorectifying images hosted by a cloud infrastructure using
WPS standard
In kind contribution to the «Open Mobility» thread of the OGC OWS-10 Testbeb
Final delivery and demonstration April/May 2014

50. Services
Client
1
SEARCH
3
PROCESS
WPS Server
4
+
CSW Server
Metadata
2
Processing
Raw images
Result
Orthorectify
VISUALIZE
Services
Data (Pléiades samples)
(OTB)
WMS Server
Interoute (location : UK)
Quality
WPS Server
VISUALIZE
WMS Server
Processing
Assess
Quality
Improve
Quality
Astrium GEO

51. 3. Result is displayed within the map trough a WMS
1. Select raw image to orthorectify within
Landsat
CSW catalogLaat
located on the
Landsat
2. Click on '+' to process a new orthorectification.
Process parameters are set by user and sent to an
asynchronous WPS orthorectification process
located on the
Landsat
4. Orthorectified image quality can be checked through
"Assess Quality" WPS process located on the
Landsat
Result is displayed within the map as a WMS quality layer
stored on the

52. Cloud Computing & Web Processing Services
https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services
Jérôme Gasperi
Jerome.Gasperi@cnes.fr
RTS Techniques de l'Information et de la Communication
CNES - Toulouse, France - October 1st, 2013