Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serving earth observation data with GeoServer: addressing real world requirements with Open Source

1,245 views

Published on

Information on the latest developments in GeoServer for the support for Earth Observation data with support for the various OGC services and OpenSearch.

Published in: Technology

Serving earth observation data with GeoServer: addressing real world requirements with Open Source

  1. 1. Serving earth observation data with GeoServer: addressing real world requirements with Open Source Ing. Simone Giannecchini Ing. Andrea Aime
  2. 2. Outline  About Us  GeoServer Intro  Current Support for EO/MetOc data  The new integrated model with OpenSearch Support EO OpenScience Conference 2017
  3. 3. Quick Facts  Founded in late 2006  Expertise • Image Processing, GeoSpatial Data Fusion • Java, Java Enterprise, C++, Python • JPEG2000, JPIP, Advanced visualization • Web Based Mashups, Mobile Solutions • OGC, ISO, INSPIRE Standards  Supporting/Developing FOSS4G projects  MapStore, GeoServer, GeoNetwork  CKAN, GeoNode  Clients  UN FAO (CIOK, FIGIS, NRL, FORESTRY, ESTG), UN WFP, World Bank, DLR, EUMETSAT, JRC, ARPAT, NATO CMRE, UNESCO, IGAD, UNEP, etc..  Private Companies all over the world EO OpenScience Conference 2017
  4. 4. Our Offer  Enterprise Support Services  Bug Fixing, Support, Customizations & New Features  5 packages  different types of needs  Deployment Warranty  Deploy with confidence, we monitor and fix when needed  Professional Training  End-To-End Projects (Integration)  Tell me what you need, I’ll put it together for you  We take our core products and  Bend them, twist them, embed them  Hammer them to make clients happy  FOSS4G Software is core for us  Not simple use but develop and support EO OpenScience Conference 2017
  5. 5. GeoServer
  6. 6. GeoServer  GeoSpatial enterprise gateway Java Enterprise Management and Dissemination of raster and vector data  Standards compliant OGC WCS 1.0, 1.1.1 (RI), 2.0.1 OGC WFS 1.0, 1.1 (RI), 2.0.0 OGC WMS 1.1.1, 1.3.0 OGC WPS 1.0.0 OGC CSW 2.0.2 OGC EO WMS, EO WCS  Google Earth/Maps support KML, GeoSearch, etc.. EO OpenScience Conference 2017
  7. 7. This is GeoServer! EO OpenScience Conference 2017
  8. 8. Enough fuss, here’s the real stuff
  9. 9. Reference scenario EO OpenScience Conference 2017
  10. 10. ImageMosaic – the EO/MetOc workhorse
  11. 11. Terminology  Granule/Tile  individual raster element composing the mosaic  (Granule) Index  collection of metadata records describing the location, spatial coverage and other attributes of each single granule  Dimensions/Domains  dimensions besides the spatial ones used to distinguish individual granules EO OpenScience Conference 2017
  12. 12. Dimensions  Map to alphanumeric attributes in the index  TIME and ELEVATION receive special treatment for WMS and WCS  Custom/Additional dimensions  Everything besides TIME & ELEVATION  They can be dynamically discovered from WMS GetCapabilities and WCS DescribeCoverage TIME Dimension EO OpenScience Conference 2017
  13. 13. REST configuration  (Granule) Index CRUD Operations via REST  Create mosaic  Read its index schema  Read its index contents  Update/Delete/Insert granules GS Pixels Index Client  Support for programmatic access  Workflow ETL Integration EO OpenScience Conference 2017
  14. 14. NetCDF Formats Family Support
  15. 15. NetCDF format support  NetCDF support  Support COARDS/CF* conventions loosely  Expose NetCDF internal data as a set of 2D slices  Fast 2D (time, elevation) slice extraction  Supports NCML EO OpenScience Conference 2017
  16. 16. One or more variables  Polyphemus Sample Dataset  1 File  Multiple Coverages! EO OpenScience Conference 2017
  17. 17. ImageMosaic – NetCDF integration
  18. 18. NetCDF/Mosaic integration  ImageMosaic NetCDF integration  Allow the ImageMosaic to handle multiple NetCDF files  Expose NetCDF internal structure (times, elevations)  Make ImageMosaic handle slices of the NetCDF file as granules  5-dim mosaic (space/time/runtime other dimensions) Mosaic NetCDF 1 NetCDF 2 NetCDF … N … EO OpenScience Conference 2017
  19. 19. The services - visualize
  20. 20. WMS-TIME support TIME, ELEVATION & Custom dimensions http://localhost:8080/geoserver/geosolutions/wms?... &time=2013-03-1T00:00:00.000Z &elevation=35.0 &DIM_FILEDATE=2013-03-01T00:00:00.000Z &DIM_UPDATED=2013-04-08T08:18:41.597Z EO OpenScience Conference 2017
  21. 21. &CQL_FILTER=platform=SENTINEL2  filter WMS on the fly! WMS – custom filtering and sorting &sortBy=time A&sortBy=cloudCoverage D EO OpenScience Conference 2017
  22. 22. Rendering transformations  SLD Based transformations  On-the-fly contouring  On-the-fly poligonalization  Current arrows  Wind Barbs  Pluggable: add your own windbarbs contouring currents EO OpenScience Conference 2017
  23. 23. WMTS with Multidim Domain Discovery
  24. 24. Multidim Domain Discovery  Spatiotemporal data has complex domains  Multiple dimensions  Spatial, elevation, acquisition time, forecast time, reference_time, etc…  Data scattered along space and time  Think about different products within a mission  Values vary in near real-time  New acquisition (EO)  New model runs (MetOc, EMR..)  Difficult to convey info about complex domains  WMS and WMTS very clunky, (EO) WCS much better  We miss (at least) two things in the standards above:  Ability to convey domain coverage quickly  Ability to drill down in dimension for refined queries ESA EOP-GTS Meeting 14th December 2016 EO OpenScience Conference 2017
  25. 25.  Extending OGC services (proposal here from OWS12)  A few Domain Discovery Operation  WMTS operations, might move to other services (e.g. WMS 1.4)  #1: DescribeDomains  Which dimensions do you have in your domain?  xx?REQUEST=DescribeDomains&Version=1.0.0&Layer=test:flexpart_time_series&TileMatrix Set=EPSG:900913&elevation=0.0/500.0&time=2016-02-23T03:00:00.000Z/2016-02- 23T12:00:00.000Z&reference_time=2016-02-23T00:00:00.000Z Multidim Domain Discovery EO OpenScience Conference 2017
  26. 26.  #2) DescribeDomains  Tell me which dimensions you expose  xx?REQUEST=DescribeDomains&Version=1.0.0&Layer=test:flexpart_time_series&TileMatrix Set=EPSG:900913&elevation=0.0/500.0&time=2016-02-23T03:00:00.000Z/2016-02- 23T12:00:00.000Z&reference_time=2016-02-23T00:00:00.000Z Multidim Domain Discovery EO OpenScience Conference 2017
  27. 27.  #3: GetHistogram  Show me how much data you have in this spatiotemporal bin  xx?REQUEST=GetHistogram&Version=1.0.0&Layer=test:flexpart_time_series&TileMatrixSet =EPSG:4326&histogram=reference_time&resolution=PT1H Multidim Domain Discovery EO OpenScience Conference 2017
  28. 28.  #4: GetFeature  Let do fine grain queries where the data is located  xx?REQUEST=GetFeature&Version=1.0.0&Layer=test:B2&TileMatrixSet=EPSG:4326&time=2 016-07-01T00:00:00.000Z/2016-07-31T00:00:00.000Z&bbox=-118,34,-115,38 Multidim Domain Discovery EO OpenScience Conference 2017
  29. 29. The services - download
  30. 30. WCS 2.0  Basics  Core service  KVP binding  XML binding  Common extensions  CRS  Scaling  Interpolation  Range subsetting  GeoTiff  GML  NetCDF  Add the output format extensions  GeoTIFF  GML Grid  NetCDF EO OpenScience Conference 2017
  31. 31. WCS 2.0 DescribeCoverage 4D BBOX TIME details as extension ELEVATION details as extension CUSTOM EO OpenScience Conference 2017
  32. 32. WCS – custom filtering and sorting &CQL_FILTER=platform=SENTINEL2  filter WCS on the fly &sortBy=time A&sortBy=cloudCoverage D EO OpenScience Conference 2017
  33. 33. Download ND NetCDF http://localhost:8080/geoserver/wcs?request=GetCoverage &service=WCS&version=2.0.1&coverageId=geosolutions__NO2 &Format=application/x-netcdf &subset=Long(5,20) &subset=Lat(40,50) &subset=elevation(300,1250) &subset=time("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z") • Can also handle custom dimensions • Described in DescribeCoverage in a vendor metadata section 4D BBOX EO OpenScience Conference 2017
  34. 34. The new kid on the block: Integrated model with OpenSearch
  35. 35. OpenSearch Support: Putting it all together (and finding it)  Going beyond flat index ImageMosaic  Support Collections and Granules  two steps search  Use case  Lots of satellite imagery  Multiple satellites  Multiple sensors  Deep time and space distribution  What to do about it?  “OpenSearch for EO“ to the rescue  Geo and Time Extension to OpenSearch  keywords for time and space  Earth Observation extension  keywords for and EO properties  Cloud cover, Snow cover, Off nadir… OpenSearch OGC OpenSearch Geo and Time OGC OpenSearch for EO EO OpenScience Conference 2017
  36. 36. Integrated Model: Single Model with OGC Services Support  Based on PostGIS  ElasticSearch is on the radar  Core Elements  Collection, Product, Granule  Single integrated model  Drives all services  Exposing imagery catalog as OpenSearch for discovery  Exposing imagery catalog as WFS for discovery  Different collections in different FeatureTypes  Simpler (?) Alternative to OpenSearch for discovery  Exposing Collections as WCS ImageMosaic layers  Exposing Collections as WMS ImageMosaic layers  Support for CQL_Filter and SortBy  Rich query lang  1st discovery then jump to the other OGC services!  Full Admin REST Interface with automation Swagger documentation here EO OpenScience Conference 2017
  37. 37.  http://cloudsdi.geo-solutions.it/geoserver/oseo/search?parentId=SENTINEL2&cloudCover=30]  Output is RSS with description, thumb and links  Links to metadata  ISO metadata for collection  GML O&M for products  Links to OGC services (cross linking)  WMS/WMTS to see collection/product  WFS to get “masks” (validity, sea, cloud, snow, …)  WCS to extract raster  Direct download to get original package  One “feature type” per collection  Can be used as the index to a mosaic  One mosaic per collection  Filter using CQL and sort by attribute on the WMS/WCS service  Use all the attributes already searchable by OpenSearch Simple search + Tight integration with mosaic EO OpenScience Conference 2017
  38. 38. More on exposed services
  39. 39. GeoServer – REST Interface  Administrative Programmatic Interface  Manage EO Model Content  Create/Update/Remove a collection  Add/Update/Remove products with support for mass unpublish/delete  Swagger Description here  Collections  Listing  Listing with paging  Create a new collection  Collection  Inspect a collection  Delete a collection  Update a collection  Manage individual elements (ogcLinks, description, metadata, thumbnail) EO OpenScience Conference 2017
  40. 40. GeoServer – REST Interface  Products  Listing  Listing with paging  Add new product  Product  View Single Product  Update a product  Delete a product  Manage individual elements (searchable elements, ogcLinks, description, metadata, thumbnail, granules)  ogcLinks use CQL_FILTER to filter individual elements in the time series (WMS, WFS and WCS) EO OpenScience Conference 2017
  41. 41. GeoServer – WFS  Exposing imagery catalog as WFS  Different collections in different FeatureTypes  Alternative to OpenSearch for discovery  Lower resistence path for web clients  Support for CQL_Filter and SortBy  Rich query syntax (might not work over time as data changes in the prototype over time)  All Sentinel 1 products  Sentinel 1, OrbitDirection ASCENDING  Get Single Product by ID (uses CQL)  Sentinel 1, OrbitDirection ASCENDING, SortBy startTime  Once discovery is done we can jump to the other services! EO OpenScience Conference 2017
  42. 42. GeoServer – WCS  Exposing Collections as WCS ImageMosaic layers  Different collections in different layer  Support for CQL_Filter and SortBy for filtering  Rich query syntax (might not work over time as data changes)  All Sentinel 1 products, SortBy timeStart  Ascending  Descending  Get Single Product by ID (uses CQL)  Sentinel 1, OrbitDirection ASCENDING, SortBy startTime  Ascending  Descending EO OpenScience Conference 2017
  43. 43.  Exposing Collections as WMS ImageMosaic layers  Different collections in different layer  Support for CQL_Filter and SortBy for filtering  Rich query syntax (might not work over time as data changes)  All Sentinel 1 products  Sentinel 1, OrbitDirection ASCENDING  View Single Product by ID (uses CQL)  Sentinel 1, OrbitDirection DESCENDING, SortBy startTime Descending  Sentinel 1, OrbitDirection ASCENDING, SortBy startTime Ascending  Sentinel 1, OrbitDirection DESCENDING and startTime BEFORE 2017 GeoServer – WMS EO OpenScience Conference 2017
  44. 44. Bonus Track: Multiresolution, MultiCRS Mosaicks
  45. 45. WMS – heterogeneous CRS mosaic UTM60N UTM1N WGS84 EO OpenScience Conference 2017  Support for mosaicking data in multiple CRS  Support for mosaicking data over the dateline
  46. 46. WMS – heterogeneous resolution view  Sentinel2 bands have variable resolution, e.g., 10m, 20m, 60m  N separate single band mosaics merged into a single view EO OpenScience Conference 2017
  47. 47. WMS – heterogeneous resolution view  Sentinel2 bands have variable resolution, e.g., 10m, 20m, 60m  Merged into a single view EO OpenScience Conference 2017
  48. 48. That’s all! Questions? info@geo-solutions.it EO OpenScience Conference 2017

×