Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments
Upcoming SlideShare
Loading in...5
×
 

Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments

on

  • 3,130 views

The purpose of this presentation is to discuss the developments during last years in raster data support in GeoTools and GeoServer, and also to introduce and discuss future development directions.

The purpose of this presentation is to discuss the developments during last years in raster data support in GeoTools and GeoServer, and also to introduce and discuss future development directions.

Statistics

Views

Total Views
3,130
Views on SlideShare
2,185
Embed Views
945

Actions

Likes
2
Downloads
41
Comments
0

44 Embeds 945

http://geo-solutions.blogspot.it 273
http://geo-solutions.blogspot.com 200
http://geo-solutions.blogspot.pt 49
http://geo-solutions.blogspot.co.uk 38
http://geo-solutions.blogspot.de 34
http://cloud.feedly.com 30
http://geo-solutions.blogspot.com.es 25
http://cageyjames.webfactional.com 24
http://geo-solutions.blogspot.in 21
http://geo-solutions.blogspot.fr 20
http://www.directrss.co.il 19
http://geo-solutions.blogspot.com.au 18
http://geo-solutions.blogspot.se 15
http://geo-solutions.blogspot.ca 14
http://geo-solutions.blogspot.nl 13
http://geo-solutions.blogspot.be 13
http://geo-solutions.blogspot.sk 11
http://geo-solutions.blogspot.ro 11
http://geo-solutions.blogspot.dk 10
http://geo-solutions.blogspot.no 10
http://geo-solutions.blogspot.ru 9
http://geo-solutions.blogspot.com.ar 8
http://plus.url.google.com 7
http://geo-solutions.blogspot.co.il 7
http://geo-solutions.blogspot.gr 7
http://geo-solutions.blogspot.sg 6
http://geo-solutions.blogspot.mx 6
http://geo-solutions.blogspot.com.br 6
http://geo-solutions.blogspot.jp 5
http://geo-solutions.blogspot.fi 5
http://geo-solutions.blogspot.ch 4
http://geo-solutions.blogspot.kr 4
http://geo-solutions.blogspot.hk 3
http://geo-solutions.blogspot.cz 3
http://geo-solutions.blogspot.hu 3
http://geo-solutions.blogspot.co.at 3
http://translate.googleusercontent.com 2
http://geo-solutions.blogspot.co.nz 2
http://digg.com 2
http://feedly.com 1
http://www.tuicool.com 1
http://www.blogger.com 1
http://geo-solutions.blogspot.ie 1
http://geo-solutions.blogspot.tw 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments Raster Data In GeoServer And GeoTools: Achievements, Issues And Future Developments Presentation Transcript

  • Raster data in GeoServer and GeoTools: Achievements, issues and future developments Ing. Simone Giannecchini, GeoSolutions Ing. Daniele Romagnoli FOSS4G 2013, Nottingham 21th September 2013
  • Outline  Who we are  The building blocks  Recent Achievements  What’s cooking FOSS4G 2013, Nottingham 21th September 2013
  • GeoSolutions  Founded in Italy in late 2006  Expertise • • Java, Java Enterprise, C++, Python •  Image Processing, GeoSpatial Data Fusion JPEG2000, JPIP, Advanced 2D visualization Supporting/Developing FOSS4G projects    MapStore, GeoServer GeoBatch, GeoNetwork Clients    Public Agencies Private Companies http://www.geo-solutions.it FOSS4G 2013, Nottingham 21th September 2013
  • The Stack we’ll talk about FOSS4G 2013, Nottingham 21th September 2013
  • JAI          Java Advanced Imaging Library SUN then Oracle Image Processing library Huge number of basic building operations Tile-based, deferred execution model Integrate Tile Caching (In Memory) Easily Extensible Natively Accelerated Code available for many operations No source code for native ops Development more or less stopped! FOSS4G 2013, Nottingham 21th September 2013
  • ImageIO        SUN then Oracle Image I/O library Partly ships with JDK partly as extension Tile-based, deferred & immediate execution model Easily Extensible Natively Accelerated Code available for many formats No source code for native (de)coders Development more or less stopped! FOSS4G 2013, Nottingham 21th September 2013
  • JAITools  High performance raster processing  Extends, complements, replaces Oracle Java Advanced Imaging (JAI) Library   Low Level Operations   Open Source, BSD license e.g. RangeLookup Higher Level Operations  Vectorize, Contour  Works mostly in pure raster space (Image Processing)  Stable release is 1.3.0 FOSS4G 2013, Nottingham 21th September 2013
  • ImageIO-Ext  High Performance raster I/O    Extends, complements, replaces Oracle JAI ImageIO Library Open Source, LGPL and BSD licenses Plugins:    BigTiff*, MatFile 5, TurboJPEG JP2000 (Kakadu) GDAL integration via ImageIO-ext    JPEG2000, MrSID, ECW BigTIFF, ERDAS Image, HDF4 Complete list here FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  The Open Source Java GIS Toolkit   LGPL Management of Geospatial data  CRUD Operations  Raster and vector    Rendering GDAL Support Based on best-of-breed Open Source projects  Java Advanced Imaging (JAI), ImageIO  Java Topology Suite (JTS) FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  GeoSpatial enterprise gateway    Java Enterprise Management of raster and vector data Standards compliant   OGC WFS 1.0, 1.1 (RI), 2.0  OGC WMS 1.1.1, 1.3.0   OGC WCS 1.0, 1.1.1 (RI), 2.0.1 OGC WPS 1.0.0 Google Earth/Maps support  KML, GeoSearch, etc.. FOSS4G 2013, Nottingham 21th September 2013
  • All together now… WCS WMS WFS WPS REST Google GWC GeoTools ImageIO JTS JAI GDAL Kakadu Vector JAITools Style FOSS4G 2013, Nottingham 21th September 2013 Raster
  • Recent Achievements FOSS4G 2013, Nottingham 21th September 2013
  • ImageIO-Ext  LibJPEG-Turbo      derivative library of libjpeg based on SIMD instructions (MMX, SSE2, NEON) accelerate baseline JPEG compression and decompression on x86, x86-64, and ARM systems available on many platforms Open Source, LGPL ImageIO-Ext-TurboJPEG      A JPEG ImageIO plugin built on top of the JAVA wrapper for LibJPEG-Turbo (TurboJPEG) 20%-30% SpeedUp* on JPEG encoding 10%-20% SpeedUp* on JPEG decoding More info here Integrates with TIFF reader and writer FOSS4G 2013, Nottingham 21th September 2013
  • ImageIO-Ext     NITF encoding Support (ImageIO-Ext-NITF) Based on NITRO Support for Tagged Record Extensions (TREs) Support ImageSegments JPEG2000 compression through Kakadu ImageIO-Ext plugin       (Numerically Lossless/VisuallyLossless/Lossy) NPJE, EPJE BIIF (Basic Image Interchange Format) profiles J2KLRA TRE support Support for multiple TextSegments Support for multiple ImageSegments Support for ShapeFile Data Extension Segment (CSSHPA) FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools      Improved internal JAI Concurrency Introduced and Improved JAI Tile Recycling Reduced depth of JAI Processing Chains More aggressive factory caching On-the-fly Polygonalization Added a few specific high perf JAI operations    On-the-fly Contouring Created Specific high perf ROI class Improving Tile Caching   e.g. crop Work ongoing Band Transparent File vs Memory Caching when writing Combine + Raster to Symbology raster point vector FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools    Optimized Rendering/Processing for raster data Optimized Crop Chains  Coalescing multiple Crops into a single one Optimize Affine Transformation Chains  Coalescing multiple Affine Transformations into a single one NEAREST FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools BICUBIC FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  ImageMosaic  Custom Dimensions     More than simple time/elevation support  E.g.: wavelength, cloudLevel, … Dimensions with ranges: (startTime, endTime), (minElevation,maxElevation), … Dynamic read parameters Filtering  Supports for OGC Filters through the “filter” param   E.g.: Return only ImageMosaic granules where attribute Country = Italy. filter=“Country=‘Italy’ ” FOSS4G 2013, Nottingham 21th September 2013
  • ImageMosaic  Custom/Additional dimensions FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  ImageMosaic  Ordering    SORT_BY clause support on DBMS based mosaics sorting=“time A, elevation D” Stacking VS Merging  Merging: different granules  merged together in the final image as default  Stacking: different granules  stacked on different bands   E.g. MeteoSat data: 12 channels (1 granule for each Channel) 1 = VIS0.6, 2 = VIS0.8, …, 11 = IR13.4, 12 = HRV. Want return a RGB image made of channels 1,3,7  Stack them FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  New API (unsupported/coverage-experiment)     Driver: Get a connection to a coverage store. CoverageAccess: a physical storage of coverage data. Get access to a source of coverages. CoverageSource: provides data access to a coverage. Updated GridCoverage core API Old GridCoverageReader:  inefficient and thread unsafe way to access multiple coverages  read GridCoverages from the input stream in a sequential order New GridCoverage2DReader:  Coverages and properties may be accessed by coverage name:   get Envelope(name), get GridRange(name), get CRS(name), read(name), get dynamicParameters (name), … dynamicParameters support (for custom dimensions)   FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  NetCDF Support  Based on Unidata NetCDF JAVA library  Implements the new coverage-experiment API  Exposes current interfaces (GridCoverage2DReader,…) too for GeoServer compatibility before changes  1 coverage for each Dataset’s variable (multiple coverages).  Internal data exposed as set of 2D slices  Custom dimensions support  Ancillary XML file:  filter on NetCDF variables/coverages  define low level index schema (for dimensions) FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  ImageMosaic NetCDF integration  Allow the ImageMosaic to handle NetCDF files  Expose NetCDF internal structure (times, elevations)  Make ImageMosaic handle slices of the NetCDF file as granules Mosaic NetCDF 1 NetCDF … N NetCDF 2 … FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  NetCDF Polyphemus Sample Dataset FOSS4G 2013, Nottingham 20th September 2013
  • GeoTools  Multiple Coverages per Mosaic FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools    StructuredGridCoverage2DReader exposes the underlying granule structure and dimensions descriptors of a specific coverage  getGranules (return a GranuleSource)  getDimensionDescriptors allows to create/remove coverages and add new sources  createCoverage  removeCoverage  Harvest  GetGranules FOSS4G 2013, Nottingham 21th September 2013
  • ImageMosaic  Granule Index CRUD Operations via REST  CREATE curl -u admin:Geos -XPUT -H "Content-type:application/zip" --data-binary @http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/t emperature/file.imagemosaic  READ index schema curl -v -u admin:Geos -XGET "http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/polyphemus /coverages/NO2/index.xml"  READ WFS like with CQL filtering and paging curl -v -u admin:Geos -XGET "http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/p olyphemus/coverages/NO2/index/granules.xml?limit=1&filter=time='2013-0303T00:00:00Z'" FOSS4G 2013, Nottingham 21th September 2013
  • ImageMosaic  Granule Index CRUD Operations via REST  UPDATE curl -v -u admin:Geos -XPOST -H "Content-type: text/plain" -d "/polyphemus_20130303.nc" "http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/pol yphemus/external.imagemosaic"  DELETE WFS like with CQL filtering and paging or by ID curl -v -u admin:geoserver -XDELETE "http://localhost:8080/geoserver/rest/workspaces/topp/coveragestores/polyphemusv1/coverages/NO2/index/granules.xml?filter=location='polyphemus_20130301.nc'" curl -v -u admin:geoserver -XGET "http://localhost:8080/geoserver/rest/workspaces/topp/coveragestores/polyphemusv1/coverages/NO2/index/granules/NO2.2689.xml" FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  GranuleSource/GranuleStore  GranuleSource:  ReadOnly access to Granules (with Query support)      getGranules(Query)  SimpleFeatureCollection getSchema  SimpleFeatureType getCount(Query), getBounds(Query) E.g.: get all granules having acquisitionYear > 2012 and Country=Italy GranuleStore:  add/modify/delete capabilities using Transactions     addGranules (SimpleFeatureCollection) removeGranules (Filter) updateGranules E.g.: remove all granules having acquisitionYear < 1990 FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer       LibJPEG-Turbo Output format Encoding WMS output JPEG images through LibJPEG Turbo library Based on the TurboJpeg ImageIO-Ext plugin built on top of the JAVA interface for LibJPEG-Turbo (TurboJPEG) Provided as a GeoServer extension Up to 80% SpeedUp measured on JPEG encoding Can be disabled at startup:   -Ddisable.turbojpeg=true More Info here FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  More Color Quantizers MEDIANCUT OCTTREE Speed loss is around 10% FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  WMS Custom Dimensions  Work on GeoTools raster Additional Domains      NetCDF ImageMosaic Turn additional querying dimensions into WCS/WMS Dimensions  ImageMosaic Domain Category  WMS 1.1.1 DIM_CATEGORY  WMS 1.3.0 DIM_CATEGORY Perfect for large multidimensional mosaics! E.g. meteo (forecast time, elevation, band, base time, model name) FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  WMS TIME, ELEVATION & More http://localhost:8080/geoserver/geosolutions/wms?...&time=2013-0301T00:00:00.000Z&elevation=35.0&DIM_FILEDATE=2013-0301T00:00:00.000Z&DIM_UPDATED=2013-04-08T08:18:41.597Z FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  WMS-EO  GSIP 84 Wizard to configure EO layer groups Extending LayerGroup concept Support same style on both raster and vector data Support custom dimensions Alter map on the fly to support band combination FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  WMS-EO Quirks  Root layer must respond with a specific layer rather than the composition of the children    E.g. when I send a GetMap for the root layer I could get back the browse image default layer Same styles for both Raster as well as Vector data   Kind of a default visualization for a certain Dataset E.g. yellow can be use for both flags as well as outlines Peculiar behavior for Band layer ( raw data )   Can request either 1 (grayscale image) or 3 (RGB image)   E.g. multiple bands at different wavelengths Different combinations are prohibited Peculiar Behavior for GetFeatureInfo FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  WCS 2.0  The basics     GetCoverage extensions         CRS Scaling Interpolation Range subsetting GeoTiff GML NetCDF Output format extensions     Core service KVP binding XML binding GeoTIFF GML Grid NetCDF Ensure CITE tests compliancy FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer WCS 2.0 Processing Chain  Crop Core RangeSubset Extension Scale & Interpolate Extension Reproject & Interpolate Extension Encode Extension FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer     NetCDF Output Format GetCoverage request parameter: &Format=NetCDF Based on Unidata NetCDF JAVA library Create a NetCDF output file on top of a GranuleStack:  A GridCoverage2D List:    Supports WCS 2.0.1    a GridCoverage2D for each combination of requested Dimension values GridCoverages should come from a StructuredGridCoverage2DReader Trimming: specify values range for a dimension Slicing: specify single value for a dimension Supports timeRanges/numberRanges on output FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  NetCDF Output http://localhost:8080/geoserver/wcs?request=Get Coverage&service=WCS&version=2.0.1&coverag eId=geosolutions__NO2&Format=NetCDF&subse t=http://www.opengis.net/def/axis/OGC/0/Long(5, 20)&subset=http://www.opengis.net/def/axis/OG C/0/Lat(40,50)&subset=http://www.opengis.net/de f/axis/OGC/0/elevation(300,1250)&subset=http://w ww.opengis.net/def/axis/OGC/0/time("2013-0301T10:00:00.000Z","2013-03-01T22:00:00.000Z") FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  NetCDF Output FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  WCS-EO  Build on top of a working WCS 2.0 with full extensions         WCS 2.0 CRS extension Range subsetting extension Interpolation extension Scaling extension GeoTiff extension NetCDF extension Add support for the WCS-EO extras    Listing coverage datasets in the capabilities documents (based on image mosaic contents, which will have to be marked as “exposed” so that we show their inner structure for EO) Support describe coverage dataset Support returning results for an entire dataset in GetCoverage FOSS4G 2013, Nottingham 21th September 2013
  • What’s cooking Plus what’s on our wish list FOSS4G 2013, Nottingham 21th September 2013
  • JAI-Ext (temporary name)         Oracle JAI Drop-In Replacement Open Source (License TBD) High Performance Pure Java Support for No-Data Support for Processing ROI Support for Band Masks GPU Acceleration  OpenCL VS JCuda VS Aparapi FOSS4G 2013, Nottingham 21th September 2013
  • JAI-Ext (temporary name)  Status            Mosaic Scale Affine Translate Lookup BandSelect BandMerge Warp ColorQuantize Format    Rescale Histogram Extrema …. FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  Improved Processing & Rendering  Raster Algebra embedded in Styles    Improved Contrast Stretch      Processing on-the-fly with a Rendering Transformation Embed into RasterSymbolizer (ChannelSelection) Clip to Min-Max, Gaussian, etc… Suport for Date-line crossing raster Support for raster in Polar projections Better No-Data Management Support for VirtualRaster a-la GDAL FOSS4G 2013, Nottingham 21th September 2013
  • GeoTools  Mosaic Masking FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer     High Performance/Pure Java PNG output format No more separate Native ImageIO Install More Control on bugs/enhancements Status       First prototype working Same performance as Native Imageio Much Faster then JDK Encoder (check this) Code here, GPL| Discussion ongoing with the PNGJ Open Source project for joining efforts More info here FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  OSM Generated Image FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  Syntethic Image FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer   Raster Algebra as WPS Gets an OGC Filter containing logical operations between raster layers and produces a binary (0/1) layer FOSS4G 2013, Nottingham 21th September 2013
  • GeoServer  Raster Algebra as WPS  JIFFLE FOSS4G 2013, Nottingham 21th September 2013
  • The End Thanks for not sleeping (loudly)! daniele.romagnoli@geo-solutions.it simone.giannecchini@geo-solutions.it FOSS4G 2013, Nottingham 21th September 2013