Raster data in GeoServer and GeoTools:Achievements, issues and future developments       Ing. Simone Giannecchini, GeoSolu...
Outline   Who we are   Raster Developments       JAITools       ImageIO-Ext       GeoTools       GeoServer       Ge...
GeoSolutions   Founded in Italy in 2006   Expertise    •   Image Processing, GeoSpatial Data Fusion    •   Java, Java En...
JAITools   High performance raster processing       Extends, complements, replaces Oracle Java Advanced        Imaging (...
JAITools – Low Level Operations   Basic Extensions to JAI processing   RangeLookup       Raster (re)classification   R...
JAITools – High Level Operations   Vectorize       Polygon Extraction   Contour       Isolines extraction   Raster Al...
ImageIO-Ext   High Performance raster I/O        Extends, complements, replaces Oracle JAI ImageIO         Library     ...
ImageIO-Ext – Improving Tiff   Improved standard Tiff reader and writer         Support for r/w double and float data ty...
ImageIO-Ext – JPEG2000   Kakadu based reader/writer       Wraps kakadu JNI APIs as ImageIO        ImageReader/ImageWrite...
ImageIO-Ext – GDAL   Wraps GDAL behind ImageIO API       Mainly Read Capabilities       Write support experimental (JP2...
GeoTools   The Open Source Java GIS Toolkit        LGPL   Management of Geospatial data        CRUD Operations        ...
GeoTools - JPEG2K Improvements   Leverage on ImageIO API       Read/Write Capabilities       Fine grain support for wri...
GeoTools - Tiff & BigTiff   Leverages on ImageIO API       Read/Write Capabilities   Internal/External Overviews suppor...
GeoTools - Raster Processes   Basic Building Blocks for GeoServer WPS       Clip and Ship (raster and vector), buffering...
GeoTools - ImageMosaic   Wraps multiple homogeneus raster as a single one   Improvements       Support for footprint on...
GeoTools - ImagePyramid   Multiresolution/Multiscale tile based raster    plugin   Take a large raster  split in small ...
GeoTools – Optimized Reprojection   Standard Approach  Accurate Transformation       Each pixel is back mapped using th...
GeoTools – Optimized Reprojection          Linearly interpolate on the top and bottom lines          Linearly interpolate ...
GeoTools – Optimized Reprojection           FOSS4G 2011, Denver          12th-16th September 2011
GeoServer   GeoSpatial enterprise gateway        Java Enterprise        Management of raster and vector data   Standar...
GeoServer - Rendering Transformations   In Place Transformations during rendering     Specified in SLD docs     Applied...
GeoServer - Rendering Transformations                                             On-the-fly                              ...
GeoServer – Improved Raster Rendering   Hit JAVA2D Bottleneck – Scalability Issue       http://bit.ly/qJcZBi       http...
GeoServer – Improved Raster Chains               Performance   Improved internal JAI Concurrency   Introduced and Improv...
GeoServer – More Raster WPS Processes   Relies on GeoTools Processes       Vectorize       Contouring       RasterClas...
GeoServer – Support for TIME &        ELEVATION dimensions + Filters   WMS TIME and ELEVATION dimensions       …&TIME=20...
GeoBatch   Real-time produced data       Remote Sensing, In situ, Models, etc..   Recognizable, isolate data streams  ...
GeoBatch – GeoTiff Preprocessing   GeoTiff serving optimizations       Internal Tiling       Compression       Interna...
GeoBatch – MetOc Data Preprocessing   Optimization of MetOc for WMS/WCS serving     Conversion from           GRiB 1   ...
The End        Questions?simone.giannecchini@geo-solutions.it            FOSS4G 2011, Denver           12th-16th September...
GeoTools - ImageMosaic     FOSS4G 2011, Denver    12th-16th September 2011
GeoTools – Optimized Reprojection                                     A raster reprojection                               ...
GeoTools – Optimized Reprojection                       Set of evaluation vectors used in the                       Recurs...
GeoTools – Optimized Reprojection           FOSS4G 2011, Denver          12th-16th September 2011
Upcoming SlideShare
Loading in...5
×

Raster data in GeoServer and GeoTools: Achievements, issues and future developments

3,264

Published on

Status and future plans of raster support in GeoTools and GeoServer open source projects

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,264
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
71
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Raster data in GeoServer and GeoTools: Achievements, issues and future developments

  1. 1. Raster data in GeoServer and GeoTools:Achievements, issues and future developments Ing. Simone Giannecchini, GeoSolutions Ing. Daniele Romagnoli FOSS4G 2011, Denver 12th-16th September 2011
  2. 2. Outline Who we are Raster Developments  JAITools  ImageIO-Ext  GeoTools  GeoServer  GeoBatch FOSS4G 2011, Denver 12th-16th September 2011
  3. 3. GeoSolutions Founded in Italy in 2006 Expertise • Image Processing, GeoSpatial Data Fusion • Java, Java Enterprise, C++, Python • JPEG2000, JPIP, Advanced 2D visualization Supporting/Developing FOSS4G projects  GeoTools, GeoServer  GeoBatch, GeoNetwork Clients  Public Agencies  Private Companies http://www.geo-solutions.it FOSS4G 2011, Denver 12th-16th September 2011
  4. 4. JAITools High performance raster processing  Extends, complements, replaces Oracle Java Advanced Imaging (JAI) Library  Open Source, BSD license Low Level Operations  e.g. RangeLookup Higher Level Operations  Vectorize, Contour Works mostly in pure raster space (Image Processing) Stable release is 1.2.0 FOSS4G 2011, Denver 12th-16th September 2011
  5. 5. JAITools – Low Level Operations Basic Extensions to JAI processing RangeLookup  Raster (re)classification Raster Statistics  ZonalStats  ClassifiedStats*  KernelStats High Perf ColorMap*  ColorMap Extraction/Inversion High Perf VectorBinarize  Vector to raster conversion FOSS4G 2011, Denver 12th-16th September 2011
  6. 6. JAITools – High Level Operations Vectorize  Polygon Extraction Contour  Isolines extraction Raster Algebra (Jiffle)  MapCalc like extension  Generic Raster Processing FOSS4G 2011, Denver 12th-16th September 2011
  7. 7. ImageIO-Ext High Performance raster I/O  Extends, complements, replaces Oracle JAI ImageIO Library  Open Source, LGPL and BSD licenses Plugins:  BigTiff*, netCDF-CF,GriB1,MatFile 5, HDF  JP2000 (Kakadu) GDAL integration via ImageIO-ext  JPEG2000, MrSID, ECW  BigTIFF, ERDAS Image, HDF4  Complete list here FOSS4G 2011, Denver 12th-16th September 2011
  8. 8. ImageIO-Ext – Improving Tiff Improved standard Tiff reader and writer  Support for r/w double and float data type  JP2 compressor/decompressor* Improved metadata Management  More aggressive caching  Based on soft references BigTiff support  Read/write  Reduced resolution page support  Overviews! FOSS4G 2011, Denver 12th-16th September 2011
  9. 9. ImageIO-Ext – JPEG2000 Kakadu based reader/writer  Wraps kakadu JNI APIs as ImageIO ImageReader/ImageWriter Support for GeoJP2 Supporting additional JPEG2000 code params  Tiling, Corder, Tile Marker, Part Tile Marker, Sprofile , etc…  Some of them needed for NITF support Fine grain control over JPEG2000 codestream creation FOSS4G 2011, Denver 12th-16th September 2011
  10. 10. ImageIO-Ext – GDAL Wraps GDAL behind ImageIO API  Mainly Read Capabilities  Write support experimental (JP2Kak, GeoTiff)  Geotiff, JP2Kak, ECW, MrSID, JP2MrSid, Erdas, IDRIS, Esri Binary Grid, Envisat, NITF, DTED, DOQ1, DOQ2, etc.. Expose Generic IIOMetadata for basic Dataset info  Georeferencing, Band layout, CRS, etc.. Expose Specific IIoMetadata for Driver specific info  E.g. MrSid metadata 1.1.0 supports GDAL 1.7.3 directly (no custom bindings!) FOSS4G 2011, Denver 12th-16th September 2011
  11. 11. GeoTools The Open Source Java GIS Toolkit  LGPL Management of Geospatial data  CRUD Operations  Raster and vector  Rendering  GDAL Support Based on best-of-breed Open Source projects  Java Advanced Imaging (JAI), ImageIO  Java Topology Suite (JTS) FOSS4G 2011, Denver 12th-16th September 2011
  12. 12. GeoTools - JPEG2K Improvements Leverage on ImageIO API  Read/Write Capabilities  Fine grain support for writing with code params GeoJP2 support Other capabilities  Re-Tiling on the fly  ColorMasking  Immediate (Java2D) VS Deferred Mode (JAI)  Multithreaded loading (per tile) FOSS4G 2011, Denver 12th-16th September 2011
  13. 13. GeoTools - Tiff & BigTiff Leverages on ImageIO API  Read/Write Capabilities Internal/External Overviews support Improved Metadata management  Less memory  Wise caching – more speed Heavy Refactor ongoing  Re-Tiling on the fly  ColorMasking  Immediate (Java2D) VS Deferred Mode (JAI) FOSS4G 2011, Denver 12th-16th September 2011
  14. 14. GeoTools - Raster Processes Basic Building Blocks for GeoServer WPS  Clip and Ship (raster and vector), buffering, vector intersection, vector union, etc… JAITOOLs  Vectorize, Contour  Statistics GDAL based*  gdal_translate wrapper  gdalwarp wrapper  org2ogr wrapper FOSS4G 2011, Denver 12th-16th September 2011
  15. 15. GeoTools - ImageMosaic Wraps multiple homogeneus raster as a single one Improvements  Support for footprint on single tiles  Support for external catalog of tiles  Autoindexing of directories  Support for TIME, ELEVATION  Support for OGC & CQL Filtering  Multithreaded loading  Support for heterogeneous overviews layout  Supports any format with an ImageIO reader  Re-Tiling on the fly  ColorMasking on the fly  Immediate VS Deferred Mode FOSS4G 2011, Denver 12th-16th September 2011
  16. 16. GeoTools - ImagePyramid Multiresolution/Multiscale tile based raster plugin Take a large raster  split in small chunks/tiles  mosaic and downsample start over until happy Upon request  Select right resolution level  select needed chunks/tiles  assemble  do some magic and return Improvements  Automatically recognize output of gdal_retile  Automatic tile indexing  Other improvements inherited from ImageMosaic FOSS4G 2011, Denver 12th-16th September 2011
  17. 17. GeoTools – Optimized Reprojection Standard Approach  Accurate Transformation  Each pixel is back mapped using the full transformation  Grid to world on the target  Backwards projection towards the source  World to grid on the source  Slow!, cannot be optimized in hardware Optimization  Approximate Transformation  Local Linear Approximation using Grid Warps  Use a set of known positions and their transformations  Everything else is linearly interpolated between the known positions  Estimation through iterative process (speed vs quality) FOSS4G 2011, Denver 12th-16th September 2011
  18. 18. GeoTools – Optimized Reprojection Linearly interpolate on the top and bottom lines Linearly interpolate between the two resulting points FOSS4G 2011, Denver 12th-16th September 2011
  19. 19. GeoTools – Optimized Reprojection FOSS4G 2011, Denver 12th-16th September 2011
  20. 20. GeoServer GeoSpatial enterprise gateway  Java Enterprise  Management of raster and vector data Standards compliant  OGC WCS 1.0, 1.1.1 (RI)  OGC WFS 1.0, 1.1 (RI), 2.0  OGC WMS 1.1.1, 1.3  OGC WPS 1.0.0 Google Earth/Maps support  KML, GeoSearch, etc.. FOSS4G 2011, Denver 12th-16th September 2011
  21. 21. GeoServer - Rendering Transformations In Place Transformations during rendering Specified in SLD docs  Applied right before the rest of the directives Can be used to:  Extract and draw contours on the fly  Extract and draw polygons on the fly  Extract and draw symbols (e.g. arrows) on the fly  Transform geometries on the fly  (Add your own!) FOSS4G 2011, Denver 12th-16th September 2011
  22. 22. GeoServer - Rendering Transformations On-the-fly Polygonalization On-the-fly Contouring Band Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  23. 23. GeoServer – Improved Raster Rendering Hit JAVA2D Bottleneck – Scalability Issue  http://bit.ly/qJcZBi  http://bit.ly/oe0CHo Created direct raster rendering path with JAI On-the-fly Polygonalization  Drop-in replacement (1 raster at time) On-the-fly  2x speedup Contouring  2x/3x scalability improvement under heavy load Enabled/Disabled via Java Switch Band Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  24. 24. GeoServer – Improved Raster Chains Performance Improved internal JAI Concurrency Introduced and Improved JAI Tile Recycling Reduced depth of JAI Processing Chains More aggressive factory caching On-the-fly Added a few specific high perf JAI operations Polygonalization  e.g. crop On-the-fly Contouring Created Specific high perf ROI class Improving Tile Caching  Work ongoing Band Transparent File vs Memory Caching when writing Raster to Combine + raster point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  25. 25. GeoServer – More Raster WPS Processes Relies on GeoTools Processes  Vectorize  Contouring  RasterClassification On-the-fly Polygonalization  ZonalStats On-the-fly Contouring  GDAL Processes  gdal_translate  gdalwarp Band  ogr2ogr Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  26. 26. GeoServer – Support for TIME & ELEVATION dimensions + Filters WMS TIME and ELEVATION dimensions  …&TIME=20110914T00:00:000Z&ELEVATION=10  Implemented thanks to ImageMosaic  Support for the Current keyword On-the-fly  Support for Default values Polygonalization  Support for On-the-fly  Contouring TimeInstant  Discrete Range Tb/Te/Res, Eb/Ee/Res  Continuous Range Tb/Te, Eb/Ee  Any combination of the above Support for OGC and CQL filters Band  Raster to Filter Mosaic granules based on index Combine + attributes point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  27. 27. GeoBatch Real-time produced data  Remote Sensing, In situ, Models, etc.. Recognizable, isolate data streams Flow  Incoming series of geospatial data  Specific format  Defined rules for preprocessing  Defined rules for ingestion and/or exploitation  Automatic Ingestion and preprocessing  No manual intervention Example, GeoTiff preprocessing GeoNetworkProducer Tile Overviews Publish GeoServer FOSS4G 2011, Denver 12th-16th September 2011
  28. 28. GeoBatch – GeoTiff Preprocessing GeoTiff serving optimizations  Internal Tiling  Compression  Internal/External Overviews  Mosaicking On-the-fly Polygonalization  Pyramiding  On-the-fly GeoServer publishing Contouring 2 Options  Pure Java based on ImageIO and GeoTools  External Processes with calls to GDAL Band Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  29. 29. GeoBatch – MetOc Data Preprocessing Optimization of MetOc for WMS/WCS serving  Conversion from  GRiB 1  netCDF-CF  COAMPS On-the-fly Polygonalization  BUFr* On-the-fly  To Contouring  Multiple 2D geotiff (Internal Overviews)  Indexing with ImageMosaic More preprocessing  Fixing No Data Band  Change UoM Raster to Combine + point vector Symbology  Band Processing Wind(U,V)  Wind ( mod, theta) FOSS4G 2011, Denver 12th-16th September 2011
  30. 30. The End Questions?simone.giannecchini@geo-solutions.it FOSS4G 2011, Denver 12th-16th September 2011
  31. 31. GeoTools - ImageMosaic FOSS4G 2011, Denver 12th-16th September 2011
  32. 32. GeoTools – Optimized Reprojection A raster reprojection involves chaining togheter the standard reprojection with the grid to world transformations of source and target FOSS4G 2011, Denver 12th-16th September 2011
  33. 33. GeoTools – Optimized Reprojection Set of evaluation vectors used in the Recursive algorithm Test distance between actual position of the reprojected point and the predicted by a linear interpolation FOSS4G 2011, Denver 12th-16th September 2011
  34. 34. GeoTools – Optimized Reprojection FOSS4G 2011, Denver 12th-16th September 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×