Raster data in GeoServer and GeoTools: Achievements, issues and future developments
Upcoming SlideShare
Loading in...5
×
 

Raster data in GeoServer and GeoTools: Achievements, issues and future developments

on

  • 3,548 views

Status and future plans of raster support in GeoTools and GeoServer open source projects

Status and future plans of raster support in GeoTools and GeoServer open source projects

Statistics

Views

Total Views
3,548
Views on SlideShare
3,548
Embed Views
0

Actions

Likes
0
Downloads
65
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Raster data in GeoServer and GeoTools: Achievements, issues and future developments Raster data in GeoServer and GeoTools: Achievements, issues and future developments Presentation Transcript

  • Raster data in GeoServer and GeoTools:Achievements, issues and future developments Ing. Simone Giannecchini, GeoSolutions Ing. Daniele Romagnoli FOSS4G 2011, Denver 12th-16th September 2011
  • Outline Who we are Raster Developments  JAITools  ImageIO-Ext  GeoTools  GeoServer  GeoBatch FOSS4G 2011, Denver 12th-16th September 2011
  • GeoSolutions Founded in Italy in 2006 Expertise • Image Processing, GeoSpatial Data Fusion • Java, Java Enterprise, C++, Python • JPEG2000, JPIP, Advanced 2D visualization Supporting/Developing FOSS4G projects  GeoTools, GeoServer  GeoBatch, GeoNetwork Clients  Public Agencies  Private Companies http://www.geo-solutions.it FOSS4G 2011, Denver 12th-16th September 2011
  • JAITools High performance raster processing  Extends, complements, replaces Oracle Java Advanced Imaging (JAI) Library  Open Source, BSD license Low Level Operations  e.g. RangeLookup Higher Level Operations  Vectorize, Contour Works mostly in pure raster space (Image Processing) Stable release is 1.2.0 FOSS4G 2011, Denver 12th-16th September 2011
  • JAITools – Low Level Operations Basic Extensions to JAI processing RangeLookup  Raster (re)classification Raster Statistics  ZonalStats  ClassifiedStats*  KernelStats High Perf ColorMap*  ColorMap Extraction/Inversion High Perf VectorBinarize  Vector to raster conversion FOSS4G 2011, Denver 12th-16th September 2011
  • JAITools – High Level Operations Vectorize  Polygon Extraction Contour  Isolines extraction Raster Algebra (Jiffle)  MapCalc like extension  Generic Raster Processing FOSS4G 2011, Denver 12th-16th September 2011
  • ImageIO-Ext High Performance raster I/O  Extends, complements, replaces Oracle JAI ImageIO Library  Open Source, LGPL and BSD licenses Plugins:  BigTiff*, netCDF-CF,GriB1,MatFile 5, HDF  JP2000 (Kakadu) GDAL integration via ImageIO-ext  JPEG2000, MrSID, ECW  BigTIFF, ERDAS Image, HDF4  Complete list here FOSS4G 2011, Denver 12th-16th September 2011
  • ImageIO-Ext – Improving Tiff Improved standard Tiff reader and writer  Support for r/w double and float data type  JP2 compressor/decompressor* Improved metadata Management  More aggressive caching  Based on soft references BigTiff support  Read/write  Reduced resolution page support  Overviews! FOSS4G 2011, Denver 12th-16th September 2011
  • ImageIO-Ext – JPEG2000 Kakadu based reader/writer  Wraps kakadu JNI APIs as ImageIO ImageReader/ImageWriter Support for GeoJP2 Supporting additional JPEG2000 code params  Tiling, Corder, Tile Marker, Part Tile Marker, Sprofile , etc…  Some of them needed for NITF support Fine grain control over JPEG2000 codestream creation FOSS4G 2011, Denver 12th-16th September 2011
  • ImageIO-Ext – GDAL Wraps GDAL behind ImageIO API  Mainly Read Capabilities  Write support experimental (JP2Kak, GeoTiff)  Geotiff, JP2Kak, ECW, MrSID, JP2MrSid, Erdas, IDRIS, Esri Binary Grid, Envisat, NITF, DTED, DOQ1, DOQ2, etc.. Expose Generic IIOMetadata for basic Dataset info  Georeferencing, Band layout, CRS, etc.. Expose Specific IIoMetadata for Driver specific info  E.g. MrSid metadata 1.1.0 supports GDAL 1.7.3 directly (no custom bindings!) FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools The Open Source Java GIS Toolkit  LGPL Management of Geospatial data  CRUD Operations  Raster and vector  Rendering  GDAL Support Based on best-of-breed Open Source projects  Java Advanced Imaging (JAI), ImageIO  Java Topology Suite (JTS) FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools - JPEG2K Improvements Leverage on ImageIO API  Read/Write Capabilities  Fine grain support for writing with code params GeoJP2 support Other capabilities  Re-Tiling on the fly  ColorMasking  Immediate (Java2D) VS Deferred Mode (JAI)  Multithreaded loading (per tile) FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools - Tiff & BigTiff Leverages on ImageIO API  Read/Write Capabilities Internal/External Overviews support Improved Metadata management  Less memory  Wise caching – more speed Heavy Refactor ongoing  Re-Tiling on the fly  ColorMasking  Immediate (Java2D) VS Deferred Mode (JAI) FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools - Raster Processes Basic Building Blocks for GeoServer WPS  Clip and Ship (raster and vector), buffering, vector intersection, vector union, etc… JAITOOLs  Vectorize, Contour  Statistics GDAL based*  gdal_translate wrapper  gdalwarp wrapper  org2ogr wrapper FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools - ImageMosaic Wraps multiple homogeneus raster as a single one Improvements  Support for footprint on single tiles  Support for external catalog of tiles  Autoindexing of directories  Support for TIME, ELEVATION  Support for OGC & CQL Filtering  Multithreaded loading  Support for heterogeneous overviews layout  Supports any format with an ImageIO reader  Re-Tiling on the fly  ColorMasking on the fly  Immediate VS Deferred Mode FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools - ImagePyramid Multiresolution/Multiscale tile based raster plugin Take a large raster  split in small chunks/tiles  mosaic and downsample start over until happy Upon request  Select right resolution level  select needed chunks/tiles  assemble  do some magic and return Improvements  Automatically recognize output of gdal_retile  Automatic tile indexing  Other improvements inherited from ImageMosaic FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools – Optimized Reprojection Standard Approach  Accurate Transformation  Each pixel is back mapped using the full transformation  Grid to world on the target  Backwards projection towards the source  World to grid on the source  Slow!, cannot be optimized in hardware Optimization  Approximate Transformation  Local Linear Approximation using Grid Warps  Use a set of known positions and their transformations  Everything else is linearly interpolated between the known positions  Estimation through iterative process (speed vs quality) FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools – Optimized Reprojection Linearly interpolate on the top and bottom lines Linearly interpolate between the two resulting points FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools – Optimized Reprojection FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer GeoSpatial enterprise gateway  Java Enterprise  Management of raster and vector data Standards compliant  OGC WCS 1.0, 1.1.1 (RI)  OGC WFS 1.0, 1.1 (RI), 2.0  OGC WMS 1.1.1, 1.3  OGC WPS 1.0.0 Google Earth/Maps support  KML, GeoSearch, etc.. FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer - Rendering Transformations In Place Transformations during rendering Specified in SLD docs  Applied right before the rest of the directives Can be used to:  Extract and draw contours on the fly  Extract and draw polygons on the fly  Extract and draw symbols (e.g. arrows) on the fly  Transform geometries on the fly  (Add your own!) FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer - Rendering Transformations On-the-fly Polygonalization On-the-fly Contouring Band Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer – Improved Raster Rendering Hit JAVA2D Bottleneck – Scalability Issue  http://bit.ly/qJcZBi  http://bit.ly/oe0CHo Created direct raster rendering path with JAI On-the-fly Polygonalization  Drop-in replacement (1 raster at time) On-the-fly  2x speedup Contouring  2x/3x scalability improvement under heavy load Enabled/Disabled via Java Switch Band Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer – Improved Raster Chains Performance Improved internal JAI Concurrency Introduced and Improved JAI Tile Recycling Reduced depth of JAI Processing Chains More aggressive factory caching On-the-fly Added a few specific high perf JAI operations Polygonalization  e.g. crop On-the-fly Contouring Created Specific high perf ROI class Improving Tile Caching  Work ongoing Band Transparent File vs Memory Caching when writing Raster to Combine + raster point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer – More Raster WPS Processes Relies on GeoTools Processes  Vectorize  Contouring  RasterClassification On-the-fly Polygonalization  ZonalStats On-the-fly Contouring  GDAL Processes  gdal_translate  gdalwarp Band  ogr2ogr Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  • GeoServer – Support for TIME & ELEVATION dimensions + Filters WMS TIME and ELEVATION dimensions  …&TIME=20110914T00:00:000Z&ELEVATION=10  Implemented thanks to ImageMosaic  Support for the Current keyword On-the-fly  Support for Default values Polygonalization  Support for On-the-fly  Contouring TimeInstant  Discrete Range Tb/Te/Res, Eb/Ee/Res  Continuous Range Tb/Te, Eb/Ee  Any combination of the above Support for OGC and CQL filters Band  Raster to Filter Mosaic granules based on index Combine + attributes point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  • GeoBatch Real-time produced data  Remote Sensing, In situ, Models, etc.. Recognizable, isolate data streams Flow  Incoming series of geospatial data  Specific format  Defined rules for preprocessing  Defined rules for ingestion and/or exploitation  Automatic Ingestion and preprocessing  No manual intervention Example, GeoTiff preprocessing GeoNetworkProducer Tile Overviews Publish GeoServer FOSS4G 2011, Denver 12th-16th September 2011
  • GeoBatch – GeoTiff Preprocessing GeoTiff serving optimizations  Internal Tiling  Compression  Internal/External Overviews  Mosaicking On-the-fly Polygonalization  Pyramiding  On-the-fly GeoServer publishing Contouring 2 Options  Pure Java based on ImageIO and GeoTools  External Processes with calls to GDAL Band Raster to Combine + point vector Symbology FOSS4G 2011, Denver 12th-16th September 2011
  • GeoBatch – MetOc Data Preprocessing Optimization of MetOc for WMS/WCS serving  Conversion from  GRiB 1  netCDF-CF  COAMPS On-the-fly Polygonalization  BUFr* On-the-fly  To Contouring  Multiple 2D geotiff (Internal Overviews)  Indexing with ImageMosaic More preprocessing  Fixing No Data Band  Change UoM Raster to Combine + point vector Symbology  Band Processing Wind(U,V)  Wind ( mod, theta) FOSS4G 2011, Denver 12th-16th September 2011
  • The End Questions?simone.giannecchini@geo-solutions.it FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools - ImageMosaic FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools – Optimized Reprojection A raster reprojection involves chaining togheter the standard reprojection with the grid to world transformations of source and target FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools – Optimized Reprojection Set of evaluation vectors used in the Recursive algorithm Test distance between actual position of the reprojected point and the predicted by a linear interpolation FOSS4G 2011, Denver 12th-16th September 2011
  • GeoTools – Optimized Reprojection FOSS4G 2011, Denver 12th-16th September 2011