Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

State of GeoServer, GeoTools and Friends 2014


Published on

GeoServer and the Java toolkit GeoTools comprise one of OSGeo’s fastest moving mapping ecosystems. In addition to this core we will cover java-stack developments in JTS, GeoWebCache and ImageIO-Ext.

First up is “State of GeoServer” reviewing the new and noteworthy features introduced in the past year. Our six month release cycle sees GeoServer 2.5 and 2.6 being released this year. These releases bring together exciting new features: WCS 2.0 and WCS 2.0 Earth Observation profile, batch importer, and a fresh implementation of GetFeatureInfo.

Switching to technical for a GeoTools update. A passion for performance sees improvements in PNG and JPEG encoding, rendering from PostGIS and experimental JDK work. This presentation provides a review of new features, api changes and community modules. We have simplified the core FeatureCollection interface, introduced partial 3D support, structured grid coverages, multiple grid coverages and extended JTS Geometry with Curves.

Data formats support continues to grow with the latest GeoPackage, native NetCDF support and a new shapefile and wfs client implementation. Database users have table and index management methods to look forward to. Extensions have seen a lot of activity with WMS client improvements, vector grids, and a transform extension for dynamic feature collection processing.

Attend this talk for a cheerful update on what is happening with these great OSGeo projects! Whether you are an expert user, a developer, or simply curious what these projects can do for you, this talk is for you.

Published in: Software
  • Be the first to comment

State of GeoServer, GeoTools and Friends 2014

  1. 1. GeoServer, GeoTools and Friends 2014 FOSS4G Tech Talk
  2. 2. About your Presenters Contributors, Guests, and Cameos Andrea Aime (GeoSolutions) Martin Davis (JTS) Justin Deoliveira (Boundless) Jody Garnett (Boundless) Kevin Smith (GeoWebCache)
  3. 3. Outline ● Going to go fast - and have time for Q&A ● Intro o GeoServer 2014 o GeoTools 2014 o JTS 2014 (Martin Davis) o GWC 2014 o GeoScript 2014 o Friends ● Features ● Roadmap Wishlist ● Q&A
  4. 4. Money - Here is how this works ● Thanks to customers/employers/sponsors we can share a preview of some work scheduled for 2014 o We thank customers early and often o it is how open source software gets made
  5. 5. Love - here is how this works ● If you have an idea or can help out volunteers are always needed o Contact details provided for active development o Some wishes are ten years old ● Do not assume o Anyone will pay for a visual SLD editor (sorry users) o A volunteer is testing Java 8 (sorry developers)
  6. 6. GeoServer 2014
  7. 7. GeoServer Designed for interoperability, it publishes data from any major spatial data source using open standards.
  8. 8. GeoServer: History & Organisations ● Quick History … o OpenPlans: WFS 2002 o Refractions: UI 2003 o Axios: WMS 2003 ● Open Source Geospatial Foundation o Incubation 2009 o Graduated 2013 o New! © OpenPlans → © OSGeo 2014
  9. 9. GeoServer Highlights for 2014 ● GeoServer 2.4: released September 2014 o Served as stable branch for six months o Extended for six months as maintenance branch ● GeoServer 2.5: released April 2014 o Current stable branch o Scheduled six months of maintenance updates ● Governance o Transfer of codebase from OpenPlans to OSGeo o New “community” modules → Graduated extensions
  10. 10. GeoServer: Health Checkup ● 28 members of team geoserver o 779 geoserver-devel! o Core group of very active contributors o New contributors being added ● How about team user-list? o 2290 geoserver-users o 169 geoserver italia o ??? GeoServer en Espanol o ??? (GeoServer and more) o Some help with 2.5-beta … but many wait for 2.5.2 ● Active well managed codebase o 200 pull requests this year o New! file for GitHub
  11. 11. GeoServer: 2.5 Series
  12. 12. GeoServer: Downloads this Week
  13. 13. GeoServer moves to OSGeo ● Open plans is stepping back from software development ● Strong relationship with OSGeo o Vendor Neutral custodian to encourage participation from additional parties o Lazy: Participants had often already signed OSGeo Contributors License for GeoTools o See: GSIP-118
  14. 14. GeoTools 2014
  15. 15. GeoTools GeoTools is an open source Java library that provides tools for geospatial data.
  16. 16. GeoTools: History and Organisation ● GeoTools 1: 1996-2001 o Leeds University ● GeoTools 2: 2002-Present o OpenPlans, Refractions, GeoSolutions, etc ... ● Governance o OSGeo Incubation 2006 o Graduated 2008 o Copyright OSGeo as a vendor neutral organisation
  17. 17. GeoTools Highlights for 2014 ● GeoTools 10: released September 2014 o Served as stable branch for six months o Extended for six months as maintenance branch ● GeoTools 11.5: released April 2014 o Current stable branch o Scheduled six months of maintenance updates ● Governance o New “Unsupported” plugins → Graduated plugins o New Committers
  18. 18. GeoTools Health Checkup ● 27 members of team geotools o Active in bursts near release o New contributors being added ● Active well managed codebase o 203 pull requests so far this year o New! file for GitHub
  19. 19. GeoTools 11 Series Downloads
  20. 20. JTS Topology Suite 2014 Putting the Geometry in “Geo” since 2001
  21. 21. JTS Topology Suite 2014 ● Geometry ● SFSQL Implementation ● News ● LocationTech proposal ● LGPL → BSD / EPL ● Version 1.13 (2012) ● Version 1.14 (2014 Q4) Martin Davis Senior Technical Architect Vivid Solutions Inc.
  22. 22. GeoWebCache 2014
  23. 23. GeoWebCache 2014 ● Tile Cache o WMS-C o TMS o WMTS ● News o OpenPlans → OSGeo o Version 1.5 (2013 Q3) o Version 1.6 (2014 Q3) Kevin Smith Junior Software Developer Boundless Spatial Inc.
  24. 24. GWC: Mass Truncation via Rest External management of GeoWebCache! POST http://localhost:8080/geowebcache/rest/masstruncate <truncateLayer> <layerName>trekConventionDensity</layerName> </truncateLayer> Boundless Kevin Smith 1.5.0
  25. 25. GWC Features / Fixes ● GeoWebCache 1.5.0: August 2014 o Extensible Parameter Filters o First release synchronized with GeoServer ● Fixes/Improvements added to 1.5.3 o WMS Metadata Passthrough o WMS GetLegend Passthrough Volunteer Eric Dahl 1.5.1 GeoSolutions Andrea Aime 1.5.1
  26. 26. GeoWebCache Roadmap (1.6.0) ● Read ArcGIS Compact Caches o Generate with ArcGIS and Serve with GWC ● Improvements to Tile Fuser Independent Bjoern Saxe 1.6.0 GeoSolutions Nicola 1.6.0
  27. 27. GeoScript
  28. 28. Spatial capabilities for dynamic scripting languages ● Groovy ● Scala ● JavaScript ● Python Justin Deolivera GeoScript
  29. 29. GeoScript: Update ● Jared working hard o Curved geometries, GeoPackage, MBTiles ● Working on a common versioning scheme ● Standardizing on GeoTools version ● Early stages of incubation ● TODO: Logo :)
  30. 30. GeoScript for GeoServer
  31. 31. Friends 2014
  32. 32. ImageIO-Ext (more formats) Extensions, fixes and for Java ImageIO: ● Support for GDAL ● JPEG2000 via Kakadu ● Improved TIFF ● Now on GitHub! Simone Giannecchini
  33. 33. JAI-Ext (more image processing) ● more features to existing operations ● improved performances ● developing new operations Nicola Lagomarsini
  34. 34. JAI Tools Image Processing ● Tools for analysis ● Zonal-Stats ● Contours, vectorize Michael Bedward
  35. 35. Jiffle Raster Calculator ● Domain specific language for image processing ● C-Syntax for that curly brace comfort Michael Bedward
  36. 36. Features and Fixes The magic of Open Source Customer Implementor Release
  37. 37. WCS 2.0 A WCS version for human beings DLR Andrea Aime Simone Giannecchini Daniele Romagnoli 2.6.0
  38. 38. WCS protocol support ● New GeoTools classes to parse/encode the XML documents in the WCS 2.0 protocol
  39. 39. Full GeoServer implementation ● Implementing the full protocol ● Including the GeoTiff Extensions, controlling details of the generated GeoTiff file ● NetCDF output (see later)
  40. 40. Did I mention the protocol looks sane? http://localhost:8080/geoserver/wcs?request=GetCoverage&service=WCS&ver sion=2.0.1&coverageId=geosolutions__NO2 &Format=NetCDF &subset=,20) &subset=,50) &subset=,1250) &subset= ("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")
  41. 41. NetCDF/Grib Natively multidimensional DLR/Eumetsat Daniele Romagnoli 12.0/2.6.0
  42. 42. Many coverages per reader ● A GeoTools reader can read and advertise metadata for multiple coverages now ● GeoServer updated to follow suit
  43. 43. StructuredCoverageGridReader ● Drill down in the various dimensions ● Interact with slices, list, filter, add, update, delete (GranuleSource/GranuleStore) ● Harvest entire new files into the sources (mosaic only)
  44. 44. GeoServer support ● Publish NetCDF/Grib as any other format ● Configure WMS-T dimensions ● Return a multidimensional array via WCS 2.0 NetCDF output format http://localhost:8080/geoserver/wcs?request=GetCoverage &service=WCS&version=2.0.1&coverageId=geosolutions__NO2 &Format=NetCDF &subset=,20) &subset=,50) &subset=,1250) &subset= ("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")
  45. 45. Coverage reprojection improvements Handle difficult projections GeoSolutions Andrea Aime 12.0/2.6.0
  46. 46. Chilling beauty of polar stereographics Before After
  47. 47. Crossing the dateline with rasters
  48. 48. Faster output encoding Speeding up your PNG and JPEGs GeoSolutions Andrea Aime 2.5.0
  49. 49. GeoServer faster PNG encoder ● PNG encoder based on the PNGJ open source library ● Pure java, yet quite a bit faster
  50. 50. Put a turbo in that JPEG ● Based on the libJPEGTurbo encoder ● Significant speedup compared to JDK own encoder
  51. 51. Pixel precise GetFeatureInfo Because we care about your icons Eskilstuna Andrea Aime 2.5.0
  52. 52. GetFeatureInfo issues ● Most software does a small square bbox query ● Symbol sizes took into account to an extent (issues with dynamically sized symbols) ● No care for symbol shape ● Solution → actually draw stuff and check pixels
  53. 53. Dynamic sizes ● Size proportional to attribute ● Before, either miss the big circle, or report the small one as being hit ● Now right match
  54. 54. Shapes and offsets ● The new code won’t report a hit if you did not click the symbol ● Takes care of odd symbol shapes, and offsets
  55. 55. Curves support Everybody loves curves City of Vienna Andrea Aime 12.0/2.6.0
  56. 56. Represent curves in memory ● Circular arc by 3 points ● CircularString: sequence of the above ● CompoundString: mix straight and circular arcs ● Use in polygons as well
  57. 57. Smart linearization ● When rendering, linearize according to the rendering resolution ● Otherwise, if necessary, do so with a given tolerance (e.g., when using JTS topological operations) ● Selected code paths just keep the curves
  58. 58. GML encoding
  59. 59. Reading Curves ● Only from Oracle spatial
  60. 60. Reading Curves ● Contributions/funding to read from other data sources welcomed!
  61. 61. Coverage views Restructure your rasters in GeoServer GeoSolutions Daniele Romagnoli 2.6.0
  62. 62. Creating coverage views
  63. 63. Band selection
  64. 64. Coming soon: computed bands
  65. 65. Marlin Performance and scalability on all platforms Volunteer Laurent Bourges 2.6.0
  66. 66. Java rasterizer scalability issues ● Rasterizing: turning vector data into antialiased images (maps) ● Basically two choices o Use Oracle JDK (Ductus), binary, closed source, fast rasterization, but scales up poorly o Use OpenJDK (Pisces), pure java, open source, slower rasterization, but scales up well
  67. 67. Marlin rasterizer ● Fork of OpenJDK Pisces ● As fast as Ductus o as scalable as Pisces o Testing a complex multiscale map with 10 concurrent clients at different zoom levels
  68. 68. Marlin Easy Install For Windows/Linux, Oracle or OpenJDK: 1. Download one jar from 2. Add these options to the JVM-Xbootclasspath/ a:/path/to/marlin-X.Y.jar - Dsun.java2d.renderer=org.marlin.pisces.PiscesRenderingEngine 3. Done!
  69. 69. Marlin in Java 9? ● The author, Laurent Bourges, is trying to get Marlin merged in Java 9 ● Lack of response from Oracle ● Please Oracle, we care about performance… don’t you?
  70. 70. Filter 1.0 Classes Technical Debt / Cleanup Boundless Jody Garnett 12.0
  71. 71. Filter Interfaces - Technical Debt GeoTools 2.3 introduced Filter 1.1 Interfaces ● Budget only covered new implementation o GeoTools tests still used classes directly o GeoServer and other apps also ● Two options for Filter confuses developers o Class or Interface/Factory? o What do I import to make this code work? OpenGeo Justin Deolivera 2.3
  72. 72. Filter Cleanup - Cleanup GeoTools 12.0 Cleanup ● Cleaned up with / careful refactoring! ● Test for XML Parsers / WFS were hardest GeoServer 2.6.0 Cleanup ● Cleanup went smoothly ● BBOX Filters caused trouble Boundless Jody Garnett 12.0
  73. 73. wfs-ng Boundless Niels Charlier 12.0
  74. 74. WFS “Next Generation” client Ground up implementation of wfs-client ● Using GTXML parser (yay!) ● Based on ContentDataStore (implementation is much easier to follow) ● WFS-T 1.1 ← exciting!
  75. 75. WFS Cascade ● GeoServer surfaces the wfs-ng as the ability to “cascade” content from remote (or internal) WFS Servers ● Performing 2.6 preflight testing wfs-ng was a marked improvement over the origional o We have cut over “cold turkey” o We asked the geoserver-users list for help testing all those tricky INSPIRE services ● wfs-ng includes configuration overrides o You can account for several common mistakes (such as servers that get confused about axis order)
  76. 76. gt-render-wkt geotools-users Group Effort! 12.0
  77. 77. gt-render-wkt merged to gt-render ● Formally an optional unsupported plugin o This was a hidden “power tool” for great maps o Create symbols using WKT directly inside your SLD <WellKnownName> wkt://LINESTRING(0.0 0.25, 0.25 0.25, 0.5 0.75, 0.75 0.25, 1.00 0.25) </WellKnownName> ● Merge with gt-render o Available by default for everyone o Go out and make maps!
  78. 78. windbarbs GeoSolutions Simone 12.0
  79. 79. New mark factory for windbarbs ● Use WMO compliant wind barbs ● Well known name template:windbarbs://$(value)[m/s]?emisph ere=(n/s) ● E.g.: windbarbs://25[m/s]?emisphere=n
  80. 80. ResourceStore Weaning GeoServer off Disk Boundless Jody Garnett 2.6.0
  81. 81. ResourceStore (Internal) ● GeoServer DataDirectory o Default is “in the web apps folder” o Copy to separate location when upgrading o Can be split up several directories ← why? ● Optional to Store in JDBC Database o “Catalog” configuration stored in Database o “Data Directory” still used for SLD, Icons, Fonts ● ResourceStore is “Step 1” of fixing this o Use “Resource” rather than “File”  Direct access to JDBC BLOB  “check out” file from the Database (if needed) Boundless Jody Garnett 2.6.0
  82. 82. New GeoServer Website One stop shop for everything GeoServer Boundless Justin Deolivera Jody Garnett 2.6.0
  83. 83. GeoServer: Home Boundless Justin Deolivera Jody Garnett 2.6.0
  84. 84. WebSite: Downloads Boundless Justin Deolivera Jody Garnett 2.6.0
  85. 85. WebSite: GitHub Wiki Boundless Justin Deolivera Jody Garnett 2.6.0
  86. 86. DataStore Tutorial Thanks FOSS4G 2014 Boundless Jody Garnett 12.0-RC1
  87. 87. ContentDataStore Boundless Jody Garnett 12.0-RC1
  88. 88. Roadmap Scheduled Features for 2014
  89. 89. SOLR data store ● Connect, display, filter data stored in your SOLR server ● Expose all of them to the web via WMS/WFS/WPS NSW-GS-AU GeoSolutions 2.7.x/2.6.x
  90. 90. Clustering asynch WPS ● The only OGC operation maintaining state ● Improve GS so that the state gets shared across cluster nodes Load balancer WPS1 WPS2 WPS3 Shared async state GeoSolutions GeoSolutions 2.7.x
  91. 91. Clustering GS config ● Instantly propagate config changes among GS nodes Load balancer GeoSolutions Boundless 2.7.x GeoSolutions Boundless GS1 GS2 GS3 Message passing bus
  92. 92. GeoFence community module ● Advanced security rules in GeoServer ● Separate web app for config, GS plugin to drive security ● Fully open source, already in use by some GS based projects (GeoOrchestra) GeoServer GeoFenc e plugin GeoFence UI and REST GeoSolutions GeoSolutions 2.6.x
  93. 93. CSS module for GeoTools ● Port of the GeoScript Scala version to Java ● Allow more people to contribute ● Faster translation to SLD ● Adding LessCSS features down the road /* @title Population < 2M */[PERSONS < 2000000] { fill: #4DFF4D; fill-opacity: 0.7; }/* @title 2M < Population < 4M */[PERSONS > 2000000] [PERSONS < 4000000] { fill: #FF4D4D; fill-opacity: 0.7; }/* @title Population > 4M */[PERSONS > 4000000] { fill: #4D4DFF; fill-opacity: 0.7; } Andrea Aime Andrea Aime 13.x
  94. 94. Wishlist Get involved - your support is needed!
  95. 95. Upgrade Wicket User Interface ● User interface is slowly falling apart o No longer works with latest browser (Current advice - use Firefox!) o Need to upgrade to newer version of “Wicket” ● Why? o User interface is not fun enough to attract volunteers o Popularity of REST API
  96. 96. Update CITE Tests ● CITE Tests are the OGC conformance tests ● Pass them and you get a cool sticker for your website o Well only if you pay o Idea: Ask OSGeo Board to talk to OGC? ● Tests keep us all safe! o We would like some time to upgrade / fix o Help out at the Code Sprint this weekend
  97. 97. GeoServer: WPS 2.0 Money/Love needed for: ● WPS 2.0 Operations o Improved Job Control (stop process?) ● WPS has CITE tests!
  98. 98. High DPI Mapping ● OL3 and GeoServer can already show wonderful images on High DPI displays ● To roll this concept out across the stack o GWC to Tile based on “dpi” format option o GeoTools rendering external graphics
  99. 99. GWC Cloud Storage ● Add options for cloud storage: o Amazon S3, Acumulo, Azure, Google, OpenShift ● Mad ideas: o Integrate S3 with AWS’s CDN Boundless Kevin Smith 1.7.x
  100. 100. GWC Storage Filter ● Processes tiles before storing them ● Specific implementations can: o Image Processing: color balance tiles o Re-encode: Changed PNG z-compression level o Change colour depth: PNG24 → PNG8 o Run external tools: PNGCrush etc ... Boundless Kevin Smith 1.7.x
  101. 101. Geometry + Geometry + Geography JTS Project has limited scope: ● GeoTools extends Geometry for Curves ● ISO 19107 Point/Curve/Surface ● Spatial4J looking at Geography ● GML Geometry wrappers around JTS Money/Love needed: Geometry needed for Open Source GIS industry
  102. 102. Questions? Thank you - for another amazing year!
  103. 103. Q: Software Downloads?
  104. 104. Q&A ● Q: When will be Java port of CSS Style be available? Interested in using it from MapFish.A: Andrea - It is a background activity need several more months on it. ● Q: Coudl not use as our State Plane defaults to meters and we measure in feet? I reported a bug last monthA: Feet? That is a terrible idea! Still hard to get answer this close to foss4g - will you be at the code sprint? Thanks for bug report.
  105. 105. Q&A ● Q: Would anyone here be interested in a GeoTools Cookbook?A: Please contact Ian Turton