GeoSolutions,
Taming OpenData and INSPIRE challenges with Open Source:
lessons learned and real-world use cases from an SME
Ing. Simone Giannecchini, GeoSolutions
Outline
 Who we are
 What we do
 What we have done
 My Perspective on INSPIRE and OpenData
GeoSolutions
 Founded in Italy in late 2006
 Expertise
• Image Processing, GeoSpatial Data Fusion
• Java, Java Enterprise, C++, Python
• JPEG2000, JPIP, Advanced 2D visualization
 Supporting/Developing FOSS4G projects
 GeoTools, GeoServer
 GeoBatch, GeoNetwork
 Clients
 Large NGOs, Public Administrations
 Private Companies
 Good balance between Italian and International Clients
 http://www.geo-solutions.it
Reference Architecture
GeoWebCacheGeoServer
GeoBatch
GeoNetwork
MapStore
Ingest & Preprocess
Metadata Maps & Data Tiles & KML
Mashups
Data & Styles WMS
Metadata
FormatsandProtocols
WFS
1.0, 1.1,
2.0
WMS
1.1.1
1.3.0
PostGIS
Oracle
H2
DB2
SQL Server
MySql
Spatialite
GeoCouch
Shapefile
----------
----------
---------
----------
----------
----------
---------
----------
----------
----------
---------
----------
ArcSDE
WFS
PNG, GIF
JPEG
TIFF,
GeoTIFF
SVG, PDF
KML/KMZ
Shapefile
GML2
GML3
GeoRSS
GeoJSON
CSV/XLS
Raw vector
data
Servers
Styled
maps
DBMS
Vector files
WCS
1.0,1.1.1
2.0.1
GeoTIFF
WMS
ArcGrid
GTopo30
Img+world
Mosaic
MrSID
JPEG 2000
ECW,Pyramid, Oracle GeoRaster, PostGis Raster
Raster files
Raw raster
data
GeoTIFF
ArcGrid
GTopo30
Img+World
GWC
(WMTS,
TMS,
WMS-C)
KML superoverlays
Google maps tiles
OGC tiles
OSGEO tiles
KML
WPS
1.0.0
GeoWebCache
 Tile Oriented Geospatial Cache
 Java Enterprise
 Maps Acceleration ( 10x to 100x )
 Standards compliant
 OGC WMS 1.1.1, WMTS 1.0
 TMS
 WMS-C
 Google Earth/Maps support
 Stand-Alone or Integrated in
GeoServer
GeoWebCache
Persistent raster/KML
tile cache
MapStore
 Create, Manage, Share Stunning Maps easily
 Mashing-up various sources of data
 Inject markers and tracks with custom
HTML pop-up
 Embed in existing sites and portal
 Use as complete geoportal solution
GeoNetwork
COMMON
LANGUAGES
EXCHANGE
FORMATS
CATALOGUES/
WEB SERVICES
• Metadata
– ISO19115:2003, ISO19119:2005, ISO19110:2005,
ISO19139:2007
– FGDC, Dublin Core
• Export/Import
– RSS & GeoRSS newsfeeds
– XML + XSL
• Catalogue interfaces
– OGC CSW 2.0.2 ISO Profile
– OAI-PMH
– Z39.50
– Geo-OpenSearch
– GeoRSS
– WebDAV
• Web Mapping Client
– OGC standards for WMS, WMC
• Web Map Server (through GeoServer)
– WMS, WFS, WFS-t, WCS, KML
GeoBatch
 Real-time produced data
 Remote Sensing, In situ, Models, etc..
 Recognizable, isolate data streams
 Flow
 Incoming series of geospatial data
 Specific format
 Defined rules for preprocessing
 Defined rules for ingestion and/or exploitation
 Automatic Ingestion and preprocessing  No manual intervention
 Example, flow of EO raster
Producer
GeoServer
GeoWebCache
Tile Pyramid Publish
INSPIRE – Who does What
 Discovery service  GeoNetwork
 View, Download, InvokeSD, Transformation
Service  GeoServer
INSPIRE Discovery Services
GeoNetwork
 GeoNetwork implements CSW 2.0.2 ISO Profile
 INPIRE extended constraints in metadata should
be explicitly enabled in GeoNetwork
 Complies with almost all main requirements in
Metadata Implementation and Discovery Services
 Main compliancy issues in:
 Handling federated catalogues
 in Discovery Service Metadata and Link Discovery Service
operations
 Federated metadata retrieving
 no CSW, only Z39.50
 Publishing in pull mode
 matching CSW Harvest operation in GN is not stable yet
INSPIRE View Services
GeoServer
 Historically well supported WMS 1.1.1
 GeoServer Stable 2.1.x Supports WMS 1.3
 Ordnance Survey (UK) funded the work
 INSPIRE compliancy still partial
 Harmonized Names (available since 2.2.0 beta)
 SOAP Support missing (recommended, not mandatory)
 Scenario 2 not supported
 Partial support for multilingualism
 INSPIRE Community Extension
 Separate plugin
 Plugs-in additional
fields for Multilingualism
and Service Metadata
INSPIRE View Services
GeoServer
 Language support:
 Single language declaration support
 Limitations
 No multilingual metadata on layers
 Missing localized support for exceptions
 Missing localization support for contents (e.g.,
GetFeatureInfo, GetMap labels)
 Robust support for Mandatory CRS (ETRS89)
 Support for Mandatory Encoding Formats (PNG,
GIF)
 Support for SLD 1.0 solid and mature
 Many Additional Vendor Options
INSPIRE View Services
GeoServer
 Support for SE 1.1 present
 extra operations available in SE 1.1 and not in
SLD 1.0 are not supported
 Some of the above are available as SLD 1.0
vendor extensions (same functionality, different
syntax)
INSPIRE View Services
GeoWebCache
 Full support for WMTS 1.0 (tiled view services)
 Harmonized names possible
 INSPIRECRS84QUAD tile matrix set not built-in,
but manually configurable
 Missing INSPIRE extensions:
 WMTS Capabilities document does not contain
the required INSPIRE extensions (WMS-C do
though)
 No layer metadata links
 Internationalization support missing
 No SOAP support
INSPIRE Download Services
GeoServer
 GeoServer 2.2.0 beta first release to support WFS
2.0 and GML 3.2.1
 IGN France Funding
 SOAP supported
 Missing some functionalities
 Standard Capabilites Extension missing
 Local and Remote Resolve
 GetPropertyValue Interactions with
AppSchema/Complex Features
 Basic Workflow for Complex features
 Store original datasets in ad hoc schema database
 Generating Object-Relational mappings
 On-the-fly output transformations
INSPIRE Coordinate
Transformation Services - GeoServer
 GeoServer provides coordinate transformation
tools with the gs:Reproject WPS process
 Requires some changes to become compliant
 Name change
 List supported SRS
 Use different mime types for GML
 Add “test transformation” mode (does not
actually transform, checks only if possible)
INSPIRE InvokeSD Services
GeoServer
 GeoServer Support OGC WPS 1.0.0
 Interaction with external WFS and WCS
 Automatic Ingestion of produced data
 Basic process chaining is supported
 Interaction with BPEL/BPMN engine to be
tested
Florence OpenGeoData Portal
Florence OpenGeoData Portal
Progetto CERCO
Progetto CERCO
LaMMa - MetOc
NURC - CMRE
Acque Wiz EU Project
 Crowdsourcing of data losses reports and data
quality
 Participative approach to water resourcing and
planning
Cultura Italia - MuseiD Portal
 Integrate with OpenCMS & SOLR
 Inject custom markers and tracks with custom
HTML pop-up
 Mobile Enabled Version
 Link-back to managed
resources
 Example here
 No INSPIRE!
Open Data
 Tremendous push for business and
innovation
 Bottom-up process
 Current Infrastructures shortcomings
 Focus on data rather than on services
 Mostly unidirectional
 Split between geospatial and non geospatial
 Interoperability between Hubs is an open
point
 There’s room for improvement
INSPIRE
 Top-down process
 Works by imposition
 Complexity, Complexity, Complexity
 We really needed more docs to read!
 Adapting to INSPIRE is expensive
 Existing products
 Existing infrastructure
 Existing data repositories
 Real world moves faster than
standardization bodies
 Open Data provides momentum, let’s use it!
Conclusions
 We work for people
 People ask for more data & services not more rules
 Complexity generate confusion
 Confusion generate failures
 Innovation
 development of new values
 solutions that meet new requirements
 inarticulate needs
 old market needs in new ways
 Open Data initiatives are fostering innovation
 INSPIRE does generate business, but innovation?
The End
Thanks for your time
simone.giannecchini@geo-solutions.it
GeoSolutions
 We don’t just use Open Source
 We contribute to creating real Open Source
products
 New Features
 Bug Fixing
 Contributions Review and Acceptance
 Community Management
 Documentation*
 Communication (conferences, events…)
 No investors, we sustain ourselves with our
revenue
Team – Key Members
 Ing. Simone Giannecchini
 Founder, GeoServer PSC, GeoTools PMC, ImageIO-Ext Architect, JAI-Tools founder,
GeoBatch founder
 Ing. Alessio Fabiani
 Founder, GeoServer PSC, GeoTools Committer
 Ing. Andrea Aime
 GeoServer PSC, GeoTools PMC, JAI-Tools Lead, ImageIO-Ext committer
 Ing. Daniele Romagnoli
 GeoServer Committer GeoTools PMC, ImageIO-Ext Lead, JAI-Tools Committer
 Ing. Emanuele Tajariol
 GeoServer Committer, GeoTools Committer, GeoNetwork PSC
 Ing Mauro Bartolomeoli
 GeoServer Committer, GeoTools Comitter, GeoBatch Committer
INSPIRE
 Infrastructure for Spatial Information in the
European Community
 2007/2/EC Directive, 14/03/2007
 Fully in place by 2019
 Implementing Rules and Technical Guidance to
implement EU-wide Interoperable SDIs
 IR are legally binding
 TC are not legally binding
 Main Aspects Covered
 Metadata
 Network Services
 Data and Service Sharing
 Data Specification
 Monitoring and Reporting
 Maximum Reuse of existing standards (OGC, ISO, W3C)
INSPIRE – Network Services
 Discovery Services
 View Services
 Download Services
 Transformation Services
 Invoke Spatial Data Services
 Rights Management Layer
INSPIRE – Discovery Services
 Discovery of spatial dataset and services
 Provide operations on services
 query about service capabilities
 register other federated Discovery Services
 Provide operations on metadata
 search
 update (both push and pull modes),
 perform federated searches
 Based on Metadata specs from ISO
 TG requires CSW 2.0.2 ISO AP as starting point
INSPIRE – Download Services
 Account for copies of spatial dataset, or parts of
them to be downloaded and/ accessed directly
 Predefined Dataset (or part) VS Direct access
 Presence of Query Capabilities
 Different Discovery Metadata
 Based on OGC WFS 2.0 and ISO/DIS 19142, 19143
for Direct Access
 May allow WFS 1.1 in the future
 Support for European CRS is Mandatory (See
Annex I of directive)
INSPIRE – Download Services
 Support for GML 3.2.1 is Mandatory
 SOAP Encoding is suggested
 Multilingualism Support is key
 Support for the Temporal dimension is Mandatory
 Support for Vector Data is rather mature
 Support for Raster Data is unclear
 WCS 1.1.2? WCS 2.0?
INSPIRE – View Services
 Display, Navigate, zoom in/out, pan or overlay
viewable dataset and legend information
 Based on OGC WMS 1.3.0 – ISO 19128 with OGC
SLD and SE
 INSPIRE Profile
 WMTS 1.0.0 for tile services with INSPIRE
extensions
 WMS 1.1.1 (INSPIRE PROFILE) is also acceptable
 2 Available Scenarios
 Scenario 1: GetCapabilities document is extended with a link to a
Catalogue
 Scenario 2: Extra elements are embedded into the GetCapabilities
document itself
INSPIRE – View Services
 Support for European CRS is Mandatory (See
Annex I of directive)
 SOAP support is suggested
 Support for the Temporal data dimension
 Mandatory output formats for Maps are PNG and
GID
 Multilingualism Support is key
INSPIRE – InvokeSD Services
 Invoking individual spatial services as well as
combinations of them
 synchronously or asynchronously
 Middleware per processing
 Technical Papers recommend OGC WPS as basic
building block
 BPEL is recommended between the others for
Service Chaining and Workflow
 SOAP should be mandatory
 IR and TG not available yet

Taming OpenData and INSPIRE challenges with Open Source: lessons learned and real-world use cases from an SME

  • 1.
    GeoSolutions, Taming OpenData andINSPIRE challenges with Open Source: lessons learned and real-world use cases from an SME Ing. Simone Giannecchini, GeoSolutions
  • 2.
    Outline  Who weare  What we do  What we have done  My Perspective on INSPIRE and OpenData
  • 3.
    GeoSolutions  Founded inItaly in late 2006  Expertise • Image Processing, GeoSpatial Data Fusion • Java, Java Enterprise, C++, Python • JPEG2000, JPIP, Advanced 2D visualization  Supporting/Developing FOSS4G projects  GeoTools, GeoServer  GeoBatch, GeoNetwork  Clients  Large NGOs, Public Administrations  Private Companies  Good balance between Italian and International Clients  http://www.geo-solutions.it
  • 4.
    Reference Architecture GeoWebCacheGeoServer GeoBatch GeoNetwork MapStore Ingest &Preprocess Metadata Maps & Data Tiles & KML Mashups Data & Styles WMS Metadata
  • 5.
    FormatsandProtocols WFS 1.0, 1.1, 2.0 WMS 1.1.1 1.3.0 PostGIS Oracle H2 DB2 SQL Server MySql Spatialite GeoCouch Shapefile ---------- ---------- --------- ---------- ---------- ---------- --------- ---------- ---------- ---------- --------- ---------- ArcSDE WFS PNG,GIF JPEG TIFF, GeoTIFF SVG, PDF KML/KMZ Shapefile GML2 GML3 GeoRSS GeoJSON CSV/XLS Raw vector data Servers Styled maps DBMS Vector files WCS 1.0,1.1.1 2.0.1 GeoTIFF WMS ArcGrid GTopo30 Img+world Mosaic MrSID JPEG 2000 ECW,Pyramid, Oracle GeoRaster, PostGis Raster Raster files Raw raster data GeoTIFF ArcGrid GTopo30 Img+World GWC (WMTS, TMS, WMS-C) KML superoverlays Google maps tiles OGC tiles OSGEO tiles KML WPS 1.0.0
  • 6.
    GeoWebCache  Tile OrientedGeospatial Cache  Java Enterprise  Maps Acceleration ( 10x to 100x )  Standards compliant  OGC WMS 1.1.1, WMTS 1.0  TMS  WMS-C  Google Earth/Maps support  Stand-Alone or Integrated in GeoServer GeoWebCache Persistent raster/KML tile cache
  • 7.
    MapStore  Create, Manage,Share Stunning Maps easily  Mashing-up various sources of data  Inject markers and tracks with custom HTML pop-up  Embed in existing sites and portal  Use as complete geoportal solution
  • 8.
    GeoNetwork COMMON LANGUAGES EXCHANGE FORMATS CATALOGUES/ WEB SERVICES • Metadata –ISO19115:2003, ISO19119:2005, ISO19110:2005, ISO19139:2007 – FGDC, Dublin Core • Export/Import – RSS & GeoRSS newsfeeds – XML + XSL • Catalogue interfaces – OGC CSW 2.0.2 ISO Profile – OAI-PMH – Z39.50 – Geo-OpenSearch – GeoRSS – WebDAV • Web Mapping Client – OGC standards for WMS, WMC • Web Map Server (through GeoServer) – WMS, WFS, WFS-t, WCS, KML
  • 9.
    GeoBatch  Real-time produceddata  Remote Sensing, In situ, Models, etc..  Recognizable, isolate data streams  Flow  Incoming series of geospatial data  Specific format  Defined rules for preprocessing  Defined rules for ingestion and/or exploitation  Automatic Ingestion and preprocessing  No manual intervention  Example, flow of EO raster Producer GeoServer GeoWebCache Tile Pyramid Publish
  • 10.
    INSPIRE – Whodoes What  Discovery service  GeoNetwork  View, Download, InvokeSD, Transformation Service  GeoServer
  • 11.
    INSPIRE Discovery Services GeoNetwork GeoNetwork implements CSW 2.0.2 ISO Profile  INPIRE extended constraints in metadata should be explicitly enabled in GeoNetwork  Complies with almost all main requirements in Metadata Implementation and Discovery Services  Main compliancy issues in:  Handling federated catalogues  in Discovery Service Metadata and Link Discovery Service operations  Federated metadata retrieving  no CSW, only Z39.50  Publishing in pull mode  matching CSW Harvest operation in GN is not stable yet
  • 12.
    INSPIRE View Services GeoServer Historically well supported WMS 1.1.1  GeoServer Stable 2.1.x Supports WMS 1.3  Ordnance Survey (UK) funded the work  INSPIRE compliancy still partial  Harmonized Names (available since 2.2.0 beta)  SOAP Support missing (recommended, not mandatory)  Scenario 2 not supported  Partial support for multilingualism  INSPIRE Community Extension  Separate plugin  Plugs-in additional fields for Multilingualism and Service Metadata
  • 13.
    INSPIRE View Services GeoServer Language support:  Single language declaration support  Limitations  No multilingual metadata on layers  Missing localized support for exceptions  Missing localization support for contents (e.g., GetFeatureInfo, GetMap labels)  Robust support for Mandatory CRS (ETRS89)  Support for Mandatory Encoding Formats (PNG, GIF)  Support for SLD 1.0 solid and mature  Many Additional Vendor Options
  • 14.
    INSPIRE View Services GeoServer Support for SE 1.1 present  extra operations available in SE 1.1 and not in SLD 1.0 are not supported  Some of the above are available as SLD 1.0 vendor extensions (same functionality, different syntax)
  • 15.
    INSPIRE View Services GeoWebCache Full support for WMTS 1.0 (tiled view services)  Harmonized names possible  INSPIRECRS84QUAD tile matrix set not built-in, but manually configurable  Missing INSPIRE extensions:  WMTS Capabilities document does not contain the required INSPIRE extensions (WMS-C do though)  No layer metadata links  Internationalization support missing  No SOAP support
  • 16.
    INSPIRE Download Services GeoServer GeoServer 2.2.0 beta first release to support WFS 2.0 and GML 3.2.1  IGN France Funding  SOAP supported  Missing some functionalities  Standard Capabilites Extension missing  Local and Remote Resolve  GetPropertyValue Interactions with AppSchema/Complex Features  Basic Workflow for Complex features  Store original datasets in ad hoc schema database  Generating Object-Relational mappings  On-the-fly output transformations
  • 17.
    INSPIRE Coordinate Transformation Services- GeoServer  GeoServer provides coordinate transformation tools with the gs:Reproject WPS process  Requires some changes to become compliant  Name change  List supported SRS  Use different mime types for GML  Add “test transformation” mode (does not actually transform, checks only if possible)
  • 18.
    INSPIRE InvokeSD Services GeoServer GeoServer Support OGC WPS 1.0.0  Interaction with external WFS and WCS  Automatic Ingestion of produced data  Basic process chaining is supported  Interaction with BPEL/BPMN engine to be tested
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
    Acque Wiz EUProject  Crowdsourcing of data losses reports and data quality  Participative approach to water resourcing and planning
  • 26.
    Cultura Italia -MuseiD Portal  Integrate with OpenCMS & SOLR  Inject custom markers and tracks with custom HTML pop-up  Mobile Enabled Version  Link-back to managed resources  Example here  No INSPIRE!
  • 27.
    Open Data  Tremendouspush for business and innovation  Bottom-up process  Current Infrastructures shortcomings  Focus on data rather than on services  Mostly unidirectional  Split between geospatial and non geospatial  Interoperability between Hubs is an open point  There’s room for improvement
  • 28.
    INSPIRE  Top-down process Works by imposition  Complexity, Complexity, Complexity  We really needed more docs to read!  Adapting to INSPIRE is expensive  Existing products  Existing infrastructure  Existing data repositories  Real world moves faster than standardization bodies  Open Data provides momentum, let’s use it!
  • 29.
    Conclusions  We workfor people  People ask for more data & services not more rules  Complexity generate confusion  Confusion generate failures  Innovation  development of new values  solutions that meet new requirements  inarticulate needs  old market needs in new ways  Open Data initiatives are fostering innovation  INSPIRE does generate business, but innovation?
  • 30.
    The End Thanks foryour time simone.giannecchini@geo-solutions.it
  • 31.
    GeoSolutions  We don’tjust use Open Source  We contribute to creating real Open Source products  New Features  Bug Fixing  Contributions Review and Acceptance  Community Management  Documentation*  Communication (conferences, events…)  No investors, we sustain ourselves with our revenue
  • 32.
    Team – KeyMembers  Ing. Simone Giannecchini  Founder, GeoServer PSC, GeoTools PMC, ImageIO-Ext Architect, JAI-Tools founder, GeoBatch founder  Ing. Alessio Fabiani  Founder, GeoServer PSC, GeoTools Committer  Ing. Andrea Aime  GeoServer PSC, GeoTools PMC, JAI-Tools Lead, ImageIO-Ext committer  Ing. Daniele Romagnoli  GeoServer Committer GeoTools PMC, ImageIO-Ext Lead, JAI-Tools Committer  Ing. Emanuele Tajariol  GeoServer Committer, GeoTools Committer, GeoNetwork PSC  Ing Mauro Bartolomeoli  GeoServer Committer, GeoTools Comitter, GeoBatch Committer
  • 33.
    INSPIRE  Infrastructure forSpatial Information in the European Community  2007/2/EC Directive, 14/03/2007  Fully in place by 2019  Implementing Rules and Technical Guidance to implement EU-wide Interoperable SDIs  IR are legally binding  TC are not legally binding  Main Aspects Covered  Metadata  Network Services  Data and Service Sharing  Data Specification  Monitoring and Reporting  Maximum Reuse of existing standards (OGC, ISO, W3C)
  • 34.
    INSPIRE – NetworkServices  Discovery Services  View Services  Download Services  Transformation Services  Invoke Spatial Data Services  Rights Management Layer
  • 35.
    INSPIRE – DiscoveryServices  Discovery of spatial dataset and services  Provide operations on services  query about service capabilities  register other federated Discovery Services  Provide operations on metadata  search  update (both push and pull modes),  perform federated searches  Based on Metadata specs from ISO  TG requires CSW 2.0.2 ISO AP as starting point
  • 36.
    INSPIRE – DownloadServices  Account for copies of spatial dataset, or parts of them to be downloaded and/ accessed directly  Predefined Dataset (or part) VS Direct access  Presence of Query Capabilities  Different Discovery Metadata  Based on OGC WFS 2.0 and ISO/DIS 19142, 19143 for Direct Access  May allow WFS 1.1 in the future  Support for European CRS is Mandatory (See Annex I of directive)
  • 37.
    INSPIRE – DownloadServices  Support for GML 3.2.1 is Mandatory  SOAP Encoding is suggested  Multilingualism Support is key  Support for the Temporal dimension is Mandatory  Support for Vector Data is rather mature  Support for Raster Data is unclear  WCS 1.1.2? WCS 2.0?
  • 38.
    INSPIRE – ViewServices  Display, Navigate, zoom in/out, pan or overlay viewable dataset and legend information  Based on OGC WMS 1.3.0 – ISO 19128 with OGC SLD and SE  INSPIRE Profile  WMTS 1.0.0 for tile services with INSPIRE extensions  WMS 1.1.1 (INSPIRE PROFILE) is also acceptable  2 Available Scenarios  Scenario 1: GetCapabilities document is extended with a link to a Catalogue  Scenario 2: Extra elements are embedded into the GetCapabilities document itself
  • 39.
    INSPIRE – ViewServices  Support for European CRS is Mandatory (See Annex I of directive)  SOAP support is suggested  Support for the Temporal data dimension  Mandatory output formats for Maps are PNG and GID  Multilingualism Support is key
  • 40.
    INSPIRE – InvokeSDServices  Invoking individual spatial services as well as combinations of them  synchronously or asynchronously  Middleware per processing  Technical Papers recommend OGC WPS as basic building block  BPEL is recommended between the others for Service Chaining and Workflow  SOAP should be mandatory  IR and TG not available yet