This document summarizes a webinar about implementing INSPIRE in Luxembourg using FME. The webinar agenda covered harmonization principles for INSPIRE spatial data infrastructures, development stages using FME examples, and con terra's INSPIRE solution pack for FME. It also discussed Luxembourg's INSPIRE solutions developed by Jeff Konnen and how FME tools support INSPIRE. Additional topics were INSPIRE trends and consuming INSPIRE data by reading and building value-added services.
Why Teams call analytics are critical to your entire business
Meeting the INSPIRE Mandate with FME
1. FME INSPIRE Webinar:
Implementation in Luxembourg
Jeff Konnen, Luxembourg National Geoportal
Don Murray, Safe Software
Dean Hintz, Safe Software
July 2012
2. Agenda
Harmonization principles for INSPIRE SDIs
Development stages and FME examples:
assembly, transformation, validation and
publication
con terra’s INSPIRE Solution Pack for FME
Solutions for Luxembourg by Jeff Konnen
FME tools to support INSPIRE
INSPIRE trends
Consuming INSPIRE
Reading
Building value added services
3. INSPIRE - Goals
Assist EU environmental management
Extend Member States’ SDIs using:
Common data model
Open standards
INSPIRE SDI should:
Combine spatial data from different sources
Share spatial data between public authorities
4. INSPIRE - Challenge
You want to meet INSPIRE data provision
requirements, but your data is organized rather
differently
?
8. Harmonization Core Concepts
Harmonization: implied INSPIRE requirement.
Disparate sources must be mapped to a common
destination data model to support integration
Core to the harmonization workflow is a process
called schema mapping.
Delivered by services based on open standards
9. INSPIRE Harmonization Principles
Typical stages:
1. Evaluation
2. Assembly
3. Transformation
4. Validation
5. Publication
Based on the Spatial ETL concept (Extract,
Transform and Load), as applied to INSPIRE SDI’s
10. Evaluation
Assess destination schema and data
requirements
Assess source datasets and schema
Consider fundamental differences in
representation, resolution
Closely inspect actual representative datasets
11. Data Assembly
Assess the diversity of source data types:
vector, raster, CAD, GIS, database, text, XML, web,
3D, sensor and non-spatial
Review format and semantic translation needs
Decide how to perform necessary joins
ID joins, spatial relates, nearest neighbor, one to
many relationships
Goal is to build a data structure that corresponds
with INSPIRE requirements
12. Translation
FME supports reading from a wide array of data
formats and types
275+ formats, with more added each year
CAD, GIS, XML, raster, database, non-spatial, 3D
13. Lithuania Geographic Information
Infrastructure - Highlights
Wide range of format support required by member agencies
Solution combines FME, ArcGIS and con terra tools
Conversion rules key to harmonization process
Centralized workflow and common data model
simplified source to destination mapping
FME Server supports sharing for LGII partners, business,
education and research institutions, NGOs
LGII provides real value with broader access to
seamless data via a common INSPIRE schema
15. LGII Geoportal Central
System Components
FME Server
Data conversion for download
Transformation and schema mapping between data schemes,
data models and formats
con terra’s SDI Suite
Content and user management
E-Commerce and accounting
Reporting
ArcGIS Server
WxS services (WMS, WFS, WCS)
Catalog services (CAT)
Metadata harvesting (CSW, WAF)
Spatial data editing, redlining functionality
16. Data Transformation - Schema
Reshape source data to match required
destination schema
Schema mapping
feature type
attribute name
new attribute creation
code lists
conditional value
mappings
17. Schema Mapping in FME
Feature Type Mapping in FME Workbench
Attribute Mapping in FME Workbench
18. FME Data Model Restructuring:
Attribute Names & Values
Value Mapping
20. INSPIRE Pilot:
Swedish Protected Sites
Swedish EPA project with Metria of Sweden
Read from 3 different data sources (pilot):
Swedish Protected Areas
Helsinki data commission (Helcom)
European Natura2000 habitats
Perform required joins, generate IDs
Transform to INSPIRE schema
Load INSPIRE-like staging database (PostGIS)
Publish INSPIRE web services based on staging
database
22. Results in Staging Database
* Data provided by Lantmäteriet, the Swedish mapping, cadastral and land registration authority www.lantmateriet.se , and Metria, Sweden www.metria.se.
23. Protected Sites Schema
Mapping: NVR to INSPIRE
Filter
Attribute Destination
FilterAttribute Value SourceAttributeField_Inspire AttributeValue DestinationAttribute
INSPIRE_APPLICATIONS
Full CHEMA
DID INSPIRE_LOCALID
SE INSPIRE_NAMESPACE
VALID_FROM_DA INSPIRE_
TE LEGALFOUNDATION
DATE
DECISIONDATE INSPIRE_DATE
INSPIRE_DATETYPE
creation
IUNC_CATEGORY Empty value INSPIRE_DESIGNATION
0
IUNC_CATEGORY INSPIRE_DESIGNATION
Ia strictNatureReserve
IUNC_CATEGORY INSPIRE_DESIGNATION
Ib wildernessArea
IUNC_CATEGORY INSPIRE_DESIGNATION
II nationalPark
IUNC_CATEGORY INSPIRE_DESIGNATION
III naturalMonument
IUNC_CATEGORY null Empty value INSPIRE_DESIGNATION
OBJECTNAME INSPIRE_SITENAME
PROTECTIONTYPE 1 natureConservation INSPIRE_PROTECTIONC
LASSIFICATION
PROTECTIONTYPE 2 natureConservation INSPIRE_PROTECTIONC
LASSIFICATION
24. Transformation: Geometry
Non-spatial to spatial
Geometry extraction (spatial to GML)
Representation transform: CAD drawing lines with
labels to GIS polygonal features with attributes
Coordinate System Reprojection (ED50 to ETRF89)
Simple to complex geometry
Source point and polygon data to multiple geometric
representations (city as point / area, river as line / area)
Generalization and interpolation
Highly granular national and regional datasets often
require thinning to be usable on pan-European scales
25. Validation
INSPIRE schema validation (xsds)
Data integrity
Unique IDs
Geometric integrity (closed polygons)
Null values (nullable?)
Valid values: ranges and domain codes
Data gaps
Bounds
Network integrity
Custom validity rules specific to domain
Validation automation via FME Server upload
Ensure data quality throughout the data transformation
process.
26. Swedish Transportation Administration:
Validation
System supports propagation of municipal and
regional road data to national dataset
Data model transformation and QC to translate
2.5 million road links into NVDB
Transform between NVDB and INSPIRE
Workflows automated by FME and FME Server
Validation key to support upload services
27. Publication
Produce INSPIRE compliant GML
Provide discovery, view or download services, for
WxS, GML and other desired formats
Publish with FME Server or integrate with your
geo web server of choice:
ArcServer (ArcGIS for INSPIRE)
Deegree
GeoServer
MapServer
Spatial Data Services
28. Publication
How to make data most accessible?
Support for mandated open standards and OGC
services
Support for defacto industry standards,
proprietary and legacy agency systems
Possibilities for Invoke Spatial Data Services as a
bridge
accepts user request
queries required INSPIRE OGC services
provides desired web service (KML, PDF)
29. Swedish Transportation Administration:
Invoked Services
Supports estimation of new road costs
related to archeological and protected sites
Site potential using terrain model and
historic coast lines
WFS Sources:
The Swedish Transport Administration roads
Swedish EPA Protected sites
Swedish National Heritage Board
Archaeological findings
Result: PDF showing site potential
30. FME Tools for INSPIRE
Format translation
Schema mapping
String and list manipulation
Data validation
Database load and extract
XML,GML,WFS: reading, validation, publication
Web services: WFS, WMS, integration with others
Metadata support
Enterprise services
with FME Server
34. Other INSPIRE Examples
Finnish National Land Survey
GML to Shape conversion
Data upload and validation
WPS test pilot – FME Server / Deegree integration
Technical University Of Munich
XMI mapping of UML schema transformation rules
Maps between AAA and INSPIRE data models
Uses FME to perform schema mapping based on XMI
Nature SDI: Datasiel, Liguria Region, Italy
FME data harmonization -> loads staging database
Generates protected sites GML for WFS
Croatian NSDI: Geoportal
By State Geodetic Administration (SGA)
Central register of base map, ortho, spatial units, cadastre
FME used primarily in data migration and updates
35. Example INSPIRE Solutions
by FME Partners
INSPIRE Solution Pack from con terra (> 20
implementations across Europe)
Metria, Sweden (Protected Areas Pilot; Swedish
Department of Transportation)
HNIT Lithuania (LGII)
Technical University of Munich (multiple
INSPIRE projects)
AED Sicad (NAS to INSPIRE conversions)
Spatialworld, Finland (National Land Survey)
36. con terra – FME INSPIRE
Solution Pack Highlights
Simplify data transformation and schema mapping for
ArcGIS for INSPIRE
Predefined FME Workbench templates help jump
start the data migration and Harmonization process
Connect your existing databases to the standardized ESRI
INSPIRE Geodatabase
Automate data load and update processes
Quality assurance and metadata
Publish once and use many times
37. con terra – FME INSPIRE Solution Pack:
3 Key Tasks
Consumers
transformation & integration
Internal information/ INSPIRE fusionCenter –
storage systems Data models and web services
European SDI
2. Integration into 1. Data Models for spatial 3. Access via INSPIRE network
business processes and data sets compliant to services meeting the Quality-
transformation of data to INSPIRE data specifications of-Service requirements (COTS
INSPIRE (COTS product) product)
38. con terra - FME INSPIRE Solution Pack:
Schema Mapping
ETL Workflow description
import create
destination
import
destination
schema Load Data
source data mapping INSPIRE GDB
schema schema
FME data import process
FME INSPIRE
Solution Pack
automated
39. Components of the FME
INSPIRE Solution Pack
Additional INSPIRE information
Tutorial workspace (complete sample mapping [AdminUnits])
Additional HTML Workbench Help (description of INSPIRE GDB data model)
Direct access to the INSPIRE data specification (link to specific themes)
Additional functionality
Template workspaces (destination schema and predefined workspaces)
Destination data schema for all Annex 1 themes (INSPIRE GDB of ArcGIS for
INSPIRE http://resources.arcgis.com/de/content//arcgis-inspire/1.0/about )
More than 100 additional INSPIRE transformers
INSPIRE specific value and attribute mapping (voidable values)
transformer for recurrent tasks (ID management, lifespan setter)
40. Solution Pack
Template Workspaces
one template Workspace per Annex I theme
predefined datasets (INSPIRE GDB)
predefined attribute-creation and settings
predefined attribute-mapping
integrated description for mapping file
completion
41. INSPIRE Transformers
(Attribute and Value Mapping)
Automated filling of obligated attribute
legalStatus
legalStatus_void
Possible values for obligated attributes (if not void)
agreed
notAgreed
Predefined void value reasons (if void)
0 = no reason given
1 = reason: unknown
2 = reason: unpopulated
AttributeRenamer functionality
usable if attribute already
existing
42. FME – the pillar of an
INSPIRE Infrastructure
Jeff Konnen
Administration du Cadastre et de la Topographie
Service Géoportail
Grand-Duché de Luxembourg
43. Or ...
Thanks to FME, the red lion
is juggling with INSPIRE
44. About the ACT
ACT : Administration du Cadastre et de la
Topographie
Administration depending on the Ministry of Finance
130 people
Responsible for:
Cadastral documentation
Administrative boundaries and frontiers
Delimitation of properties
Management of property-related and topographic databases
Management of the national address register
Topographic documentation / maps
Management of geodetic networks
Management and hosting of the national Geoportal
50. map.geoportal.lu
About 130 different layers
Very fast (everything is tiled and precalculated)
Easy to use (Google Style)
Based on Mapfish Technology
Single Search Field searching in 1Mio different
entries (Addresses, Parcels, Localities, Rivers etc)
Stats: 1300+ single users per day, tendency
growing
API
51. About me
Jeff Konnen
Project Leader at the ACT / Geoportal
Long time FME User
FME Professional & Trainer
jeff.konnen@act.etat.lu
52. Geoportal vs INSPIRE
ACT and especially the Geoportal crew have been
designated as INSPIRE Point of Contact (NPOC)
for Luxembourg
The Geoportal crew has to manage metadata and
Network Services for the Luxembourgish Node of
INSPIRE
Since May 2010, Luxembourg is reporting and
monitoring about its Network Services and since
November 2011, the Discovery and View Services
for Annex I & II Themes are available online
(http://inspire.geoportal.lu/geoportal)
Since June 2012, a first set of Download services
is available (IOC)
54. Geoportal vs INSPIRE
The Geoportal crew of Luxembourg consists of 5
people
1 Manager
1 Project Leader
2 Developers
1 IT Manager
There are not a lot ressources for INSPIRE
A complete solution for serving INSPIRE relevant data
had to be found
ArcGIS for INSPIRE
55. ArcGIS for INSPIRE
INSPIRE Datamodel for the Geodatabase
We didn’t have to implement the INSPIRE datamodels by
ourselves
INSPIRE Geoportal
Management of the INSPIRE Metadata and the INSPIRE
Discovery Service
Caveat: Our main Metadata management is multilingual (4
languages). By using INSPIRE Geoportal we decided to go English
only for the INSPIRE relevant metadata
INSPIRE Add-on for ArcGIS Desktop
Create maps for INSPIRE based on the INSPIRE datamodel
INSPIRE Add-on for ArcGIS Server
Creates INSPIRE View Services and Download Services with
one click
57. ArcGIS for INSPIRE ..
What ArcGIS for INSPIRE does not do:
Convert your data into the INSPIRE Datamodel
That’s where we need FME
FME is the tool used to convert data from the
production databases into the INSPIRE datamodel
The con terra ISP (INSPIRE Solution Pack) helps to
understand the technical aspects of the mapping that
has to be done and to streamline your efforts
The semantical mapping of the data has to be
done by the thematic experts, no solution will
do it for you
58. FME and ISP
Using FME and ISP, a set of scripts has been
created
These scripts can be executed regularly to fill the
INSPIRE Datamodel
The INSPIRE View Services are based on the
content of the INSPIRE Datamodel
Loading takes about 24 hours
Executed once a year or more often
Themes:
Annex I: AU,AD,CP,GN,TN,(PS),HY as INSPIRE View
Services
Annex II: OI,EL,LC,GE as WMS 1.3.0 as A4I not
ready yet
60. FME and INSPIRE : Lessons
learned
FME is the right tool to transform your data
ISP helps you to streamline your efforts
The complicated part is not FME,
it’s the semantic mapping (!)
Plan a lot of time for this part
A lot of discussions / compromises to find
61. FME and INSPIRE : Next steps
Activate some more download services for Annex
I (and II) until the end of the year
Only « one click » but a lot of questions to clarify
(pricing)
Do the mapping for Annexes II and III
Wait for the datamodels to be final
Activate their view and download services
Permanently : Keep up with new data
63. Uses of FME Server: Shop
Shop: Automatic Extraction of orders
64. Uses of FME Server: Shop
(Orchestration with Syncserv)
65. Other Uses of FME
Weekly syncronization of production data in the
geoportal databases
STAR (CAD) -> ORACLE
ORACLE -> PostGIS
Daily conversion of measurement files (STAR) to
DWG
Quality control
QC of Administrative Unit data
Statistics
Count features issued by the different WFS services
66. Other uses of FME (Server)..
Online coordinate conversion (CSV File
conversion from WGS84 -> LUREF (EPSG:2169)
using FME Server
67. Other uses of FME (Server)..
Preview of DWG in files in the shop before
ordering
Streaming service doing a generic transformation
from DWG to PNG
68. Other uses of FME ..
Weekly generation of an address point database
based on the centerpoints of the biggest building
inside a parcel
69. Other uses of FME ..
Weekly transformation of heterogeneous data
stored in production tables into one
homegeneous search table for the Full-Text-
Search used in the geoportal
70. Other uses of FME ..
One – shot analysis and export jobs
Mosaicking and Production of ECW – Files for
background rasters
72. About FME and its use
The general credo of the geoportal is to use a
maximum of FOSS components:
PostGIS
Mapserver
Mapfish (OpenLayers, ExtJS, GeoExt)
Exceptions:
Use of ArcGIS:
Complex symbologies
Data delivered by FGDB / MXD
ArcGIS for INSPIRE
Use of FME
FME is the only tool which allows us to really
MANIPULATE geodata, there is no alternative
74. INSPIRE Trends
Integration between proprietary and open
standards – especially as deployment moves to
regional and local agencies
Increased focus on consuming INSPIRE services
and GML
More opportunities for spatial data and invoke
services to bridge this gap
Examples: machine to machine communication to
provide seamless support for KML and PDF
streaming
Success = User isn’t aware they are
accessing INSPIRE
76. Now What?
“How is anyone going to use INSPIRE?”
FME allows you to consume INSPIRE compliant
GML as well as create it.
Why not build value on the web services every
one else is investing in?
Extract and flatten complex INSPIRE structures
for use within your GIS applications
Add spatial data services to transform INSPIRE into
data streams suitable for widely accessible client
applications (PDF, KML, GeoJSON)
77. XML vs. Relational
Relational
attribute1 attribute2 Active Active
Date_From Date_To
John Vancouver 11-22-99 12-11-09
June Surrey 02-25-05 9-15-10
82. FME Tools for INSPIRE
Format translation
Schema mapping
String and list manipulation
Data validation
Database load and extract
XML,GML,WFS: reading, validation, publication
Web services: WFS, WMS, integration with others
Metadata support
Enterprise services
with FME Server
83. FME Tools for INSPIRE XML
GML reading and writing (3.2.1 and earlier)
XML Reading – read virtually any XML and extract
features
XML Processing – validate, format, and update XML
Geometry Extraction (features > GML objects)
XML Writing - Write XML or GML no matter how
complex
XMLTemplater
FME merges attribute values into a template structure
Template per feature type and dataset
XML Validation – validated to ensure compliance with
INSPIRE schemas
85. FME Tools for INSPIRE
FME provides a full range of tools for INSPIRE, our
partners provide complete solutions
Data modeling interface makes it easier to
handle schema mapping - no coding required.
Automated workflows save time in creating and
maintaining transformation processes.
Self-documenting workspaces ease collaboration
Easily integrates with INSPIRE-compliant
services:
Esri ArcGIS Server
Deegree, MapServer and GeoServer
Scalability and performance support quality of
service requirements
86. Summary: Solution Approaches
Typical stages for SDI projects: Evaluation, assembly,
transformation, validation and publication
FME provides tools that empower solutions for each
step of the INSPIRE harmonization process
Our partners provide a range of solutions
Conterra’s INSPIRE Solution Pack for FME
Luxembourg Implementation of by Jeff Konnen
INSPIRE trends
consuming INSPIRE data
need for spatial data services to support common clients
FME provides tools to support INSPIRE, our partners
apply these to provide solutions
88. Thank You!
?
Questions?
For more information:
Jeff Konnen
jeff.konnen@act.etat.lu
Dean Hintz
dean.hintz@safe.com
www.safe.com/inspire
www.fmepedia.com (examples)