Meeting the INSPIRE Mandate with FME
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Meeting the INSPIRE Mandate with FME

on

  • 2,086 views

Discover how you too can solve common INSPIRE-related data challenges using FME. You’ll see how to easily perform schema mapping and meet INSPIRE’s precise data model requirements. You’ll also ...

Discover how you too can solve common INSPIRE-related data challenges using FME. You’ll see how to easily perform schema mapping and meet INSPIRE’s precise data model requirements. You’ll also learn about the various FME-based INSPIRE solutions available and how people are using them. Finally, the project leader of Luxembourg’s public Geoportal (Jeff Konnen) will share how they’re using FME to help reach their INSPIRE objectives.

Statistics

Views

Total Views
2,086
Views on SlideShare
2,056
Embed Views
30

Actions

Likes
0
Downloads
37
Comments
0

3 Embeds 30

http://www.safe.com 28
http://pike.safe.internal 1
http://safe.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Meeting the INSPIRE Mandate with FME Presentation Transcript

  • 1. FME INSPIRE Webinar:Implementation in LuxembourgJeff Konnen, Luxembourg National GeoportalDon Murray, Safe SoftwareDean Hintz, Safe Software July 2012
  • 2. Agenda Harmonization principles for INSPIRE SDIs Development stages and FME examples:  assembly, transformation, validation and publication con terra’s INSPIRE Solution Pack for FME Solutions for Luxembourg by Jeff Konnen FME tools to support INSPIRE INSPIRE trends Consuming INSPIRE  Reading  Building value added services
  • 3. INSPIRE - Goals Assist EU environmental management Extend Member States’ SDIs using:  Common data model  Open standards INSPIRE SDI should:  Combine spatial data from different sources  Share spatial data between public authorities
  • 4. INSPIRE - Challenge You want to meet INSPIRE data provision requirements, but your data is organized rather differently ?
  • 5. Poll QuestionWhat INSPIRE efforts have you worked on to date?
  • 6. What we do … ? We build data bridges
  • 7. Poll QuestionHow long have you been using FME?
  • 8. Harmonization Core Concepts Harmonization: implied INSPIRE requirement. Disparate sources must be mapped to a common destination data model to support integration Core to the harmonization workflow is a process called schema mapping. Delivered by services based on open standards
  • 9. INSPIRE Harmonization PrinciplesTypical stages:1. Evaluation2. Assembly3. Transformation4. Validation5. PublicationBased on the Spatial ETL concept (Extract,Transform and Load), as applied to INSPIRE SDI’s
  • 10. Evaluation Assess destination schema and data requirements Assess source datasets and schema Consider fundamental differences in representation, resolution Closely inspect actual representative datasets
  • 11. Data Assembly Assess the diversity of source data types:  vector, raster, CAD, GIS, database, text, XML, web, 3D, sensor and non-spatial Review format and semantic translation needs Decide how to perform necessary joins  ID joins, spatial relates, nearest neighbor, one to many relationshipsGoal is to build a data structure that correspondswith INSPIRE requirements
  • 12. TranslationFME supports reading from a wide array of data formats and types  275+ formats, with more added each year  CAD, GIS, XML, raster, database, non-spatial, 3D
  • 13. Lithuania Geographic InformationInfrastructure - HighlightsWide range of format support required by member agenciesSolution combines FME, ArcGIS and con terra tools Conversion rules key to harmonization process Centralized workflow and common data modelsimplified source to destination mapping FME Server supports sharing for LGII partners, business,education and research institutions, NGOs LGII provides real value with broader access toseamless data via a common INSPIRE schema
  • 14. LGII – Lithuania GeospatialPortal Translation SupportFile Formats: Coordinate systems:  AutoCAD DXF / DWG  LKS94  ESRI Personal Geodatabase  WGS84  Shape  ETRS89  MapInfo TAB  1963  Microstation Design V8  Pulkovo 1942  GML  UTM (34, 35 zones)  GIF / PNG  JPEG Other data sources:  TIFF  WFS  ERDAS IMG  SDE  ECW  Files
  • 15. LGII Geoportal CentralSystem Components FME Server  Data conversion for download  Transformation and schema mapping between data schemes, data models and formats con terra’s SDI Suite  Content and user management  E-Commerce and accounting  Reporting ArcGIS Server  WxS services (WMS, WFS, WCS)  Catalog services (CAT)  Metadata harvesting (CSW, WAF)  Spatial data editing, redlining functionality
  • 16. Data Transformation - Schema Reshape source data to match required destination schema Schema mapping  feature type  attribute name  new attribute creation  code lists  conditional value mappings
  • 17. Schema Mapping in FME Feature Type Mapping in FME Workbench Attribute Mapping in FME Workbench
  • 18. FME Data Model Restructuring:Attribute Names & Values  Value Mapping
  • 19. FME SchemaMapper:INSPIRE geographic names FME Workspace Name mapping Name & value mapping
  • 20. INSPIRE Pilot:Swedish Protected Sites Swedish EPA project with Metria of Sweden Read from 3 different data sources (pilot):  Swedish Protected Areas  Helsinki data commission (Helcom)  European Natura2000 habitats Perform required joins, generate IDs Transform to INSPIRE schema Load INSPIRE-like staging database (PostGIS) Publish INSPIRE web services based on staging database
  • 21. Helsinki CommissionSource Data
  • 22. Results in Staging Database* Data provided by Lantmäteriet, the Swedish mapping, cadastral and land registration authority www.lantmateriet.se , and Metria, Sweden www.metria.se.
  • 23. Protected Sites Schema Mapping: NVR to INSPIRE Filter Attribute DestinationFilterAttribute Value SourceAttributeField_Inspire AttributeValue DestinationAttribute INSPIRE_APPLICATIONS Full CHEMA DID INSPIRE_LOCALID SE INSPIRE_NAMESPACE VALID_FROM_DA INSPIRE_ TE LEGALFOUNDATION DATE DECISIONDATE INSPIRE_DATE INSPIRE_DATETYPE creationIUNC_CATEGORY Empty value INSPIRE_DESIGNATION 0IUNC_CATEGORY INSPIRE_DESIGNATION Ia strictNatureReserveIUNC_CATEGORY INSPIRE_DESIGNATION Ib wildernessAreaIUNC_CATEGORY INSPIRE_DESIGNATION II nationalParkIUNC_CATEGORY INSPIRE_DESIGNATION III naturalMonumentIUNC_CATEGORY null Empty value INSPIRE_DESIGNATION OBJECTNAME INSPIRE_SITENAMEPROTECTIONTYPE 1 natureConservation INSPIRE_PROTECTIONC LASSIFICATIONPROTECTIONTYPE 2 natureConservation INSPIRE_PROTECTIONC LASSIFICATION
  • 24. Transformation: Geometry Non-spatial to spatial Geometry extraction (spatial to GML) Representation transform: CAD drawing lines with labels to GIS polygonal features with attributes Coordinate System Reprojection (ED50 to ETRF89) Simple to complex geometry  Source point and polygon data to multiple geometric representations (city as point / area, river as line / area) Generalization and interpolation  Highly granular national and regional datasets often require thinning to be usable on pan-European scales
  • 25. Validation INSPIRE schema validation (xsds) Data integrity  Unique IDs  Geometric integrity (closed polygons)  Null values (nullable?)  Valid values: ranges and domain codes  Data gaps  Bounds  Network integrity Custom validity rules specific to domain Validation automation via FME Server uploadEnsure data quality throughout the data transformationprocess.
  • 26. Swedish Transportation Administration:Validation System supports propagation of municipal and regional road data to national dataset Data model transformation and QC to translate 2.5 million road links into NVDB Transform between NVDB and INSPIRE Workflows automated by FME and FME Server Validation key to support upload services
  • 27. Publication Produce INSPIRE compliant GML Provide discovery, view or download services, for WxS, GML and other desired formats Publish with FME Server or integrate with your geo web server of choice:  ArcServer (ArcGIS for INSPIRE)  Deegree  GeoServer  MapServer Spatial Data Services
  • 28. PublicationHow to make data most accessible? Support for mandated open standards and OGC services Support for defacto industry standards, proprietary and legacy agency systems Possibilities for Invoke Spatial Data Services as a bridge  accepts user request  queries required INSPIRE OGC services  provides desired web service (KML, PDF)
  • 29. Swedish Transportation Administration:Invoked Services  Supports estimation of new road costs related to archeological and protected sites  Site potential using terrain model and historic coast lines  WFS Sources:  The Swedish Transport Administration roads  Swedish EPA Protected sites  Swedish National Heritage Board Archaeological findings  Result: PDF showing site potential
  • 30. FME Tools for INSPIRE  Format translation  Schema mapping  String and list manipulation  Data validation  Database load and extract  XML,GML,WFS: reading, validation, publication  Web services: WFS, WMS, integration with others  Metadata support  Enterprise services with FME Server
  • 31. Poll QuestionSpecifically related to INSPIRE, in what areas haveyou used FME?
  • 32. Harmonization Stages:Geographic Names Demo
  • 33. Geographic Names: GML Result
  • 34. Other INSPIRE Examples Finnish National Land Survey  GML to Shape conversion  Data upload and validation  WPS test pilot – FME Server / Deegree integration Technical University Of Munich  XMI mapping of UML schema transformation rules  Maps between AAA and INSPIRE data models  Uses FME to perform schema mapping based on XMI Nature SDI: Datasiel, Liguria Region, Italy  FME data harmonization -> loads staging database  Generates protected sites GML for WFS Croatian NSDI: Geoportal  By State Geodetic Administration (SGA)  Central register of base map, ortho, spatial units, cadastre  FME used primarily in data migration and updates
  • 35. Example INSPIRE Solutionsby FME Partners INSPIRE Solution Pack from con terra (> 20 implementations across Europe) Metria, Sweden (Protected Areas Pilot; Swedish Department of Transportation) HNIT Lithuania (LGII) Technical University of Munich (multiple INSPIRE projects) AED Sicad (NAS to INSPIRE conversions) Spatialworld, Finland (National Land Survey)
  • 36. con terra – FME INSPIRESolution Pack Highlights Simplify data transformation and schema mapping for ArcGIS for INSPIRE Predefined FME Workbench templates help jump start the data migration and Harmonization process Connect your existing databases to the standardized ESRI INSPIRE Geodatabase Automate data load and update processes Quality assurance and metadata Publish once and use many times
  • 37. con terra – FME INSPIRE Solution Pack: 3 Key Tasks Consumers transformation & integrationInternal information/ INSPIRE fusionCenter –storage systems Data models and web services European SDI 2. Integration into 1. Data Models for spatial 3. Access via INSPIRE network business processes and data sets compliant to services meeting the Quality- transformation of data to INSPIRE data specifications of-Service requirements (COTS INSPIRE (COTS product) product)
  • 38. con terra - FME INSPIRE Solution Pack:Schema MappingETL Workflow description import create destination import destination schema Load Data source data mapping INSPIRE GDB schema schema FME data import process FME INSPIRE Solution Pack automated
  • 39. Components of the FMEINSPIRE Solution Pack Additional INSPIRE information  Tutorial workspace (complete sample mapping [AdminUnits])  Additional HTML Workbench Help (description of INSPIRE GDB data model)  Direct access to the INSPIRE data specification (link to specific themes) Additional functionality  Template workspaces (destination schema and predefined workspaces)  Destination data schema for all Annex 1 themes (INSPIRE GDB of ArcGIS for INSPIRE http://resources.arcgis.com/de/content//arcgis-inspire/1.0/about )  More than 100 additional INSPIRE transformers  INSPIRE specific value and attribute mapping (voidable values)  transformer for recurrent tasks (ID management, lifespan setter)
  • 40. Solution PackTemplate Workspaces one template Workspace per Annex I theme predefined datasets (INSPIRE GDB) predefined attribute-creation and settings predefined attribute-mapping integrated description for mapping file completion
  • 41. INSPIRE Transformers(Attribute and Value Mapping)  Automated filling of obligated attribute  legalStatus  legalStatus_void  Possible values for obligated attributes (if not void)  agreed  notAgreed  Predefined void value reasons (if void)  0 = no reason given  1 = reason: unknown  2 = reason: unpopulated  AttributeRenamer functionality  usable if attribute already existing
  • 42. FME – the pillar of anINSPIRE Infrastructure Jeff KonnenAdministration du Cadastre et de la Topographie Service Géoportail Grand-Duché de Luxembourg
  • 43. Or ... Thanks to FME, the red lion is juggling with INSPIRE
  • 44. About the ACT ACT : Administration du Cadastre et de la Topographie  Administration depending on the Ministry of Finance  130 people  Responsible for:  Cadastral documentation  Administrative boundaries and frontiers  Delimitation of properties  Management of property-related and topographic databases  Management of the national address register  Topographic documentation / maps  Management of geodetic networks  Management and hosting of the national Geoportal
  • 45. About the ACT: website
  • 46. About the geoportal Central SDI platform for the Luxembourgish State  Visualization of national geodata  General viewer (http://map.geoportal.lu )  Thematic viewers (eg http://tourisme.geoportal.lu)  Mobile Viewer (http://m.geoportal.lu )  Geo-Applications  Routing (http://routing.geoportal.lu )  Distance map  Coordinate conversion  Shop  Maps  Cadastral extracts  Geodata  etc
  • 47. About Geoportal.lu
  • 48. Map.geoportal.lu
  • 49. Map.geoportal.luwith other sauces
  • 50. map.geoportal.lu About 130 different layers Very fast (everything is tiled and precalculated) Easy to use (Google Style) Based on Mapfish Technology Single Search Field searching in 1Mio different entries (Addresses, Parcels, Localities, Rivers etc) Stats: 1300+ single users per day, tendency growing API
  • 51. About me Jeff Konnen  Project Leader at the ACT / Geoportal  Long time FME User  FME Professional & Trainer  jeff.konnen@act.etat.lu
  • 52. Geoportal vs INSPIRE ACT and especially the Geoportal crew have been designated as INSPIRE Point of Contact (NPOC) for Luxembourg The Geoportal crew has to manage metadata and Network Services for the Luxembourgish Node of INSPIRE Since May 2010, Luxembourg is reporting and monitoring about its Network Services and since November 2011, the Discovery and View Services for Annex I & II Themes are available online (http://inspire.geoportal.lu/geoportal) Since June 2012, a first set of Download services is available (IOC)
  • 53. INSPIRE Portal
  • 54. Geoportal vs INSPIRE The Geoportal crew of Luxembourg consists of 5 people  1 Manager  1 Project Leader  2 Developers  1 IT Manager There are not a lot ressources for INSPIRE  A complete solution for serving INSPIRE relevant data had to be found  ArcGIS for INSPIRE
  • 55. ArcGIS for INSPIRE INSPIRE Datamodel for the Geodatabase  We didn’t have to implement the INSPIRE datamodels by ourselves INSPIRE Geoportal  Management of the INSPIRE Metadata and the INSPIRE Discovery Service  Caveat: Our main Metadata management is multilingual (4 languages). By using INSPIRE Geoportal we decided to go English only for the INSPIRE relevant metadata INSPIRE Add-on for ArcGIS Desktop  Create maps for INSPIRE based on the INSPIRE datamodel INSPIRE Add-on for ArcGIS Server  Creates INSPIRE View Services and Download Services with one click
  • 56. ArcGIS for INSPIRE .
  • 57. ArcGIS for INSPIRE .. What ArcGIS for INSPIRE does not do:  Convert your data into the INSPIRE Datamodel  That’s where we need FME  FME is the tool used to convert data from the production databases into the INSPIRE datamodel  The con terra ISP (INSPIRE Solution Pack) helps to understand the technical aspects of the mapping that has to be done and to streamline your efforts  The semantical mapping of the data has to be done by the thematic experts, no solution will do it for you
  • 58. FME and ISP Using FME and ISP, a set of scripts has been created  These scripts can be executed regularly to fill the INSPIRE Datamodel  The INSPIRE View Services are based on the content of the INSPIRE Datamodel  Loading takes about 24 hours  Executed once a year or more often Themes:  Annex I: AU,AD,CP,GN,TN,(PS),HY as INSPIRE View Services  Annex II: OI,EL,LC,GE as WMS 1.3.0 as A4I not ready yet
  • 59. Example: Mapping of the Annex ITheme AD
  • 60. FME and INSPIRE : Lessonslearned FME is the right tool to transform your data ISP helps you to streamline your efforts The complicated part is not FME, it’s the semantic mapping (!)  Plan a lot of time for this part  A lot of discussions / compromises to find
  • 61. FME and INSPIRE : Next steps Activate some more download services for Annex I (and II) until the end of the year  Only « one click » but a lot of questions to clarify (pricing) Do the mapping for Annexes II and III  Wait for the datamodels to be final  Activate their view and download services Permanently : Keep up with new data
  • 62. Uses of FME Server: Shop
  • 63. Uses of FME Server: Shop Shop: Automatic Extraction of orders
  • 64. Uses of FME Server: Shop(Orchestration with Syncserv)
  • 65. Other Uses of FME Weekly syncronization of production data in the geoportal databases  STAR (CAD) -> ORACLE  ORACLE -> PostGIS Daily conversion of measurement files (STAR) to DWG Quality control  QC of Administrative Unit data Statistics  Count features issued by the different WFS services
  • 66. Other uses of FME (Server).. Online coordinate conversion (CSV File conversion from WGS84 -> LUREF (EPSG:2169) using FME Server
  • 67. Other uses of FME (Server).. Preview of DWG in files in the shop before ordering  Streaming service doing a generic transformation from DWG to PNG
  • 68. Other uses of FME .. Weekly generation of an address point database based on the centerpoints of the biggest building inside a parcel
  • 69. Other uses of FME .. Weekly transformation of heterogeneous data stored in production tables into one homegeneous search table for the Full-Text- Search used in the geoportal
  • 70. Other uses of FME .. One – shot analysis and export jobs Mosaicking and Production of ECW – Files for background rasters
  • 71. Other uses of FME:Grayscale conversion
  • 72. About FME and its use The general credo of the geoportal is to use a maximum of FOSS components:  PostGIS  Mapserver  Mapfish (OpenLayers, ExtJS, GeoExt) Exceptions:  Use of ArcGIS:  Complex symbologies  Data delivered by FGDB / MXD  ArcGIS for INSPIRE  Use of FME  FME is the only tool which allows us to really MANIPULATE geodata, there is no alternative
  • 73. Any questions? Email: Jeff.konnen@act.etat.lu Follow us on twitter: @geoportal_lux
  • 74. INSPIRE Trends Integration between proprietary and open standards – especially as deployment moves to regional and local agencies Increased focus on consuming INSPIRE services and GML More opportunities for spatial data and invoke services to bridge this gap Examples: machine to machine communication to provide seamless support for KML and PDF streaming Success = User isn’t aware they are accessing INSPIRE
  • 75. INSPIRE Compliant GML – Now What?<?xml version="1.0" encoding="UTF-8"?><gml:FeatureCollection xmlns="http://www.w3.org/2001/XMLSchema" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:ps-f="urn:x-inspire:specification:gmlas:ProtectedSitesFull:3.0" xmlns:ps-n2000="urn:x- inspire:specification:gmlas:ProtectedSitesNatura2000:3.0" gml:id="id-0"> <gml:boundedBy> <gml:Envelope srsName="EPSG:4326" srsDimension="2"> <gml:lowerCorner>0 360</gml:lowerCorner> <gml:upperCorner>0 360</gml:upperCorner> </gml:Envelope> </gml:boundedBy> <gml:featureMembers> <ps-f:ProtectedSite xmlns:ps="urn:x-inspire:specification:gmlas:ProtectedSites:3.0" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tn-a="urn:x-inspire:specification:gmlas:AirTransportNetwork:3.0" xmlns:base="urn:x-inspire:specification:gmlas:BaseTypes:3.2" xmlns:tn="urn:x-inspire:specification:gmlas:CommonTransportElements:3.0" xmlns:gn="urn:x-inspire:specification:gmlas:GeographicalNames:3.0" xmlns:net="urn:x-inspire:specification:gmlas:Network:3.2" gml:id="ID_0A16FE5BFD6C9CB51DE1DEE17804A757"> <ps:geometry> <gml:MultiSurface gml:id="id-9e3bfd62-aa14-4f81-a3ef-9af6d5129518-0" srsName="EPSG:4326" srsDimension="2"> <gml:surfaceMember> <gml:Surface gml:id="id-9e3bfd62-aa14-4f81-a3ef-9af6d5129518-1"> <gml:patches> <gml:PolygonPatch> <gml:exterior> <gml:LinearRing> <gml:posList>59.42 18.87 59.42... </gml:posList> </gml:LinearRing> </gml:exterior> <gml:interior> <gml:LinearRing> <gml:posList>59.41 18.89 59.47 ... </gml:posList> </gml:LinearRing> </gml:interior> </gml:PolygonPatch> …
  • 76. Now What?“How is anyone going to use INSPIRE?” FME allows you to consume INSPIRE compliant GML as well as create it. Why not build value on the web services every one else is investing in? Extract and flatten complex INSPIRE structures for use within your GIS applicationsAdd spatial data services to transform INSPIRE intodata streams suitable for widely accessible clientapplications (PDF, KML, GeoJSON)
  • 77. XML vs. Relational Relational attribute1 attribute2 Active Active Date_From Date_To John Vancouver 11-22-99 12-11-09 June Surrey 02-25-05 9-15-10
  • 78. Reading INSPIRE GML:with flattening
  • 79. Output to PDF
  • 80. INSPIRE Hydrography fromLuxembourg WFS on FME Viewer
  • 81. Poll QuestionWhich areas would you like to see enhancements toFME’s support for INSPIRE?
  • 82. FME Tools for INSPIRE  Format translation  Schema mapping  String and list manipulation  Data validation  Database load and extract  XML,GML,WFS: reading, validation, publication  Web services: WFS, WMS, integration with others  Metadata support  Enterprise services with FME Server
  • 83. FME Tools for INSPIRE XML GML reading and writing (3.2.1 and earlier) XML Reading – read virtually any XML and extract features XML Processing – validate, format, and update XML Geometry Extraction (features > GML objects) XML Writing - Write XML or GML no matter how complex XMLTemplater  FME merges attribute values into a template structure  Template per feature type and dataset XML Validation – validated to ensure compliance with INSPIRE schemas
  • 84. Complex GML Geometry Support:CityGML (basis for Annex III Buildings)
  • 85. FME Tools for INSPIREFME provides a full range of tools for INSPIRE, ourpartners provide complete solutions Data modeling interface makes it easier to handle schema mapping - no coding required. Automated workflows save time in creating and maintaining transformation processes. Self-documenting workspaces ease collaboration Easily integrates with INSPIRE-compliant services:  Esri ArcGIS Server  Deegree, MapServer and GeoServer Scalability and performance support quality of service requirements
  • 86. Summary: Solution Approaches Typical stages for SDI projects: Evaluation, assembly, transformation, validation and publication FME provides tools that empower solutions for each step of the INSPIRE harmonization process Our partners provide a range of solutions  Conterra’s INSPIRE Solution Pack for FME Luxembourg Implementation of by Jeff Konnen INSPIRE trends  consuming INSPIRE data  need for spatial data services to support common clients FME provides tools to support INSPIRE, our partners apply these to provide solutions
  • 87. Poll QuestionInterested in a free seat of online training?
  • 88. Thank You! ?Questions?For more information: Jeff Konnen jeff.konnen@act.etat.lu Dean Hintz dean.hintz@safe.com www.safe.com/inspire www.fmepedia.com (examples)