This document discusses how FME (Feature Manipulation Engine) software can be used to support Open Data initiatives and INSPIRE requirements by transforming data into common schemas. It describes the key steps in the harmonization process including evaluation, assembly, transformation, validation, and publication of data. Specific FME capabilities that help with schema mapping, metadata, validation, and publishing data as web services are also outlined. The document concludes with an example of how FME has been used to demonstrate transforming data to Open Data standards.

1. Using FME to support Open
Data initiatives and INSPIRE
Ciaran Kirk Tony Kent
Technical Director GIS Technician
6th September 2012

3. The Challenge
 You want to meet Open data requirements, but
your data is organized rather differently
?

4. What we do …
?
We build data bridges – to your SDI

5. Harmonization Core Concepts
 Harmonization: implied requirement for building an
SDI
 Disparate sources must be mapped to a common
destination data model
 Core to the harmonization workflow is a process called
schema mapping.
 Delivered by services based on open standards

6. INSPIRE Harmonization Principles
Typical stages:
1. Evaluation
2. Assembly
3. Transformation
4. Validation
5. Publication
Based on the Spatial ETL concept (Extract,
Transform and Load), as applied to INSPIRE SDI’s

7. Evaluation
 Assess destination schema and data requirements
 Assess source datasets and schema
 Consider fundamental differences in representation,
resolution
 Closely inspect actual representative datasets

8. Data Assembly
 Assess the diversity of source data types:
 vector, raster, CAD, GIS, database, text, XML, web,
3D, sensor and non-spatial
 Review format and semantic translation needs
 Decide how to perform necessary joins
 ID joins, spatial relates, nearest neighbor, one to
many relationships
Goal is to build a data structure that corresponds
with your published standard

9. Metadata – Data about data
Describes data structures
• tables
• geometry types
• data types
• fields
Describes data content
• coordinate system
• extent
• modification date
• quality, ownership, etc.

10. Metadata - Purpose

11. Key FME Metadata Capabilities
• Reading
• Writing
• Updating
• Harvesting
• Validating
• Integration with web services

12. CSW: Catalog Service for the Web
• Deegree
• GeoNetwork
• OpenGIS
• TerraCatalog

13. Metadata Services: CSW
• Reading
• Writing / Publishing
• Updating

14. Data Transformation - Schema
 Reshape source data to match required
destination schema
 Schema mapping
 feature type
 attribute name
 new attribute creation
 code lists
 conditional value
mappings

15. Schema Mapping in FME
Feature Type Mapping in FME Workbench
Attribute Mapping in FME Workbench

16. FME Data Model Restructuring:
Attribute Names & Values
 Value Mapping

17. FME SchemaMapper:
INSPIRE geographic names
FME Workspace
Name mapping
Name &
value mapping

18. Transformation: Geometry
 Non-spatial to spatial
 Geometry extraction (spatial to GML)
 Representation transform: CAD drawing lines with
labels to GIS polygonal features with attributes
 Coordinate System Reprojection (ED50 to ETRF89)
 Simple to complex geometry
 Source point and polygon data to multiple geometric
representations (city as point / area, river as line / area)
 Generalization and interpolation
 Highly granular national and regional datasets often require
thinning to be usable on pan-European scales

19. Validation
 Schema validation i.e. INSPIRE (xsds)
 Data integrity
 Unique IDs
 Geometric integrity (closed polygons)
 Null values (nullable?)
 Valid values: ranges and domain codes
 Data gaps
 Bounds
 Network integrity
 Custom validity rules specific to domain
 Validation automation via FME Server upload
Ensure data quality throughout the data transformation process.

20. Publication
 Produce INSPIRE compliant GML
 Provide discovery, view or download services, for
WxS, GML and other desired formats
 Publish with FME Server or integrate with your
geo web server of choice
 Spatial Data Services

21. Publication with FME Server
 Publish workspace to FME Server
 Store the workspace in a central repository
 Make your FME workspaces available to others –
over the web
 Register the workspace with one or more services
(Data Streaming, Data Download, etc.)
21

22. Open Data Demonstration

23. FME Tools for INSPIRE
 Format translation
 Schema mapping
 String and list manipulation
 Data validation
 Database load and extract
 XML,GML,WFS: reading, validation,
publication
 Web services: WFS, WMS, integration
with others
 Metadata support
 Enterprise services
with FME Server

24. Summary
 FME can provide all the tools to help you support
your data sharing needs:
 Integrate your data sources
 Manage your meta data catalogues
 Transform your data to standard schemas
 Publish the data in the required formats

25. Thank You!
 Questions?
 For more information:
 Ciaran Kirk – ckirk@imgs.ie
 Tony Kent – tkent@imgs.ie
 support@imgs.ie