Validation and Quality Control with FME and FME Server

  • 538 views
Uploaded on

See more FME World Tour 2014 presentations at www.safe.com/recap2014

See more FME World Tour 2014 presentations at www.safe.com/recap2014

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
538
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
18
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Etc:Number of featuresNetwork topologyMeta data….
  • Can have schema reader AND reader in workspaceStoakes:Schema(any format) is very useful. You can compare the schema of submitted data with the schema ‘standard’ and if there is a mismatch you can reject the data. If the data matches the data model standard then you can proceed with the data load.

Transcript

  • 1. CONNECT. TRANSFORM. AUTOMATE. Validation and Quality Control with FME and FME Server
  • 2. CONNECT. TRANSFORM. AUTOMATE. Validation and Quality Control with FME and FME Server
  • 3. CONNECT. TRANSFORM. AUTOMATE. Validation and Quality Control with FME and FME Server
  • 4. Outline  Why are we talking about Validation and QC ?  What can we check?  Schema  Attribute Values  Geometry  Etc.  Validation Workflows on FME Server  Partner Examples
  • 5. Why are we talking about this?  Bad data costs time, money and even lives.
  • 6. Why are we talking about Validation and Quality Control?  FME and FME Server are powerful tools for checking data and providing feedback  Many partners, integrators and users use FME today for Validation and Quality Control
  • 7. What can we check?  Schema  Feature Type Name  Attribute Present  Attribute Names  Attribute Data Types  Coordinate System
  • 8. What can we check? Schema Checking – Key Tools  Schema (Any Format) Reader  FeatureTypeFilter  StatisticsCalculator  AttributeClassifier transformer  XMLValidator Transformer  JSONValidator Transformer
  • 9. What can we check?  Attribute Values  Values exist  Data Type (numeric, alpha etc)  Value within in range  No duplicates
  • 10. What can we check? Attribute Values – Key Tools  Tester and TestFilter  AttributeClassifier  AttributeRangeFilter & AttributeFilter  NullAttributeMapper  Matcher  DuplicateRemover  StringSearcher  StatisticsCalculator
  • 11. What can we check?  Geometry 2D and 3D  Intersections  Orientation  Duplicate Vertices  Missing Surface Normals  Invalid Solid Voids  etc
  • 12. What can we check? Geometry Checking – Key Tools  GeometryValidator (12 tests and counting)  Detection and repair  old transformers bundles in eg Self-Intersector  GeometryFilter  Matcher  SpatialFilter (Sweco example)  SliverRemover & SpikeRemover  AreaCalculator & LengthCalculator  OrientationExtractor
  • 13. Workspace Example
  • 14. Validation Workflows on FME Server  Submit data via email, upload, directory watcher, scheduled task  Provide report or results via email, form, data download, PDF
  • 15. Validation Workflows on FME Server
  • 16. Partner Validation Examples #1 Axmann GeoChecker 1. Users upload data 2. Workspaces Check Schema, Geometry, Attributes 3. Email sent with PDF Report and Bad Data in AutoCAD
  • 17. Partner Validation Examples #1 Axmann GeoChecker Detailed Reports & AutoCAD File of Errors
  • 18. Partner Validation Examples #2 Veremes - Qualigéo  Generic Approach to avoid “Chamber of Horrors”
  • 19. It's easy Constraint FME tool Geometry GeometryValidator Spatial Relationship SpatialFilter/SpatialRelator Integrity Constraint Feature Merger Topology Attribute Domain String Searcher Statistic Statistic Calculator Schema Schema reader Unique Key Duplicate Remover
  • 20. It's complicated
  • 21.  Hyper generic workspace • 1 unique workspace to validate all formats, all data models, all constraints...  Using an external description of the data model and constraints Solution
  • 22. Partner Validation Examples #3 Metria Sweden Environmental Monitoring Data – ISO QC-service on FME server Metadat a XML INSPIRE WFS/SOS QC- report Metadata portal Dataset GML
  • 23. Partner Validation Examples #3 Metria Sweden Environmental Monitoring Data – ISO
  • 24. Partner Example #4  1Spatial - UKPostcodeValidator
  • 25. Thank You!  Questions?  For more information:  info@safe.com  www.safe.com