Producing Standardized Data Using a Master FME Workspace

3,166 views
3,031 views

Published on

Presented by Victor Rutherford of Nexen Inc
Abstract:
We have successfully implemented Corporate-wide standards for our spatial data through the use of a Standard Data Model, some custom technical tools and a Generic Master FME script at the heart of it all. Nexen Inc. had a large amount of spatial data being consumed by various departments, and an effort was made to try to standardize it to help manage it all which initially missed the mark. The initial attempt led to various FME scripts with varying styles of design by multiple authors of varying degrees of expertise in FME. Management regrouped and succeeded with a different approach: One Generic Master FME script flexible enough to manage most of the data-manipulation scenarios and which also implemented the Spatial Data Standards which were desired into each data load and update. This also tremendously improved maintenance of the ETL environment.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,166
On SlideShare
0
From Embeds
0
Number of Embeds
1,883
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Producing Standardized Data Using a Master FME Workspace

  1. 1. Producing Standardized DataUsing a Master FME WorkspaceVictor RutherfordGIS Data Analyst, Developer April 25, 2012
  2. 2. Spatial Database Geomatics Vendor Vendor Cartography GIS Geology DB Geophysics Analysis Vendor Vendor Engineering
  3. 3. Loading and Updating Data - Unstandardized - Difficult to maintain - Confusing to use GIS DB
  4. 4. Master FME Workbench GIS DB (Custom) (Generic) Standardized
  5. 5. Magic Formula? 3 1 2 + + Standard Data Model Data Modeling Tool+ Generic FME Workbench Lookup tableAttribute-mappings+ PSDM PSDM (domains) Master Manager FME Workspace
  6. 6. The Right Person ETL/FMEI understand rocks, Corporate Standards Steward FMEbut what’s an Expert / Domain FME? Data Modeling Rocks! QA Data (Generic) Tool Modeling
  7. 7. Master FME Workspace(Look Ma, No hands!)
  8. 8. Master FME Workspace
  9. 9. Master FME Workspace
  10. 10. Master FME Workspace
  11. 11. Master FME Workspace
  12. 12. Match = No ChangeSource ID CRC Mismatch = UpdateTarget ID Insert Delete?
  13. 13. Master FME Workspace
  14. 14. Master FME Workspace
  15. 15. Master FME Workspace Easier to Maintain Harder to Maintain
  16. 16. Master FME Workbench GIS DB GIS DB (Custom) (Generic) Standardized GIS DB
  17. 17. Thank You! Questions? For more information:  Victor Rutherford  vr@vrgs.ca  victor_rutherford@nexeninc.com  Nexen Inc. www.nexeninc.com  VRGS Ltd. www.vrgs.ca
  18. 18. No ID/PK?Source ID Match = No ChangeTarget CRC ID Mismatch = Insert Mismatch = Delete?
  19. 19. Published Parameters Source PSDM Parameters Output
  20. 20. Custom Transformers?

×