Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using FME to Automate Region-Wide Data Sharing


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Using FME to Automate Region-Wide Data Sharing

  1. 1. Using FME to AutomateRegion‐Wide Data SharingSteve Grisé – Vertex3 April 10, 2013
  2. 2. A Perspective Based on recent project work  Working on data models  Defining processes  New editing tools  Getting agreement And a bit of experience  ArcGIS  FME  …
  3. 3. Recent Projects Region of Waterloo  Regional data model for Roads and Addresses  Editing tools  Data exchange with Cities York Region  Parcels, Addresses and Roads data model  All Pipes Data Model  Implementation in partnership with Timmons Group
  4. 4. Current Situation Most of the data shared between organizations is almost the same  Different Names, same data Differences Roads Addresses  Dataset names ROADSEGMENTID STREET_NAME HOUSE_NUMBER UNIT_NUMBER  ROAD vs. Roads LEFT_FROM STREET_NAME  Field names LEFT_MUNI ... ADDRESS_POINT_ID ...  ROADID vs. GeoID GeoID HouseNumber  Field values FullStreetName UnitNumber LeftFrom FullStreetName  CAM vs. Cambridge LeftMunicipality AddressID ... ...  DI vs. Ductile Iron
  5. 5. Region of Waterloo ExampleCity Extract, Transform, and Load (ETL), QA QA and Update to Region-wide Database Regional Systems ETL, QA c c Regional update using sources Cambridge Regional Change Sync ETL, QA k k QA Detector Kitchener Region-wide Internal & Database External ETL, QA w w Databases Waterloo QA Issues fixed at source Partner read only access to Region-wide Database ETL can be done on local systems or regional Region-wide Update done by Regional Staff, server. Partners manage their own ETL and QA semi-automated processes to Region-wide model
  6. 6. Our options haven’t changed
  7. 7. A new idea We need a new user experience Arcgis and/or FME on the desktop Source-target mapping vs. programming Eventually a knowledge-based system that does 80% of the work for you FME becomes the engine that drives your data movement
  8. 8. Gizinta.com1.0 – process and some UI2.0 – knowledge-based ETL
  9. 9. Approach Gizinta.fmw Extract Sources Transform Jdbdjbjdb Jdbdjdj Jnjdbjn Add fields Map values Jndjdbjnj nGizinta Project Calculate values Load Multiple options
  10. 10. Why Use FME? Faster Larger datasets You prefer it More challenging extract and load needs Use a bit of FME Server…
  11. 11. Demo Initial workbench Move Cambridge roads to regional data model
  12. 12. Next Steps July 1.0 release Exploring team and approach for 2.0 Free downloads on  ArcGIS Galleries  FME Store Premium subscription options Looking for a few good projects over the next several months
  13. 13. Longer Term Ideas FME cloud implementation  Build and test locally  Submit data and Gizinta project to online service  Download or distribute via Gizinta site Benefits  Pay-for-use model  Reduced processing time
  14. 14. Thank You! Questions? For more information:  Steve Grisé   226-444-0323