1. Using FME to Automate
Region‐Wide Data Sharing
Steve Grisé – Vertex3
April 10, 2013

2. A Perspective
 Based on recent project work
 Working on data models
 Defining processes
 New editing tools
 Getting agreement
 And a bit of experience
 ArcGIS
 FME
 …

3. Recent Projects
 Region of Waterloo
 Regional data model for Roads and Addresses
 Editing tools
 Data exchange with Cities
 York Region
 Parcels, Addresses and Roads data model
 All Pipes Data Model
 Implementation in partnership with Timmons
Group

4. Current Situation
 Most of the data shared between organizations is
almost the same
 Different Names, same data
 Differences Roads Addresses
 Dataset names ROADSEGMENTID
STREET_NAME
HOUSE_NUMBER
UNIT_NUMBER
 ROAD vs. Roads LEFT_FROM STREET_NAME
 Field names LEFT_MUNI
...
ADDRESS_POINT_ID
...
 ROADID vs. GeoID GeoID HouseNumber
 Field values
FullStreetName UnitNumber
LeftFrom FullStreetName
 CAM vs. Cambridge LeftMunicipality AddressID
... ...
 DI vs. Ductile Iron

5. Region of Waterloo Example
City Extract, Transform, and Load (ETL), QA QA and Update to Region-wide Database Regional Systems
ETL, QA c c Regional update
using sources
Cambridge
Regional Change Sync
ETL, QA k k QA Detector
Kitchener Region-wide Internal &
Database External
ETL, QA w w Databases
Waterloo
QA Issues fixed at source
Partner read only access to Region-wide Database
ETL can be done on local systems or regional
Region-wide Update done by Regional Staff,
server. Partners manage their own ETL and QA
semi-automated processes
to Region-wide model

6. Our options haven’t changed

7. A new idea
 We need a new user experience
 Arcgis and/or FME on the desktop
 Source-target mapping vs. programming
 Eventually a knowledge-based system that does
80% of the work for you
 FME becomes the engine that drives your data
movement

8. Gizinta.com
1.0 – process and some UI
2.0 – knowledge-based ETL

9. Approach
Gizinta.fmw
Extract
Sources
Transform
Jdbdjbjdb
Jdbdjdj
Jnjdbjn
Add fields
Map values
Jndjdbjnj
n
Gizinta Project Calculate values
Load
Multiple options

10. Why Use FME?
 Faster
 Larger datasets
 You prefer it
 More challenging extract and load needs
 Use a bit of FME Server…

11. Demo
 Initial workbench
 Move Cambridge roads to regional data model

12. Next Steps
 July 1.0 release
 Exploring team and approach for 2.0
 Free downloads on
 ArcGIS Galleries
 FME Store
 Premium subscription options
 Looking for a few good projects over the next
several months

13. Longer Term Ideas
 FME cloud implementation
 Build and test locally
 Submit data and Gizinta project to online service
 Download or distribute via Gizinta site
 Benefits
 Pay-for-use model
 Reduced processing time

14. Thank You!
 Questions?
 For more information:
 Steve Grisé
 steve@vertex3.com
 226-444-0323