FME World Tour 2017 - Blending Enterprise Data with FME Server
•Download as PPTX, PDF•
0 likes•81 views
GIS has become the foundation of our organization’s asset management program. Many of our business systems rely on our GIS services for authoritative spatial data and map services. Other business systems (SunGard ERP, Cityworks CMMS, and legacy WinCan CCTV inspection system) contain critical non-spatial data that must be linked to spatial asset information. This presentation demonstrates how we are using FME Server to “blend” spatial and non-spatial data together for the ArcGIS Server map services that power our Geocortex applications, our Cityworks CMMS, and our new ITpipes CCTV inspection system.
Recommended
Recommended
More Related Content
Similar to FME World Tour 2017 - Blending Enterprise Data with FME Server
Similar to FME World Tour 2017 - Blending Enterprise Data with FME Server (20)
More from Carl Von Stetten
More from Carl Von Stetten (6)
Recently uploaded
Recently uploaded (20)
FME World Tour 2017 - Blending Enterprise Data with FME Server
- 2. About Me GIS Analyst, Central San Carl Von Stetten
- 4. What is “Enterprise Data”? • Non-spatial tabular data • Often part of Commercial Off-The- Shelf (COTS) software • Computerized Maintenance Management Systems (CMMS) • Enterprise Resource Planning (ERP) systems • CCTV or other inspection systems • Stored in an enterprise DBMS • Oracle • Microsoft SQL Server • IBM DB2 • SAP • Microsoft Access** • Frequently involves lots of related tables
- 6. Old Autodesk MapGuide Web GIS
- 7. Old Web GIS Pop-Up Reports Access to Enterprise Data via Links only
- 8. New Geocortex Web GIS
- 9. Geocortex Details Pane This is the perfect place for some Enterprise data!
- 12. Evolution of a Gravity Sewer Main 1989 – Pipe installedM1 M2 Pipe ID: M2/M1 2003 – CCTV inspection M1 M2M5 M6 Pipe ID: M5/M1 Pipe ID: M2/M5 2005 – New MH installed
- 13. CCTV Pipes vs. GIS Pipes M1 M4PR2 PR3 M4/PR3PR3/PR2PR2/M1 GIS Pipes M1 M4PR2 PR3 M4/M1 “Nonstop MH to MH” CCTV Inspection M1 M4PR2 PR3 “Perfect Match” CCTV Inspection M4/PR3PR3/PR2PR2/M1 M1 M4PR2 PR3 “Reverse” CCTV Inspection PR3/M4PR2/PR3M1/PR2
- 14. Identification Formats Change Structure Type Old GIS ID Format CCTV ID Format GIS ID Format Manhole 75A1 M 12 75A1M12 75A1-M12 Cleanout 9E7 CO10 9E7CO10 13B7-CO10 Junction Structure 47C3 JS5 47C3JS5 47C3-JS5 Rodding Inlet 101E7 R 22 101E7R22 101E7-RI22 Pressure Relief 47D6 P 6 47D6P6 47D6-PR6 Wye 13B5 W 5.3 13B5W5.3 13B5-W5.3
- 16. 1 1 Gravity Sewer Main Evolution FME Server Process Builds “Junction Table” 2 2 2 CCTV vs. GIS Pipes 4 Create Junction Table Records 4 3 3 Identification Format Change3
- 17. FME Server Process Builds “Junction Table”
- 18. FME Server Process Builds “Junction Table”
- 20. Cross- Database joins • May not be common primary/foreign key columns • May not be optimally indexed • Datatypes may not match (casting is slow) • May mix DBMS platforms • MS Sql Server • Oracle • IBM DB2 • Sybase • MS Access
- 21. Enterprise database table joins • 1:Many relationships may need aggregation • May not be optimally indexed for joins • Data may not be consistent • May be deeply nested Table1 idPK blahblahblah yadayadayada Table2 idPK blahblahblah yadayadayada fk_idFK 1:M Table3 idPK blahblahblah yadayadayada fk_idFK 1:M
- 22. Geocortex supports joining to Enterprise databases, but… • Still subject to the “pain points” • poor performance • data inconsistencies • aggregation • Users must “drill down” through relationships to get to the joined data (e.g.: extra clicks) • Doesn’t support nested table joins and 1:Many relationships
- 24. “Blend” the Enterprise data into the GIS features 1. Add extra fields to relevant geodatabase feature classes 2. Use FME Server to write the enterprise data into those fields 3. Profit!
- 27. “Blend” the Enterprise and GIS data into a non-versioned “Publish” geodatabase 1. Duplicate the geodatabase schema to new “Publish” geodatabase 2. Add extra fields to relevant geodatabase feature classes 3. Use FME Server to truncate/repopulate the geodatabase with the “blended” data 4. Profit!
- 28. FME Server Publishes “Blended” Sewer Features
- 29. InlineQuerier gets the job done
- 30. Cross-Database Joins Citworks work order data pulled in via SQLCreator transformers GIS Features WinCan CCTV data pulled in via SQLCreator transformer CCTV “Junction Table” created in Solution #1
- 31. 7 tables for parcel address and ownership data in ERP Does parcel has permits in ERP? Does Environmental Compliance monitor businesses on this parcel? GIS Features/Attribute Tables Results from FME PointOnAreaOverlayer Transformers Cross-Database and Nested Table Joins
- 33. FME Server Works While I Sleep! Parcel Blender Mon-Fri @ 10:30pm Sewer Blender Mon-Fri @ 11:30pm CCTV Junction Table Mon-Fri @ 10:30pm
- 34. Blended Sewer Data in 3 Clicks Place your screenshot here
- 35. Blended Sewer Data in a Themed Layer Place your screenshot here
- 36. Blended Parcel Data in 3 Clicks Place your screenshot here
Editor's Notes
- Our Enterprise data: CCTV Data Cityworks CMMS with work order data Sungard ERP with parcel address/ownership information
- But how did we get here?
- But how did we get here?
- Manually store the “new” structures in a comma-separated list in the CCTV inspection database – I’ll use this in the FME process I’ll show a little later in this presentation
- Doing some string manipulation with that CCTV comma-separated list I mentioned earlier for new structures, and then create links between the CCTV pipe and the new multiple GIS pipes Deals with when the inspector “skips over” the minor structures separating pipes or does a reverse inspection.
- Cityworks Join – IDs in GIS are numeric, but text in Cityworks Sungard ERP parcel IDs are fragmented across three separate columns CCTV issues (already talked about)
- Parcel address/ownership information in ERP stored in seven related tables
- Not too complex… but check this out!