Coupling Data Processing and Map Production
•Download as PPTX, PDF•
2 likes•1,206 views
Presented by Chinmaya Lewis, City of Santa Rosa Utilities Department See more FME World Tour 2014 presentations at www.safe.com/recap2014
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Coupling Data Processing and Map Production
Similar to Coupling Data Processing and Map Production (20)
More from Safe Software
More from Safe Software (20)
Recently uploaded
Recently uploaded (20)
Coupling Data Processing and Map Production
- 1. CONNECT. TRANSFORM. AUTOMATE. Chinmaya Lewis Geographic Information Systems Analyst
- 2. Asset Management Division Engineering Services
- 4. 200+ Traffic Signals 16,000 Street Lights 35,000 Street Signs 6,781 – Hydrants 589 Miles of Sewer Main 12,000+ Sewer Manholes 19 Sewer Lift Stations 336 Miles of Storm Drain 10,800 Inlets 90 Miles of Creeks 497 Miles of Roads 619 Miles of Water Mains 27,000+ Water Valves 20 Pump Stations 25 Reservoirs
- 5. CIP Project Planning, Tracking and Maps CIP Database for Project Info and Tracking Project Boundaries in GIS Boundaries are Asset Based (Ex. buffered pipes) Enables location based tracking, asset ranking and conflicts analysis FME is used to integrate, update and export data Maps can now be automatically generated or updated for all CIP Projects Data and Maps are consistent and up to date!
- 6. Map Production and Automation Data Processing& Integration Coupling Data Processing and Map Production “The Software Stack”
- 7. Reads – 5,300 Features Writes – 4,000 Features Produces – 550+ MapImages CIP Processing and Maps FME Workspace and Python Code
- 8. CIP Project Map Image produced by ArcMap Data Driven Pages
- 9. Maps are widely used Budget Reports Public Outreach
- 10. Online CIP Project Info Detailed online GIS Google Map
- 11. Street Lights Processing and Maps FME Workspace and Python Code Reads ~ 65,000 Features Writes ~ 48,000 Features Produces ~ 265 PDF Maps
- 12. Street Lights map produced by ArcMap Data Driven Pages
- 13. Wrap-Up Consistent repeatable data processing using FME Automated production of quality maps using ArcGIS Data Driven Pages Coupling Data Processing and Map Production means: GIS data and maps are consistent and up to date through a scheduled overnight process!
- 14. Thank You! Questions? For more information: Chinmaya Lewis – clewis@srcity.org City of Santa Rosa Utilities Department http://srcity.org/
Editor's Notes
- So before I get into the details, let me give you some quick background about who I work within the city. I’m part of the Asset Management Division of Engineering Services within the Utilities Dept<CLICK>
- Engineering Services supports departments Citywide with staff from both Public Works and Utilities.<CLICK>
- Asset Management - inventory City Wide Assets for installation, maintenance, inspection and tracking purposes. We also do Capital Improvement Program project planning and budgeting.Data is entered and maintained in either CAD, GIS or Computerized Maintenance Management System and in some cases all three.On this slide you can see examples of these assets and there total numbers. Water Hydrants and water delivery system, Sanitary Sewer system, Roads, Traffic system and lights and Storm System and Creeks. <CLICK>Managing city wide assets is not a simple task with multiple complex systems that need to tracked and maintained. While not all information/data can be quantified and digitized, we collect the data needed to effectively manage city assets.FME is used to integrate and process asset data. We started using FME for CAD to GIS translation in 2006.
- Capital Improvement Program for the City has numerous projects planned, in design or construction throughout the city. Last years (FY 2013/14) CIP Budget was $50 million. In Asset Management we have developed a CIP database used for project info and tracking.This data is stored in a stand alone SQL database.Project extents are created in GIS for all locatable CIP Projects using a asset based approach, ie buffered pipes.The CIP boundaries in GIS enables location based tracking, asset ranking and conflicts analysis as well as showing or discovering opportunities for combining projects.FME is used to integrate GIS and CIP database as well as update and export data.Now we do this integrate/update along with generating maps. Each project gets a map image created as part of our data updates.Data and Maps are consistent and up to date!<CLICK>
- the “software stack”Basically its <CLICK> FME and <CLICK> ArcGISFME Desktop does all the data processing from/to SDE, Access, AutoCAD, SQL Server and ArcGIS does the map production/automation. Data driven pages was new with ArcGIS 10Ask if anyone is using Data Driven pages in ArcMap. Before 10 it was a DS Sample called Map Book Generator.So, here’s how it went down… in setting up the data driven pages in ArcMap, I discovered that JPG export was not possible.I called ESRI Support about this issue and they explained it is possible through scripting and sent me sample python code. <CLICK> This spurred my “aha” moment. I realized I could have a FME Shutdown Script “run” the Data Driven Pages using Python to produces maps and do the data processing. <CLICK> basically, Python squeezes it all together.Now we have updated GIS data for Desktop and Web GIS Applications and updated Maps for viewing, printing or doc insert.Data and Maps are consistent and up-to-date. <CLICK>
- This FME workspace merges <CLICK> GIS features (in SDE) with our CIP database (in SQL) pulling Project info and doing a number of data transformations. <CLICK> Various copies are output via writers to SDE, SQL and shapefiles. This output data is accessed by ArcGIS desktop users and ArcGIS Server for online GIS.You can see the number of features this reads and writes which actually isnt’ that much. That’s the Data Processing.<CLICK> Maps are produced using a Python shutdown script that references the <CLICK>ArcMap MXD with Data Driven Pages enabled to produce 550+ JPG images nightly. Map images are also put in different folders on our network. That is the Sample Code provided by ESRI support albeit modified for our needs. Again, the JPEG export is only only available through scripting as are other parameter settings such as resolution, image quality and output file name control - No underscores in the name!As a shutdown script within the FME workspace it is run after the data processing.Scheduled Process on a Server that runs nightly.GIS Data and Web applications are up to date and so are the maps.
- Here are some examples of these maps that are produced. They show project location and extents for all Capital Improvement Project areas. 250+ Maps produced/updated nightly.6 <CLICKS>Dynamic text and labeling, scale dependencies, and page definition queries were all setup in ArcMap.You can see the quality of these maps and the varied extents and scales.Irregular project areas are the data used to generate map pages.
- These maps are used throughout the organization by Finance, Public Works and Utilities staff for Budget reporting and Public Outreach.This is where the JPG image format for the maps was needed.Departments have been very pleased with the maps which are also included in the Annual CIP Budget that is presented to the City Council.
- Online GIS with detailed boundaries and CIP project along with numerous other GIS layers.<CLICK>Something new is a Google map using Fusion tables to display project location icons with project info and links to a public project report.Do a live Demohttp://srcity.org/departments/publicworks/capitalprojects/Pages/CapitalImprovementProjects.aspx
- We are using a similar setup for Street Lights GIS data processing and maps.Using Python Shutdown Script within FME that references a Data Driven Pages MXD to produce 265 PDF’s nightly.Reads features from GIS and CMMS and writes multiple copies to GIS databases and shapefiles. The 265 Maps for this process are produced in PDF format.Scheduled Process on a Server.Data is up to date and so are the maps.
- These maps show Street Light locations, numbers, and lighting Status for the City’s Street Light Reduction ProgramMaps are generated by a regular tiled map grid. Dynamic text is also used for these.
- Wrap-UpSo what are we achieving with all this:Consistent repeatable data processing using FMEAutomated production of quality maps using ArcGIS Data Driven PagesCoupling Data Processing and Map Production means:<CLICK>GIS data and maps are consistent and up to date through a scheduled overnight process!The lizard looks happy about this and, more importantly, I sleep better at night!But seriously, we’ve had great success with this and I hope this may spur an “aha” moment for some of you.