The document discusses how EPA automated the monthly refresh of spatial data in its Facility Registry Service (FRS) map service. It used FME to extract data from Oracle Spatial into a file geodatabase, then updated the underlying data for an ArcGIS Server map service. A Python script in FME packaged and deployed the new geodatabase, refreshed the map service, and notified other workflows. This automated a previously manual process and improved performance by moving from Oracle to a file-based system. EPA plans to trigger the FME workflow directly from Oracle and integrate it into other refresh flows.

1. From Oracle to the Web -
Automating Spatial Data Updates
David Smith, FRS Program Manager, EPA
Amy Ramsdell, GIS Manager, Blue Raster
April 8

2. Overview
 The Need: Automate spatial data refresh
 The Solution: Technologies used
 ArcGIS Server (AGS) map service final output
 FME workspaces that automate refresh process
 File geodatabase (fgdb) creation
 Metadata refresh
 Python shutdown script in workspace
 FME Server web services used
 EPA future plans for FME
 Questions

3. The Need:
Automate refresh of map service serving spatial data
 Operational database system
 The Facility Registry Service (FRS) is a centrally managed
database that identifies facilities, sites or places subject to
environmental regulations or of environmental interest.
 Disconnected static file for map service
 Fgdb faster performance in AGS
 Lessen activity on operational database
 Monthly manual refresh

4. The Solution:

5. ESRI ArcGIS Server Final Output
 Web server hosting GIS web
services
 REST endpoint for map service
http://geodata.epa.gov/ArcGIS/rest/services/OEI
/FRS_INTERESTS/MapServer
 Underlying data: file gdb
 28 layers (tables in fgdb)

6. Data Refresh FME Workspace
 Oracle Spatial reader
 ESRI File Geodatabase
writer
 Whole bunch of filtering
in-between to create
separate tables in fgdb
 Could build the filtering
in Oracle before it
reaches FME

7. Oracle Spatial Type Reader
Required for spatial object:
 Table, materialized view, or view
 Shape field is SDO_GEOMETRY
CREATE TYPE sdo_geometry AS OBJECT ( SDO_GTYPE NUMBER, SDO_SRID
NUMBER, SDO_POINT SDO_POINT_TYPE, SDO_ELEM_INFO
SDO_ELEM_INFO_ARRAY, SDO_ORDINATES SDO_ORDINATE_ARRAY);
 USER_SDO_GEOM_METADATA table entry
 Metadata for spatial tables owned by schema

8. ESRI File Geodatabase API Writer
 Non-ArcObjects access
to fgdb
 C++ API
 Does not require an ArcGIS license on FME
Server or Desktop
 Works in ArcGIS Server 10x

9. Python Shutdown Script in Workspace
 In Workspace Parameters – Advanced option

10. Shutdown Script in 3 Steps
1. Package the output
 zipfgdb(geodatabase, path)
2. Refresh the map service
 updateArcGISServer(server, services, _ZIP,
_ZIPDESTINATION)
3. Notify dependent subscription workspace
 NotifySuccess()

11. Shutdown Script: 1. Package the Output
 Zip the file geodatabase

12. Shutdown Script: 2. Refresh Map Service
1. Stop map service
2. Delete current fgdb
1
3. Copy to shared 2
network drive
3
4. Extract new fgdb
4
5. Start map service
5

13. Refresh Map Service: Stop/Start Services
 Stopping and starting AGS map service

14. Refresh Map Service: AGS Token Access
 Generate token for admin access

15. Refresh Map Service: Replace Fgdb
 Delete existing fgdb and extract zip

16. Shutdown Script: 3. Notify Subscription

17. FME Server

18. FME Notification Service - Topics
 Create a Success and a Fail notification topic to perform some action
based on the data refresh workspace execution status

19. FME Notification Service - Subscriptions
 2 subscribers
 Email service to notify when the workspace fails
 Metadata workspace runs if data refresh workspace completes successfully

20. Metadata about Map Service
 FGDC – Compliant metadata required in XML format
 XML file updated for date of fgdb refresh
https://edg.epa.gov/

21. Subscriber: Metadata Refresh Workspace

22. EPA Future Plans for FME
 Kick off workspace from Oracle instead of manually in FME Server’s
Services interface – good for more frequent refresh cycles than monthly
 using FME’s Job Submitter service
http://servername.com/fmejobsubmitter/Samples/workspacename.fmw?FMESer
verHost=http://servername.com&SuccessTopic=InternetFGDB&Token=4fab5b76
c6yufed2c4731s3d0387bfb366569949&FailTopic=InternetFGDBFail&UserName=a
dmin&Password=password&opt_showresult=false&opt_servicemode=sync
 Possibly enhance FRS Query Tools by adding spatial selections and making
more file formats available
 Integrate into other EPA program map service refresh flows
 Research FME’s ability to translate Resource Description Framework (RDF),
a semantic web data format

23. EPA Facility Registry Services (FRS)

24. Thank You!
 Questions?
 For more information:
 David Smith, EPA Facility Registry Service
Smith.DavidG@epamail.epa.gov
@DruidSmith
 Judith Doherty, FRS Project Manager,
Indus Corporation
jdoherty@induscorp.com
 Amy Ramsdell, Blue Raster
aramsdell@blueraster.com
www.blueraster.com/blog
@AmyRams