From Oracle to the Web - Automating Spatial Data Updates


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • FME REST services for database extractsUsed in front end Clip and Ship applicationOn-demand requests to services for custom appsScheduled data refreshes
  • From Oracle to the Web - Automating Spatial Data Updates

    1. 1. From Oracle to the Web -Automating Spatial Data UpdatesDavid Smith, FRS Program Manager, EPAAmy Ramsdell, GIS Manager, Blue Raster April 8
    2. 2. Overview The Need: Automate spatial data refresh The Solution: Technologies used ArcGIS Server (AGS) map service final output FME workspaces that automate refresh process  File geodatabase (fgdb) creation  Metadata refresh Python shutdown script in workspace FME Server web services used EPA future plans for FME Questions
    3. 3. The Need:Automate refresh of map service serving spatial data  Operational database system  The Facility Registry Service (FRS) is a centrally managed database that identifies facilities, sites or places subject to environmental regulations or of environmental interest.  Disconnected static file for map service  Fgdb faster performance in AGS  Lessen activity on operational database  Monthly manual refresh
    4. 4. The Solution:
    5. 5. ESRI ArcGIS Server Final Output Web server hosting GIS web services REST endpoint for map service /FRS_INTERESTS/MapServer Underlying data: file gdb  28 layers (tables in fgdb)
    6. 6. Data Refresh FME Workspace Oracle Spatial reader ESRI File Geodatabase writer Whole bunch of filtering in-between to create separate tables in fgdb  Could build the filtering in Oracle before it reaches FME
    7. 7. Oracle Spatial Type ReaderRequired for spatial object: Table, materialized view, or view  Shape field is SDO_GEOMETRY CREATE TYPE sdo_geometry AS OBJECT ( SDO_GTYPE NUMBER, SDO_SRID NUMBER, SDO_POINT SDO_POINT_TYPE, SDO_ELEM_INFO SDO_ELEM_INFO_ARRAY, SDO_ORDINATES SDO_ORDINATE_ARRAY); USER_SDO_GEOM_METADATA table entry  Metadata for spatial tables owned by schema
    8. 8. ESRI File Geodatabase API Writer Non-ArcObjects access to fgdb  C++ API Does not require an ArcGIS license on FME Server or Desktop Works in ArcGIS Server 10x
    9. 9. Python Shutdown Script in Workspace In Workspace Parameters – Advanced option
    10. 10. Shutdown Script in 3 Steps1. Package the output  zipfgdb(geodatabase, path)2. Refresh the map service  updateArcGISServer(server, services, _ZIP, _ZIPDESTINATION)3. Notify dependent subscription workspace  NotifySuccess()
    11. 11. Shutdown Script: 1. Package the Output Zip the file geodatabase
    12. 12. Shutdown Script: 2. Refresh Map Service1. Stop map service2. Delete current fgdb 13. Copy to shared 2 network drive 34. Extract new fgdb 45. Start map service 5
    13. 13. Refresh Map Service: Stop/Start Services  Stopping and starting AGS map service
    14. 14. Refresh Map Service: AGS Token Access Generate token for admin access
    15. 15. Refresh Map Service: Replace Fgdb Delete existing fgdb and extract zip
    16. 16. Shutdown Script: 3. Notify Subscription
    17. 17. FME Server
    18. 18. FME Notification Service - Topics Create a Success and a Fail notification topic to perform some action based on the data refresh workspace execution status
    19. 19. FME Notification Service - Subscriptions 2 subscribers  Email service to notify when the workspace fails  Metadata workspace runs if data refresh workspace completes successfully
    20. 20. Metadata about Map Service FGDC – Compliant metadata required in XML format XML file updated for date of fgdb refresh
    21. 21. Subscriber: Metadata Refresh Workspace
    22. 22. EPA Future Plans for FME Kick off workspace from Oracle instead of manually in FME Server’s Services interface – good for more frequent refresh cycles than monthly  using FME’s Job Submitter service verHost= c6yufed2c4731s3d0387bfb366569949&FailTopic=InternetFGDBFail&UserName=a dmin&Password=password&opt_showresult=false&opt_servicemode=sync Possibly enhance FRS Query Tools by adding spatial selections and making more file formats available Integrate into other EPA program map service refresh flows Research FME’s ability to translate Resource Description Framework (RDF), a semantic web data format
    23. 23. EPA Facility Registry Services (FRS)
    24. 24. Thank You! Questions? For more information:  David Smith, EPA Facility Registry Service @DruidSmith  Judith Doherty, FRS Project Manager, Indus Corporation  Amy Ramsdell, Blue Raster @AmyRams