Implementing a Simple Web Application with FME Server


Published on

Presented by Logan Pugh, City of Austin

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Spell checking built into most web browsers. That was easy! We call those “quick wins” around here.
  • We still have to do the verification manually, but we can certainly speed up the rest of the process, and give the GIS operator hints on the verification step as well
  • Workspaces are published to FME Server with the Data Streaming service configured
  • Workspaces are published to FME Server with the Data Streaming service configured
  • 1. Browser requests FME Server workspace2. FME Server runs workspace, returns HTML result3. JavaScript on HTML form makes calls to FME Server REST API4. FME Server runs workspaces, returns JSON results5. JavaScript populates UI elements (Dojo widgets)6. User enters form data, submits request for PDF7. FME Server runs workspace, returns PDF result8. Operator sends PDF to customer after verifying all data
  • JavaScript, and by extension, JSON considered “lingua franca” of the web; useful data exchange format for web applications
  • Implementing a Simple Web Application with FME Server

    1. 1. CONNECT. TRANSFORM. AUTOMATE. Implementing a simple web application with FME Server Logan Pugh Programmer Analyst, City of Austin Tony Castro GIS Analyst, City of Austin Ben Vanderford GIS Analyst Senior, City of Austin
    2. 2. Who, what, when, where, why and how?  Who: Small GIS shop in a big (and growing) city  What: Provide zoning verifications (among other services) to property owners and developers  When: 50 to 80 requests a month  Where: City of Austin, Texas  Why: Each request used to take 1-2 hours  How: Tedious manual process of verifying zoning, hand-editing a PDF and sending it to customer
    3. 3. What is zoning verification?  Zoning in Austin started in 1930s  Zoning verifications started in 1960s  Developers and property owners need zoning verification to get financing for (re)development
    4. 4. Old Process was Cumbersome: The Fatigue Factor  Lots of research and keying in values  Name of applicant and address  Parcel ID  Index grid  Zoning type, definition and ordinance  Print, sign and mail/fax/hand-deliver letter to sales office for customer pickup
    5. 5. Old letter format New letter format
    6. 6. The Tipping Point  With an ever-increasing workload and not-so- increasing budget/staffing, we need to increase efficiencies  Our staff time stretched, a new zoning verification process became a high priority  Along comes FME Server: we had a new option!
    7. 7. Project Requirements  Accept a parcel number in various formats  Perform zoning info drilldown with complex filtering logic  Allow operator to edit/correct info  Generate PDF with electronic signature automatically  Ability to easily be configured for future changes (new users, jurisdictions, etc.)  Spell checking
    8. 8. Challenges  How to automate what is by definition a human- verified process?  How can we own the process but provide access to people outside of our workgroup?  How can we build this without any additional infrastructure?
    9. 9. FME Server Data Streaming Service  Serves up the results of workspaces directly  Can be HTML, JSON, PDF, etc.  Result must be a single file  Allows everything to work within the context of a web browser
    10. 10. FME Server REST API  Simple protocol for programmatically interacting with FME Server  Uses HTTP requests, e.g. GET, PUT, DELETE  Authentication using simple token service  Tokens are per-user, but can be shared for specific applications  FME REST API Developer Playground:
    11. 11. Workspaces  Zoning info drilldown workspace  Given a parcel number and parcel agency, returns the zoning types, ordinances and case numbers intersecting a given parcel  Negative buffer to reduce erroneous results caused by inaccuracies between zoning and parcel data  Filter out non-applicable zoning ordinances
    12. 12. Zoning Info Drilldown workspace Workspaces, cont. ZoningOrdinanceDateCalculator custom transformer ZoningOrdinanceFilter custom transformer
    13. 13. Workspaces, cont.  PDF generation workspace  Given various parameters, produces a complete, signed zoning verification letter in PDF format  CSV to JSON workspaces  Editors  Requestors  Agencies
    14. 14. Workspaces, cont. PDFTextBlockCreator custom transformerPDF Builder workspace PDFRectangleTextCreatorcustom transformer PDFImageCreatorcustom transformer
    15. 15. Workspaces, cont.  HTML form builder workspace  Serves up an HTML file that ties everything together  Uses JavaScript (Dojo Toolkit) and the FME Server REST API to request, display, edit, and submit data  Can be pre-populated with data via parameters submitted to FME Server
    16. 16. Demo
    17. 17. Behind the scenes Browser requests FME Server workspace FME Server runs workspace, returns HTML result JavaScript on HTML form makes calls to FME Server REST API FME Server runs workspaces, returns JSON results JavaScript populates UI elements (Dojo widgets) User enters form data, submits request for PDF FME Server runs workspace, returns PDF result Operator sends PDF to customer after verifying all data
    18. 18. Key Transformers  FeatureReader: Performs queries against any FME-supported format. The queries can have both a spatial and a nonspatial component.  One query is issued to the FME-supported format for each feature that enters the transformer. The results of the query are then output.  This means workspaces can read data dynamically  One of the most powerful transformers in FME!
    19. 19. Key Transformers, cont.  PythonCaller: Executes a Python script to manipulate the feature. Useful when implementing logic too complex to handle with other transformers.  Examples:  Fixed-width word-wrapping, justification and special character escaping for text elements on output PDF  Using regular expressions to replace tokens in HTML file with parameter values  Building SQL expressions and sanitizing user input
    20. 20. Key Transformers, cont.  JSONTemplater: Populates a JSON document with FME feature attribute values { "status": xs:int(fme:get-attribute("_statuscode")), "message": fme:get-attribute("_message"), "data":{ "parcel_agency": fme:get-attribute("_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] } }
    21. 21. Key Transformers, cont.
    22. 22. Key Transformers, cont. { "status": xs:int(fme:get-attribute("_statuscode")), "message": fme:get-attribute("_message"), "data":{ "parcel_agency": fme:get-attribute(“_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list- attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] } }
    23. 23. Key Transformers, cont. { "status": xs:int(fme:get-attribute("_statuscode")), "message": fme:get-attribute("_message"), "data":{ "parcel_agency": fme:get-attribute("_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list- attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] } }
    24. 24. Configuration  Enterprise applications require configurability  Avoid putting file paths, user names, passwords, etc. directly in FME workspaces  Allows workspaces to be shared with confidence, placed under source control (e.g. Git), and configured externally  Uses simple text-based configuration files
    25. 25. Configuration, cont. FME Server Repositories Workspace 1 Workspace 2 Workspace n dev Secure Network Folders Workspace 1 Workspace 2 Workspace n test Workspace 1 Workspace 2 Workspace n prod
    26. 26. Configuration, cont. importos, ConfigParser globalconfig_dict# Make variable accessible to other scripted parameters config_dict = None # Get path to config file located alongside published workspace config_path_file = os.path.join(FME_MacroValues['FME_MF_DIR'], 'config_path.txt') # Return config key/value pairs as a dictionary withopen(config_path_file, 'r')as f: config_file = f.readline().strip() config_parser = ConfigParser.RawConfigParser() config_dict = dict(config_parser.items('config')) returnconfig_file  Use Python scripted parameters to read in configuration data. In first scripted parameter:
    27. 27. Configuration, cont. # Just reference dictionary keys to get config values returnconfig_dict['fmerest_token']  Subsequent scripted parameters are easy:
    28. 28. Conclusions  Reduced processing time from 1-2 hours to ~15 minutes  Ability to email generated PDF to customer, saving on paper use and mailing expenses  Laid groundwork for use by other departments and the public  Demonstrates flexibility of FME Server
    29. 29. Future improvements  Make app directly available to public  Tie in with existing zoning profile web application  Accept payments online  Collaboration with external parcel agencies on data alignment for improved accuracy
    30. 30. Thank You!  Questions?  For more information:  Logan Pugh   City of Austin  Related web app available for public: 