Creating an Automated Email Based Data Fulfillment System using FME Server


Published on

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Email requests from authorized addresses automatically processedPython transformers to parse email & convert to PostGISGeoSQLEmail respones w/ FTP links to ZIPped file resultsFME Server workspaces accept email requests & manage delivery process
  • From email address to which results or errors will be sentSubject identifies serviceStandard WhiteStar email address for automated requestsSubject identifies specific service (Grid Quote & Grid Order to start, others to follow)Body of email contains request attributes in published standardized format
  • Several workspaces w/ Python transformersPython library for authorization, parsing, translation to GeoSQL (interchangeable modules)PythonCallers call into library as neededIssues & errors in response emailsLibrary unit test drivers run on UNIX/OSX command line independently of FMEFME devenv: FME Desktop on Vmware Fusion under OSX, DropboxPipeable command-line interfaces planned
  • All services & data sources deployed to AWS cloudEmail forwarded to FME Server EC2 instance (Safe AMI)PostGIS on another EC2 instanceGoogle Spreadsheet for known email address auth (perms for quotes, polygon counts, suspended status)Salesforce Batch API w/FME writer/reader for quote/order submission & doc generationFuture web server cluster (w/ ELB) for ease-of-submission UI & possible API
  • Creating an Automated Email Based Data Fulfillment System using FME Server

    1. 1. Automated Email ServicesRobert White – President WhiteStar CorpKevin Weller – Software Architect, ASAP iSystems April 10, 2013
    2. 2. Project Background Business Environment  WhiteStar Delivers Vector Data to Customers  Public Land Survey (polygons).  Texas Land Survey (polygons).  Well Data (points).  Culture (vectors, polygons, points).
    3. 3. Project Background Old Way.  Manual Delivery.  DVD.  Desktop Installation.  Stressing Out Jeff and Mike.
    4. 4. Project Background New Way  Simple e-mail Interface.  Customers Self-Serve Data.  Common Process for Enterprise and Desktop.  Based on Cloud Technologies.  FME Server  PostgreSQL  ArcGIS Online?  Mike and Jeff are Happy
    5. 5. Benefits Automates Internal Manual Tasks. Automates Data Quoting. Tracks Data Usage. Repurposes Resources. Entitlements Managed by Google Docs Interface. Interfaces to Eliminates Need for Web.  Web Tools Development to Hide Complexity.
    6. 6. Features Leverage Strengths of FME:  Repeatable, Simple Ordering Process.  Leverage Predefined Output Formats.  Use of .JPG and .PDF for Quote Generation  Geodatabaseetc, for delivery.  Leverage Power of Workspaces.  Leverage Built-in Coordinate Systems / Transforms.  Store Data in Cloud.
    7. 7. Basic Data Flow
    8. 8. Email ExampleFrom: kweller@asapisoft.comTo: wsdata@fme.whitestar.comSubject: WhiteStarGrid OrderCOUNTY: CO:JeffersonLAYERS: PLSSCOORDSYS: EPSG:4267FMT: shapefile
    9. 9. Subsystem Decomposition
    10. 10. Cloud Deployment
    11. 11. Thank You! Questions? For more information:  Robert White,  Kevin Weller,  ASAP iSystemsfor WhiteStar Corporation  Twitter: @doxotter (Kevin) and @k0rcw (Robert)