This document discusses several patterns for organizing FME Server workflows. The patterns include: - Submitter Iterator with Status, which selects parts of a dataset to process independently on different engines while tracking status. - Runner/Controller Workspace, which uses FMEServerJobSubmitter for workflow control and allows running the controller on a desktop. - Dataset Tool Switch, which synchronously processes the same dataset with different modes of operation through successive refinement. - Flatten, Explode, Export, which involves three stages - flattening data, unpacking it, and exporting to different formats - to manage master data.

1. FME Server Workspace
Patterns - Continued
Reed Whittington
Burns & McDonnell

2. About Us
FME Certified Pro Services
Burns & McDonnell, Dallas, TX
Reed Whittington
Burns & McDonnell is a full-service engineering,
architecture, construction, environmental and
consulting solutions firm.
Our multidiscipline staff of more than 5,000
employee-owners includes engineers, architects,
construction experts, planners, estimators,
economists, technicians and scientists, representing
virtually all design disciplines.

3. Business Technology Services
Burns and McDonnell Technology Consulting Services is a virtual team of technical experts
around the globe offering a wide array of services and deliverables including:
• Web Sites/Information Management Portals/Doc Management solutions
• 2d/Web Maps/3D/4D Globe Applications
• Field Data Collection/Inspections/Real-Time Monitoring
• CAD/GIS/Database Integration
• Automated Report Generation and Distribution
• Project Metrics/Analytics/KPI Dashboards
Integrated, field tested, project proven solutions
Many of these services are bundled under OneTouchPM which is a hybrid cloud based, web
enabled, managed service allowing project managers to engage their customers and share
information, summary progressing with aggregated executive views from all disciplines.

4. In software engineering, a
software design pattern is a
general reusable solution to a
commonly occurring problem
within a given context in software
design. It is a description or
template for how to solve a
problem that can be used in many
different situations. Not a specific
implementation; design patterns
are formalized best practices that
the programmer can use to solve
common problems when designing
an application or system.
https://en.wikipedia.org/wiki/Software_design_pattern

5. 4 FME Server Workspace Patterns
• Workflow Submitter Iterator (or Work Status)
• Strategy for Asynchronous processing of large datasets
• Runner / Controller Workspace (or Tool Chain)
• General approach using FMEServerJobSubmitter for workflow
control
• Dataset Tool Switch (or Multi-Pass)
• Strategy for Synchronous Sequential processing
• Operates on same dataset with different modes of operation
• Flatten, Explode, Export (or Import/Merge/Publish)
• 3 stages of processing typically used for master data
management

6. Submitter Iterator with Status
• Uses a selection strategy to select mutually exclusive
parts of a source data set so that the parts can be
operated on independently and run on different fme
engines.
• Used to process files or db operations so that sql select
statements can be used.
• Needs a better name
• Work Status, Delegator, …
• This was the subject of a previous presentation.
• http://cdn.safe.com/events/FME2011WorldTour/Austin_Reed_Whittington.pdf

7. Example using Path reader to Iterate and track status.
can also use url as path or list of SQL queries
Submitter Iterator with Status
Wait=No

8. Runner / Controller Workspace
• General strategy for Job Control and Tool Chain
• Lets you run the controller on desktop
• FME 2015 improves FMEServerJobsubmitter:
• In Engine Sequential Run with Wait=Yes
• Output datasets to temp files or as specified
Excellent Article by DeWet@Safe:
https://knowledge.safe.com/articles/1439/fme-server-workflow-management-1.html

9. Dataset Tool Switch (aka multi-pass)
• The “Multi-pass" pattern presents a common interface
to iterate over datasets using successive refinement.
• Meant to be run multiple times with different RUN_MODEs
• RUN_MODE Choice defines the Switch behavior
• TestFilter on RUN_MODE controls Feature RoutingUseful for
processing directory/tree of files and/or database exports or
publishing.
• Interface includes:
• SourceDir, StagingDir, Run Modes, DB connection, Publish
location

10. Use the Path reader to Iterate and Route based
on Run Mode

11. Flatten, Explode, Export
• 3 separate workspaces that act together but run asynchronous
• Each may use “multi-pass”
• Flatten
• Denormalization into one big table
• Pack attributes into dictionaries
• Think NoSQL and Operational Data Store
• Explode
• Unpacks big table
• Data model mapping and normalization happens here
• May use templates for standard data models
• Might also use Dynamic Schema

12. Flatten, Explode, Export (cont)
• Export
• Database to files (.kmz, .csv, .json, .gdb, …)
• Heavy use of Fanout (ds and ft)
• May use Tool Switch RUN_MODE to define export
type or staging/publish processing steps

13. More possibilities with FME 2016…
What do you use?
Tool Chain Workspace
• FeatureReader
– Selective reading/caching
– Error Checking
• FeatureWriter
– Write then read back in
same .fmw
– In workspace tool chain
– Removes the need for temp
output files of the
FMEServerJobSubmitter
Before FME 2016
• Required chaining
FMEJobSubmitters
together
• Next in chain reads
output of Previous

14. Main point: FME Server
Patterns help you develop and
organize your data
management tools.

15. Thank you!
Reed Whittington
FME Certified Pro Services
Burns & McDonnell
Dallas, TX
rwhittington@burnsmcd.com