Presentation by GIM at the FME World Tour 2019 in Leuven

1. Real-time Location
Tracking with FME
Server Automations
Stijn Goedertier, GIM
#FMEWT Leuven, 23 May 2019

2. FME Server
Run FME workspace on a server to
automate data integration:
● Set up schedules and notifications
● Deliver self-service data access
● Set up data collection apps
● Receive messages and trigger actions

3. Concepts in this talk
● FME Server stream processing:
process a stream of messages when
they become available
● FME Server Automations: respond to
events and trigger actions

4. Demo: bike-to-work incentives
In Flanders: a tax-free benefit of max. 0,23€ per kilometer
● Capture real-time location data from mobile devices
● Detect arrivals at home / in the office
● Map match track and calculate benefits
● Send encouraging push message

5. Demo: bike-to-work incentives
MQTT (Message Queuing Telemetry Transport): IoT messaging protocol
JMS (Java Message Service: Java messaging standard )
Use MQTT to exchange messages with Internet of Things devices!
Mobile
App
MQTT
MQTT
JMS WebSockets

6. Challenge
“We want to process a
continuous stream of
thousands of messages
per second.”

7. Continually send and receive messages
•TCPIPSender
•TCPIPReceiver
•JMSSender
•JMSReceiver
•SQSSender
•SQSReceiver
•WebSocketReceiver
•WebSocketSender
•PythonCreator
•PythonCaller
•…

8. Demo: receiving MQQT message streams

9. Demo: calculate distance between adjacent features

10. Demo: detect event and send JMS message

11. Demo: what we have so far
● Sparse points
with
timestamps
● Arrival events
● BUT: traveled
distance is not
accurate
enough

12. Demo: what we need: map matching
● Map
matching:
derive the
most
plausible
route
● Using:
Graphhopper
and
OpenStreet-
Map

13. Challenge
“We want to watch for data
and run an FME task
whenever new data arrives.”

14. AutomationsA simple concept consisting of Triggers and Actions

15. Watching
for Data
Trigger when data arrives.
Amazon S3 Watch
Triggers when data arrives in
location in S3 Bucket
Directory Watch
Triggers when data arrives in File
Directory.
FTP Watch
Triggers when data arrives at
location on ftp site
Dropbox Watch
Triggers when data arrives in
Dropbox Directory.
Azure Event Grid
Can Trigger when data arrives in
Azure Storage

16. Actions
Respond to triggers
Internal
External

17. When
Trigger Occurs
then
Perform Action
Every Automation
has the same pattern:

18. Demo: on arrival (JMS message), start workspace

19. Demo: Map matching and benefit calculation

20. Demo: Map matching and benefit calculation

21. Demo: Map matching and benefit calculation

22. Demo: Map matching and benefit calculation

23. Demo: Map matching and benefit calculation

24. Demo: Map matching and benefit calculation

25. Demo: Query FME Server over WebSockets

26. Demo: Query FME Server over WebSockets

27. Conclusion: message/event-driven data
integration
● FME Server stream processing: process a stream of
messages when they become available. Thousands of
messages per second. A workspace keeps running.
● FME Server Automations: respond to events and trigger
actions. Orchestrate workspaces to run in parallel or
sequentially.

28. THANK YOU!
stijn.goedertier@gim.be