With more traffic than ever being generated through system to system connections, building robust integrations is a must. Keys to any complete integration include (but not limited to) zero data loss, queuing, decoupling application components, synchronous vs. asynchronous executions, throttling, logging and monitoring, and caching. This discussion will dive into many of these key components of integration. The presentation will focus on eliminating data loss with best practice integration patterns, and will demo and show methods to ensure that data is never lost. The discussion will also explore decoupling components of an application into reusable microservices, caching for APIs, and ideas on how you can accomplish synchronous and asynchronous executions, all while maintaining high visibility into your platform with logging and monitoring.

1. NEVER LOSE DATA AGAIN: ROBUST INTEGRATIONS WITH
MULESOFT
June 3, 2020
Denver MuleSoft Meetup Group

2. All contents © MuleSoft Inc.
Agenda
2
• 6:30PM - 6:35PM – Introductions
• 6:35PM - 7:30PM – Presentation/Demo
• 7:30PM - 7:45PM – Q&A
• 7:45PM - 8:00PM – Open Floor, Suggestions for Future Topics and
Speakers

3. All contents © MuleSoft Inc.
Introductions
3
• About the organizer:
– Big Compass
• About the presenter:
– Aaron Lieberman

4. • Asynchronous vs. Synchronous Considerations
• Caching
• Queuing
• Decoupling Application Components
(Microservices)
• Throttling
• Logging And Monitoring Across Decoupled
Microservices
MuleSoft Integration Best
Practices

5. All contents © MuleSoft Inc.
Integration Pain Points
5
• System visibility
• System traceability
• Supporting your integrations
• Message loss
• Scaling
• High availability
• Flexibility

6. What is the difference between Asynchronous and
Synchronous executions?
Giveaway!

7. Asynchronous vs.
Synchronous

8. All contents © MuleSoft Inc.
Asynchronous Vs. Synchronous Considerations
8
Consideration Explanation
What is async? Do not wait for workload to finish processing
What is sync? Wait for workload to finish processing
Async synonyms 1. Fire and forget
2. Non-blocking
Sync synonyms 1. Finish and move on
2. Blocking

9. Synchronous

10. All contents © MuleSoft Inc.
Synchronous Considerations
10
Consideration Explanation
What is Synchronous? Wait for workload to finish processing
Ways to Invoke
Synchronously
1. API request/response
2. Invoke via SDK and wait for response
3. Invoke via connector and wait to finish
4. Invoke flow and wait to finish
When to Invoke
Synchronously
1. GET requests requiring data to be returned
2. Response is required before moving on
3. Quick responses
When It Is Not a Good Idea
To Invoke Synchronously
1. Workloads that can process in the
background
2. Long-running processes
Benefits of Synchronous 1. Know right away if something
succeeded/failed
2. Can respond to clients without clients using
another method to lookup results

11. All contents © MuleSoft Inc.
Synchronous Data Flow
11

12. All contents © MuleSoft Inc.
Demo Data Flow
12

13. Demo
Synchronous API

14. All contents © MuleSoft Inc.
Caching
14
Consideration Explanation
What is Caching? Maintain frequently accessed data in memory or in
an external store (Object Store V2) made for quick
access
Ways to Cache 1. API Manager
2. Caching Scope
When to Cache 1. Frequently accessed data
2. GET requests
3. Static responses
4. Overworked resources
When It Is Not a
Good Idea To Cache
1. Updates are frequently made to resources
Benefits of Caching 1. Faster responses
2. Better UX
3. Less strain on downstream systems
4. Hide network failures

15. All contents © MuleSoft Inc.
Caching Data Flow
15

16. Demo
Caching

17. Asynchronous

18. All contents © MuleSoft Inc.
Asynchronous Considerations
18
Consideration Explanation
What is Asynchronous? Do not wait for workload to finish processing
Ways to Invoke
Asynchronously
1. Utilizing queues/topics
2. Fire and forget
3. Async scope
When to Invoke
Asynchronously
1. POST/PUT/DELETE requests not requiring
data to be returned
2. Response is not required before moving on
3. Slower processes that can run in the
background
When It Is Not a Good Idea
To Invoke Asynchronously
1. Clients expecting immediate response
2. Short-running processes
Benefits of Asynchronous 1. Allows the system to process high volumes
of data in parallel
2. Utilize concurrency and application
horsepower effectively

19. All contents © MuleSoft Inc.
Asynchronous Data Flow
19

20. Demo
Asynchronous API

21. All contents © MuleSoft Inc.
Queuing Considerations
21
Consideration Explanation
What is Queuing? Park messages in a message “parking lot”
Ways to Queue Messages 1. Queue providers such as Anypoint MQ or
AWS SQS
2. Various queuing connectors provided by
MuleSoft such as VM Queue
When to Queue Messages 1. Asynchronous invocations
2. Metering and throttling before a downstream
system
3. Checkpoints to guarantee delivery
When It Is Not a Good Idea
To Queue Messages
1. Synchronous invocations
2. Live event streaming
Benefits of Queuing
Messages
1. ZERO DATA LOSS - Ensures each part of a
system can reliably deliver a message
2. Achieve reliability pattern
3. Decouple application components

22. All contents © MuleSoft Inc.
Queuing Data Flow
22

23. All contents © MuleSoft Inc.
Queuing Example
23

24. All contents © MuleSoft Inc.
Decoupling Considerations
24
Consideration Explanation
What is Decoupling? Keeping parts of a system separate and isolated so
one component does not rely on another
Ways to Decouple 1. Logically split application components into
microservices
2. Use queues in between application components
When to Decouple 1. Monolithic applications
2. Failures occur that should not affect other areas of
the system
3. Too many vCores are needed to maintain one
application
4. When scaling one application is difficult
When It Is Not a Good Idea
To Decouple
1. When a microservice is small enough
2. A microservice already performs one function well
Benefits of Decoupling 1. Failures/errors are isolated
2. Fine tune each component of a system

25. All contents © MuleSoft Inc.
Decoupling A System
25

26. All contents © MuleSoft Inc.
Decoupling A System With Messaging
26

27. All contents © MuleSoft Inc.
Decoupling Example
27

28. Name 2 use cases for throttling.
Giveaway!

29. All contents © MuleSoft Inc.
Throttling Considerations
29
Consideration Explanation
What is Throttling? Limiting the number of concurrent events at any one
time
Ways to Throttle 1. Limit max concurrency on a flow in Mule 4
2. Use a processing strategy in Mule 3
3. Use queues
4. Use holding locations like a database
When to Throttle 1. Potential to flood downstream system
2. Delay processing of data
3. Control application utilization
When It Is Not a Good Idea
To Throttle
1. Your system cannot process data fast enough
2. Downstream systems can handle your volume
Benefits of Throttling 1. Control of concurrent events
2. Protection against system flooding

30. All contents © MuleSoft Inc.
Throttling Data Flow
30

31. Demo
Queuing, Decoupling, and Throttling

32. All contents © MuleSoft Inc.
Logging and Monitoring Considerations
32
Consideration Explanation
What is Logging? Exposing system events to gain visibility and trace
events in the system
Ways to Log and Monitor 1. External system such as ELK or Splunk
2. CloudHub logging
3. Logging to a file
4. Logging to a database
When to Log and Monitor 1. At logical events within the system
2. When the event will tell a part of a story
When It Is Not a Good Idea
To Log
1. Too much logging occurs – not user friendly
2. Pertinent information is not captured
Benefits of Logging and
Monitoring
1. Increased visibility into the system
2. Trace transactions from end to end
3. Decreased support cost

33. All contents © MuleSoft Inc.
Logging and Monitoring Data Flow
33

34. Demo
Logging and Monitoring

35. Name 2 use cases for a synchronous API and 2
use cases for an asynchronous API.
Giveaway!

36. All contents © MuleSoft Inc.
Synchronous vs. Asynchronous Summary
36

37. All contents © MuleSoft Inc.
References and Documentation
37
• Reliability pattern
– https://docs.mulesoft.com/mule-runtime/4.3/reliability-patterns
• Caching
– Cache scope
• https://docs.mulesoft.com/mule-runtime/4.3/cache-scope
– Caching policy
• https://help.mulesoft.com/s/article/How-to-configure-HTTP-Caching-Policy-for-a-specific-HTTP-
resource-in-API
• Log4j2
– https://docs.mulesoft.com/mule-runtime/4.3/logging-in-mule
• Throttling
– https://docs.mulesoft.com/api-manager/2.x/rate-limiting-and-throttling
• MuleSoft Anypoint MQ
– https://docs.mulesoft.com/mq/mq-queues

38. Questions?

39. All contents © MuleSoft Inc.
What’s Next?
39
• Share:
– Tweet your pictures with the hashtag #MuleMeetup
– Invite your network to join: https://meetups.mulesoft.com/denver/
• Feedback:
– Contact your organizer aaron@bigcompass.com or linda.gunn@bigcompass.com
to suggest topics
– Contact MuleSoft at meetup@mulesoft.com for ways to improve the program
• Our next meetup:
– Date: July 2020
– Location: TBD
– Topic: TBD

40. See you next time
Please send topic suggestions to the organizer