Mule 4 Engine -Reactive Programming -Non Blocking -Thread Management -Streaming

1. 29th Jun,2019
Hyderabad MuleSoft Meetup Group

2. All contents © MuleSoft Inc.
Agenda
2
• 15:30 Introduction
• 15:45 Session 1
• 16:45 Break for Snacks
• 17:05 Session 2
• 17:40 Q&A

3. Introductions

4. All contents © MuleSoft Inc.
Introductions
4
About the organizer/speaker :
– Ravishankar Mishra – Integration Architect , Accenture
– Satwinder Singh Banipal - Integration Architect, Accenture
Partner:
• Consulting services
• Professional Services

5. All contents © MuleSoft Inc.
Introductions
5
Introduce yourselves !
– Name
– Background
• What’s your experience with Anypoint platform/ Mule ESB?
• Why are you here ? Expectations?

6. New Foundation of Mule 4

7. All contents © MuleSoft Inc.
New Foundation of Mule 4
7
• What’s In-Scope
-Reactive Programming Model
-Non Blocking Operations
-Thread Management in Mule 4.x
-Streaming in Mule 4.x

8. All contents © MuleSoft Inc.
New Foundation of Mule 4
8
• We build integration solutions which cater to high volumes of real-time and batch
data
– Connectivity, Transformation, Enrichment, Validation, Routing, Error Handling, Security
• Horizontal Scalability: High rates of API calls per second(More workers)
• Vertical Scalability :Memory intensive processing (More vCore)
• In Mule 4 vertical scaling is done differently using reactive programming. The aim
is to maximize throughput and higher concurrency using non-blocking operations.
This leads to more efficient use of CPU, memory and disk space.

9. All contents © MuleSoft Inc.
New Foundation of Mule 4
9
• Non Blocking Operations
-Fundamental to Reactive Programming (eventually Mule 4)
-No Thread waits for any operation
-Increases Concurrency however Latency is also more
-Manual and E toll on highways

10. All contents © MuleSoft Inc.
New Foundation of Mule 4
10
• Reactive Programming
-Appetite for fast and responsive app (Picture Link)
-Rx in .NET==JavaRx ==Reactor in Spring==Heart of Mule 4
-Merging Iterator and Observer Design Patterns
• Giving consumer the power when to consume the data
• Giving publisher the power when to push data
-Back Pressure
• Chain of (mostly) non blocking pub/sub pairs
• Subscriber to signal to producer “slow down”
-Automatic Back Pressure (No free threads available signal)
• HTTP Listener 503-”Server Busy”
• JMS Listener would not acknowledge the message
• Avoid Out of Memory errors
-Manual Back Pressure
• maxConcurrency feature (Controls number of events)

11. All contents © MuleSoft Inc.
New Foundation of Mule 4
11
• Thread Management
-Mule 3 flow processing Strategies
• Sync (Default)
• Thread-per-processor(each processor in sequential different thread)
• Qd-thread-per-processor(Induce a SEDA Q in previous),
• QdAsync (Induce Q so no distribution across nodes)
• Non-blocking (Picture Link)
-One flow is getting executed by multiple threads in Mule 4 (Picture link)
-Typical thread switching (Picture Link)
-Transactional Scenario(Picture Link)
-Transactional with Async (Picture Link)
-Try Scope with transaction (Picture Link)

12. All contents © MuleSoft Inc.
Streaming: To process larger-than-memory Mule events, they are data structures that helps to process
large data objects such as files, documents, and records by processing data continuously as it arrives.
- Does not hold all the values in memory for processing unlike In-memory data structures
- Avoids Out-Of-Memory while processing large data objects
Mule 4 runtime by default streams large data payloads without any explicit configuration.
All the Mule4 components that returns an InputStream or a Streamable collections support repeatable
streams
Some Connectors that support repeatable streams:
-File
-FTP
-DB
-HTTP
-Sockets
-Salesforce
New Foundation of Mule 4

13. All contents © MuleSoft Inc.
New Foundation of Mule 4
13
• Streaming options in Mule 4
- File based Repeatable Streams
• Only available for Enterprise Edition
• By Default store 500 objects in its In-Memory buffer and can be configured further
• Excess objects get written to disk via Kyro Serializer
- In-Memory Repeatable Streams
• Default for Mule Community Edition
• Uses default max buffer size of 500 objects
• Increment size is 100 Objects until it reaches max buffer size
• In case stream exceed max buffer limit, then application fails
-Non Repeatable Streams
• The input stream is only Read Once
• No extra memory/performance overhead compared with repeatable streams

14. All contents © MuleSoft Inc.
New Foundation of Mule 4
14
• Types of Repeatable Streams
Binary Streaming: Stream does not understand its data structure for modules like
HTTP, SFTP, and File.(Picture Link)
Object Streaming: Generally used for operations which support paging. Eg: Database,
Salesforce select operations.(Picture Link)

15. Thank You!!!

16. Open discussion

17. All contents © MuleSoft Inc.
Pic-1 Non Blocking
17
• Back

18. All contents © MuleSoft Inc.
Pic-2 Thread Pools
18
• Back

19. All contents © MuleSoft Inc.
Pic-3 Typical Thread Switching
19
• Back

20. All contents © MuleSoft Inc.
Pic-4 Transactional with JMS
20
• Back

21. All contents © MuleSoft Inc.
Pic-5 Transactional with Async
21
• Back

22. All contents © MuleSoft Inc.
Pic-6 Transactional with Try scope
22
• Back

23. All contents © MuleSoft Inc.
Pic-7 Pub-Sub model
23
• Back

24. All contents © MuleSoft Inc.
Pic-8 Binary Streaming
24
• Back

25. All contents © MuleSoft Inc.
Pic-9 Object Streaming
25
• Back