MuleSoft Surat Meetup Group
Deep Dive into MuleSoft Batch
Processing (ELT/ETL/Batch
Processing)
Date – 27th Nov 2021
Time – 11:00 IST (GMT+05:30)
Safe Harbour Statement
● Both the speaker and the host are organizing this meet-up in individual capacity only. We are
not representing our companies here.
● This presentation is strictly for learning purposes only. Organizer/Presenter do not hold any
responsibility that same solution will work for your business requirements.
● This presentation is not meant for any promotional activities.
2
A recording of this meetup will be uploaded to events page within 24 hours.
Questions can be submitted/asked at any time in the Chat/Questions & Answers Tab.
Make it more Interactive!!!
Give us feedback! Rate this meetup session by filling feedback form at the end of the day.
We Love Feedbacks!!! Its Bread & Butter for Meetup.
Housekeeping
3
Organizers
4
Nitish Jain
Jitendra Bafna
Speaker
5
Chetan Joshi
Associate Solution Architect as Apisero
6
Agenda
Introduction
Batch Processing
Live Demonstration
Trivia Quiz
7
● Jobs that can run without end user interaction, or can be scheduled to run as resources permit
● Is a method of running high-volume, repetitive data jobs. The batch method allows users to process
data when computing resources are available, and with little or no user interaction.
● Is the processing of transaction in a group or batch. No user interaction is required once batch
processing is underway.
What is Batch Processing?
8
● Synchronizing data sets between business applications, such as syncing contacts between NetSuite
and Salesforce.
● Extracting, transforming and loading (ETL) information into a target system, such as uploading data
from a flat file (CSV) to Hadoop.
● Handling large quantities of incoming data from an API into a legacy system.
Why/When Batch Processing?
9
Components provided by Mule:
Batch Processing in Mule 4
10
Batch Processing in Mule 4
11
● Load and Dispatch.
● Process.
● On Complete
Batch Processing Phases
12
● Implicit (Behind the Scenes)
● Creates Batch Job Instance (can access through vars.batchJobInstanceId )
● Creates a persistent queue (prefix BSQ)
Load and Dispatch Phase
13
● Batch Step
● Async
● After processing a record in a step it moves back to Queue and state(stage) us updated.
● May create more internal queues ( when using Aggregator)
Process Phase
14
Process Phase
15
● States
On Complete Phase
16
DEMO
17
● Error Handling
● Performance Tuning
● Batch Performance
Extras
Get ready to WIN a Special Gift from MuleSoft Community
Quiz Time

MuleSoft Surat Virtual Meetup#29 - Deep Dive into MuleSoft Batch Processing (ELT/ETL/Batch Processing)

  • 1.
    MuleSoft Surat MeetupGroup Deep Dive into MuleSoft Batch Processing (ELT/ETL/Batch Processing) Date – 27th Nov 2021 Time – 11:00 IST (GMT+05:30)
  • 2.
    Safe Harbour Statement ●Both the speaker and the host are organizing this meet-up in individual capacity only. We are not representing our companies here. ● This presentation is strictly for learning purposes only. Organizer/Presenter do not hold any responsibility that same solution will work for your business requirements. ● This presentation is not meant for any promotional activities. 2
  • 3.
    A recording ofthis meetup will be uploaded to events page within 24 hours. Questions can be submitted/asked at any time in the Chat/Questions & Answers Tab. Make it more Interactive!!! Give us feedback! Rate this meetup session by filling feedback form at the end of the day. We Love Feedbacks!!! Its Bread & Butter for Meetup. Housekeeping 3
  • 4.
  • 5.
  • 6.
  • 7.
    7 ● Jobs thatcan run without end user interaction, or can be scheduled to run as resources permit ● Is a method of running high-volume, repetitive data jobs. The batch method allows users to process data when computing resources are available, and with little or no user interaction. ● Is the processing of transaction in a group or batch. No user interaction is required once batch processing is underway. What is Batch Processing?
  • 8.
    8 ● Synchronizing datasets between business applications, such as syncing contacts between NetSuite and Salesforce. ● Extracting, transforming and loading (ETL) information into a target system, such as uploading data from a flat file (CSV) to Hadoop. ● Handling large quantities of incoming data from an API into a legacy system. Why/When Batch Processing?
  • 9.
    9 Components provided byMule: Batch Processing in Mule 4
  • 10.
  • 11.
    11 ● Load andDispatch. ● Process. ● On Complete Batch Processing Phases
  • 12.
    12 ● Implicit (Behindthe Scenes) ● Creates Batch Job Instance (can access through vars.batchJobInstanceId ) ● Creates a persistent queue (prefix BSQ) Load and Dispatch Phase
  • 13.
    13 ● Batch Step ●Async ● After processing a record in a step it moves back to Queue and state(stage) us updated. ● May create more internal queues ( when using Aggregator) Process Phase
  • 14.
  • 15.
  • 16.
  • 17.
    17 ● Error Handling ●Performance Tuning ● Batch Performance Extras
  • 18.
    Get ready toWIN a Special Gift from MuleSoft Community Quiz Time