Slides for the Denver (Virtual) MuleSoft Meetup Group - Data Synchronization and Replication - What's Batch Got to Do with it?

1. February 24, 2021
Denver (Virtual) MuleSoft Meetup Group
Data Synchronization and Replication -
What’s Batch Got to Do with It

2. 2
● 6:30 - 6:35 - Attendees join Meetup
● 6:35 - 6:40 - Introductions
● 6:40 - 7:50 - What’s Batch Got to Do with It
● 7:50 - 8:00 - Wrap up and next Meetup
● 8:00 - Meetup concludes
Agenda

3. 3
●About Big Compass:
○ Stephanie Lawrence
○ Linda Gunn
○ Aaron Lieberman
○ Brian Statkevicus
Introductions
Use Chat - tell us where you are joining us from

4. 4
● Use Chat
● One answer per person per question
○ If you answer more than 1x per question, you are ineligible for that question
● If you’ve already won during today’s Meetup, you cannot win again
● Big Compass makes the final decision on the trivia winners that will be submitted to MuleSoft
(see next slide)
Trivia Questions - “House Rules”
More important Trivia guidelines on next slide

5. 5
● Big Compass submits trivia winners (name plus email) to MuleSoft Meetups team
● MuleSoft will verify:
○ You haven’t already won this month
○ You’re not trying other means to circumvent the 1x/month rule
● MuleSoft will send you the voucher within 10 days of Big Compass notifying MuleSoft
○ The voucher is a training voucher, not a standalone exam voucher
○ MuleSoft retains the final decision on determining your eligibility to receive a training voucher
● If you have trouble redeeming a voucher, open a ticket at training.mulesoft.com/question
Trivia Questions - MuleSoft Guidelines

6. 6
● Developer at Big Compass for the last year
● MuleSoft developer for a year and a half
● Avid board gamer and rock climber
● First time presenter for MuleSoft Meetups
About Me

7. What’s Batch Got to Do with It

8. 8
● Data sync vs data replication
● Batch Job deep dive
● Demo time
Agenda

9. Data Synchronization vs
Data Replication

10. 10
● Synchronization
○ Ensuring multiple systems have identical data
○ Takes different forms in different system hierarchies
○ Often real-time and event driven
○ Smaller data size
● Replication
○ Loading data from one system to another
■ I.e., loading from a backup, standing up a new system
○ Scheduled or manually triggered
○ Large amounts of data
Definitions

11. 11
● Synchronization
○ Typical MuleSoft use case
○ Event driven APIs are straightforward to build
●Replication
○ Not typically associated with MuleSoft
○ Requires long running, heavy ETL operations
○ Can be achieved with the Batch Scope!
Using MuleSoft

12. The Batch Job

13. Batch Basics
● Parallelized asynchronous processing handled transparently

14. Anatomy of a Batch Job
1. Load and Dispatch - implicit, behind the scenes setup
1. Process - actual execution of the job
1. On Complete - optional metric aggregation

15. Which of the following is true in a Batch Job with two Batch Steps?
Trivia Question #1
a) Every record starts step one at the same time
b) Different records can be in different steps at the same time
c) After a record finishes step one, it immediately starts step two
d) All records will finish step one before any records finish step two

16. Batch Aggregation
● Aggregate subset of records within one Batch Step
● Choose between fixed size and streaming
○ Fixed size:
■ Random access
■ Choose block size to optimize memory usage
○ Streaming:
■ Can aggregate every record in an entire job instance
■ Sequential access
■ Performance hit
● Transactions within a Batch Step end before the aggregator

17. Which of these is a reason to use a streaming aggregator instead of a fixed-size aggregator?
Trivia Question #2
a) You want to sum a value across all your records
b) You are loading data into Salesforce
c) You want to access multiple records at once in your aggregation
d) You want to optimize performance of the Batch Job

18. Variable Propagation
● Each record has its own set of variables that are propagated with it
● Think of records as self-contained units
● In aggregators, each record still has its own version of each variable
● Variable modifications or creations are not visible after completion

19. Error Handling
● Errors are handled at a record level
○ By default, records that fail one step are not processed through subsequent Batch Steps
○ Specific errors are only logged once per step
○ Error history is accessible by DataWeave per record
● Create reports in the On Complete phase
● Use filters to refine your batch processing
○ AcceptPolicy - NO_FAILURES, ONLY_FAILURES, or ALL
○ AcceptExpression - specific validations using DataWeave

20. Demo Time

21. What values does the logger in the On Complete phase print?
Trivia Question #3
A. Counter: 1, stepVar: null
B. Counter: 1, stepVar: 31
C. Counter: 31, stepVar:
null
D. Counter: 31, stepVar:
31

22. 22
● Share:
○ Tweet using the hashtag #MuleSoftMeetups
○ Invite your network to join: https://meetups.mulesoft.com/denver
● Feedback:
○ Complete the Meetup survey, provide feedback, and suggest topics for upcoming events
○ Contact MuleSoft at meetups@mulesoft.com for ways to improve the program
What’s next?

23. 23
● Next (Virtual) Meetup: Mar 31, 2021 @ 6:30p US Mountain Time
○ Topic: Delayed Message Reprocessing
● Future Meetup (tentative) schedule:
○ May 19 (virtual)
○ July 28 (Site TBD based on local guidance)
○ Sep 22 (Site TBD based on local guidance)
○ Nov 10 (Site TBD based on local guidance)
● We are looking for topic suggestions:
○ DataWeave (of course!)
○ Other suggestions:
■ Flow Designer revisited
■ Error Handling
■ Logging revisited
■ Runtime Fabric
■ Others???
● We’re also looking for speakers - please contact us if you are
interested
Denver Meetups 2021

24. Thank you