Mule soft meetup_charlotte_4__draft_v2.0

All contents © MuleSoft Inc.
Welcome To MuleSoftMeetup
1

Welcome To MuleSoftMeetup
2
MuleSoft Meetup #4 Virtual
Nov 16, 2021

Agenda
3
6:30 PM Welcome & Introductions
6:45 PM Exploring ETL use cases for Salesforce as target system using
MuleSoft's Bulk API connectors and batch processing by Amresh Kosuru
7:15 PM Deep Dive into Dataweave 2.x and its Modules by Aravind Babu
Ramadugu
7:45 PM Q/A, Trivia & Event Close

Announcements
Hi Everyone, Thanks for joining our 4th Virtual Charlotte
Meetup today.
Today we are going to have talks about Exploring ETL use
cases for Salesforce and Deep Dive into Dataweave 2.x
and its Modules .
At the end, we will have Trivia session, where we will ask some
random questions related to the topics which we discussed
today.
First 3 winners will get a free training and certification voucher
from MuleSoft. Do not miss the opportunity…!
Without further ado, let’s get started.

ETL with
Salesforce Bulk API V2

Agenda
6
- ETL use cases – Extract Transform and Load in context of Salesforce.
- Salesforce APIs - a brief intro.
- MuleSoft Salesforce Bulk API V2 connector.
- Crash course on Batch processing in Mule.
- Demo
- Limits and Tuning
- Conclusion

ETL Tools - Vs - Mule ESB as an ETL tool
7
Specialized for hauling
Heavier Hauls
Costs more for gas
Need some skill with driving big vehicles
Needs wider and longer parking space
Best of both worlds
Light to moderate Hauls
Costs less for gas
Easy driving
Just the regular parking lot
Just for Fun!! – a comparison!!!

Salesforce
8
What is Salesforce?
– CRM – Customer Relationship Management
– SaaS platform for managing all things related to
Customer relationship.
– Cloud based
– An online database of business objects in context of CRM
Contact Account Opportunity Activity Campaign Lead

Use Cases – C.R.U.D
9
SQL
Files
Staging
Databases
Legacy Apps
Partnering
Apps
Full Loads
Delta Loads
Data Syncing
Insert
Upsert
Delete
Query

Salesforce Platform APIs
10
SOAP APIs
REST APIs
Streaming
APIs
APEX APIs
BULK APIs
(V2)
• A specialized Restful API for larger data loads.
• 2000 records or more.
• Asynchronous.
• V2 – easier to use than V1.
• Ideal for ETL use cases.
• Batch processing in the background.
• Automatic Optimization for best possible performance.
• Processes records, in batches, under the scope of a
Job.

Salesforce Platform APIs - Continued
11
Bulk API V2 Life Cycle

Ingest
Mule Salesforce Connector – Needed operations
12
Query

Demo
13

Agenda
15
• Introduction
• Creating Data
• Reading Data
• Variables & Logical Operators
• Flow Control
• Functions
• Working With Arrays
• Working With Objects

Introduction
16
• Dataweave is Language designed for transforming data
• MuleSoft primary language for data transformation
• Also available in other contexts like command line tool
• Every system out there wants to be special and define its own data format
• Dataweave provides a simple JSON-like functional programming language
to help quickly transform and query data.

Advantages of Dataweave
17
• Easy to write, easy to maintain, and capable of supporting
simple to complex mappings between any data types
– Out of the box support for many popular file and data types
• XML, JSON, Java, CSV, EDI, Excel, fixed-width files, flat files,
and COBOL copybook
• Often more succinct than custom code
– Do not require any intermediate Java class or annotations
• Reusable and more readable by front-end developers
– Data transformations can be stored in external DWL files
(modules) and used across applications
– Powerful functional programming using JavaScript like syntax

Dataweave mechanics
18
• Internally, Dataweave includes a connectivity layer and an
engine that is fundamentally different from other transformation
technologies.
• It contains a data access layer that indexes content and
accesses the binary directly, without costly conversions
– Enables larger than memory payloads
– Random access to input documents, even larger than memory
– Performance intensive

Creating Data
19
• Data Type:
– Strings, Booleans, Numbers, Objects, Arrays
• We can check the type of the data by using following command.
– typeOf: Will give the type associated to the sample data.
• Strings:
– Strings are defined between quotes
• Numbers
– Supports number type covering both Integers and NUmbers
• Booleans
– his is the last simple type and has values of either true or false.
• Arrays
– Arrays are an ordered series of values where the values can be of any type:
• Objects
– Essentially Key Value Pairs
– Values can be of any data type
DataWeave also supports numbers with the Number type, covering both integer and floating-point numbers.

Creating Data
20
• Data Type:
– Strings, Booleans, Numbers, Objects, Arrays
• We can check the type of the data by using following command.
– typeOf: Will give the type associated to the sample data.
• Strings:
– Strings are defined between quotes
• Numbers
– Supports number type covering both Integers and NUmbers
• Booleans
– his is the last simple type and has values of either true or false.
• Arrays
– Arrays are an ordered series of values where the values can be of any type:
• Objects
– Essentially Key Value Pairs
– Values can be of any data type

Reading Data
21
• In Last slide we have seen methods of creating the data, now we will see
how we can read data.
• Dataweave selector will allow navigating any combination of Objects and
Arrays to get the data we need.
• Some of the selectors we will be looking into are:
Selector Representation Example
Single Value Selector . Payload.name
Index Selector [n] Payload[1]
Range Selector [n to m] Payload[0..2]
Multi Value Selector * Payload.*name
Descendants Selector .. Payload..echo

Variables and Operators
22
• Variables
– Way to store a value that can be used at later point of time
– Makes code cleaner and readable
– Performance Efficient as stored value can be reused.
• Operators
– Mathematical Operators
• Addition (+), Subtraction (-), Multiplication (*), Division (/)
– Equality and Relational Operators
• < (Greater Than), > (Less Than), <=(Less Than or Equal to), >= (Greater Than or equal to),
== (Equal To)
– Logical Operators
• Not, !, and, or

Flow Control
23
• Flow control is used when you want to execute certain parts of your code in some situations, while not
executing others
• In other words, it’s a way to add logic to your scripts.
• If else..
– If/else expressions allow you to make decisions using logical operators and branch as a result.
– if (<criteria_expression>) <return_if_true> else <return_if_false>
• If else If:
– Multiple conditions needed to be checked and alternate for case statement
if (<criteria_expression1>)
<return_if_true>
else if (<criteria_expression2>)
<return_if_true>
else
<return_if_no_other_match>

Flow Control
24
• Pattern matching is another method of flow control.
• it does quite a bit more under the hood than the if/else expression does.
• Like the if/else expression, pattern matching also returns a single value.

Functions
25
• Functions are one of Data Weave's most important tools
• Allow us to conveniently reuse functionality and create functionality on the fly.
• The following are the different kinds of functions
• Named functions
• Lambdas
• Passing functions to other functions
• Calling 2-arity functions with infix notation
• $, $$, $$$ syntax
• Named Functions:
– Created with fun keyword.
– associates a set of functionality with a name
– No return Keyword

Functions
26
• Dataweave allows us to use various ways to create functions
• Functions without names are called as Lambdas.
• Lambdas can be written as follows
• For Lambda to execute, we surround it in parentheses and append () to the end.

Working with Arrays
27
• We will look at the following aspects of the Arrays
– Filter
– Map
– Distinct by
– Group by
– reduce

Filter
28
• filter(Array<T>, (item: T, index: Number) -> Boolean):
Array<T>
• Iterates over an array and applies an expression that returns
matching values.
• The expression must return true or false
• If the expression returns true for a value or index in the array,
the value gets captured in the output array.
• If it returns false for a value or index in the array, that item gets
filtered out of the output.
• If there are no matches, the output array will be empty.

Map
29
• map(Array<T>, (item: T, index: Number) -> R): Array<R>
• map contains key value pairs
• Example:
%dw 2.0
output application/json
---
["jose", "pedro", "mateo"] map (value, index) -> { (index) : value}

Distinct By
30
• Iterates over an array and returns the unique elements in it.
• Other versions act on an object and handle a null value
• DataWeave uses the result of applying the provided lambda as the
uniqueness criteria.
%dw 2.0
---
[0, 1, 2, 3, 3, 2, 1, 4] distinctBy (value) -> { "unique" : value }

Group By
31
• Returns an object that groups items from an array based on specified
criteria, such as an expression or matching selector.
• This version of groupBy groups the elements of an array using the criteria
function. Other versions act on objects and handle null values.
• Example:
%dw 2.0
var myArray = [
{ "name": "Foo", "language": "Java" },
{ "name": "Bar", "language": "Scala" },
{ "name": "FooBar", "language": "Java" }
]
---
myArray groupBy (item) -> item.language

Reduce
32
• Applies a reduction expression to the elements in an array.
• or each element of the input array, in order, reduce applies the reduction
lambda expression (function), then replaces the accumulator with the new
result.
• The lambda expression can use both the current input array element and
the current accumulator value.
• Example:
%dw 2.0 output application/json --- [2, 3] reduce ($ + $$)

Working with Objects
33
• We will cover the following aspects of Dataweave with objects
– Filter Object
– Map Object
– Pluck
– Update

Working with Objects – Filter Object
34
• Iterates a list of key-value pairs in an object and applies an expression that returns only
matching objects, filtering out the rest from the output.
• The expression must return true or false.
• If the expression returns true for a key, value, or index of an object, the object gets
captured in the output.
• If it returns false for any of them, the object gets filtered out of the output.
• If there are no matches, the output array will be empty.
filterObject({ (K)?: V }, (value: V, key: K, index: Number) -> Boolean): {
(K)?: V }
Example:
%dw 2.0
---
{"a" : "apple", "b" : "banana"} filterObject ((value) -> value == "apple")

Working with Objects – Map Object
35
• Iterates over an object using a mapper that acts on keys, values, or indices of that object.
• Example:
%dw 2.0
---
{"a":"b","c":"d"} mapObject (value,key,index) -> { (index) : { (value):key} }
Example:
%dw 2.0
output application/xml
---
{
prices: payload.prices mapObject (value, key) -> {
(key): (value + 5) as Number {format: "##.00"}
}

Working with Objects – Pluck
36
• Useful for mapping an object into an array, pluck iterates over an object and returns an array of
keys, values, or indices from the object.
• It is an alternative to mapObject, which is similar but returns an object, instead of an array.
• Example:
%dw 2.0
---
{"a":"b","c":"d"} pluck (value,key,index) -> { (index) : { (value):key} }

Trivia….!
37

Trivia 1 ..!
38
what is the daily limit for # of batches and records possible in
a 24hr window
a. 1000 batches and 100,000
b. 20000 batches and 200,000,000
c. 15000 batches and 150,000,000
d. None of the above

Trivia 2 ..!
39
What is the default batch size split in salesforce bulk api v2 ?
a. 10000
b. 20000
c. 1000

Trivia 3 ..!
40
What is the URI/path for getting unfinished results from a
salesforce job ?
a. /services/data/vXX.X/jobs/jobID/ingest/unprocessedrecords
b. /services/data/vXX.X/jobs/ingest/jobID/unprocessedrecords
c. /services/data/vXX.X/jobs/unprocessedrecords

Trivia 4 ..!
41
What DataWeave 2.0 type can be used as input to a map
operation?
a. Object
b. String
c. Array
d. Map

Trivia 5 ..!
42
What does the zip operator do in DataWeave?
a. Minifies the size of value using encoding.
b. Merges elements of two objects into a single object.
c. Merges elements of two lists (arrays) into a single list.
d. None of these

Trivia 6 ..!
43
What does the minus operator do in DataWeave?
A. Decrements the value by one.
B. Removes items from a list.
C. Increments the value by one.
D. Removes characters from a string.

Trivia 7 ..!
44
How can you call a flow from Dataweave?
A. Not allowed
B. Include function
C. Tag function
D. Look Up function

Trivia 8 ..!
45
What is the value of the stepVar variable after the processing
of records in a Batch Job?
A. Null
B. 0
C. -
D. Last value from flow

Explore our new version
MuleSoft Documentation

Find the answers you need, fast.
47
https://docs.mulesoft.com/

The best place to ask questions and help others.
MuleSoft Help Center

15,000+ members ready to help.
49
https://help.mulesoft.com/
• Check out the “MuleSoft
Training” category for all
training and certification-
related questions

Join Charlotte MuleSoft Discussion Group.
50
https://help.mulesoft.com/s/
group/0F92T0000004odaSAA
/charlotte-meetups
• Check out the “Charlotte
MuleSoft Discussion Group.”
category for all your
questions which you want
to discuss within the
Charlotte Group.

Take a stand !
52
• Nominate yourself for the next meetup speaker and suggest a topic
as well.

What’s next
53
• Feedback:
– Contact your organizer Subhash Patel or Aravind Ramadugu to suggest
topics.
– Contact MuleSoft at meetup@mulesoft.com for ways to improve the program.

See you soon…. Don’t forget to send topics to organizer

Mule soft meetup_charlotte_4__draft_v2.0

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Mule soft meetup_charlotte_4__draft_v2.0

Similar to Mule soft meetup_charlotte_4__draft_v2.0 (20)

Recently uploaded

Recently uploaded (20)

Mule soft meetup_charlotte_4__draft_v2.0