DataWeave 2.0
Language Fundamentals
Joshua Erney
● Mountain State Software Solutions
● www.jerney.io
● www.linkedin.com/in/jerney
● @mulemadeeasy
Agenda
1. Introduction
2. What is DataWeave?
3. Language Fundamentals
4. Practical Tips
5. Additional Info
6. Questions?
Who Am I? Why Should You Listen To Me?!
● Programming since ‘11 (not bragging)
○ Java, Python, JavaScript
● Working w/ MuleSoft products for about 3 years
● Saw no DataWeave experts at my company or in the community, I LOVE learning programming
languages, so decided to focus efforts there
● Started learning DataWeave about the same time Functional Programming started making sense
to me (more on functional programming later)
Disclaimer
I don’t work for MuleSoft, and I’ve never seen the source code for DataWeave. Because of this, I have no
way of knowing with 100% certainty how the language works; DataWeave is a black box to me.
What I’m going to cover today is the mental model I’ve built over the last 3 years that represents how I
believe DataWeave works. In other words, what I will describe may not be how DataWeave actually
works, but it will be an effective model for you to use when creating solutions using the language.
What is DataWeave?
● Domain-specific Language created specifically to optimize data transformations regardless of
source and target formats
● Fully-fledged programming language
● Adheres to the core tenets of functional programming
Why Should You Learn DataWeave?
● Honest answer: If you’re planning to use MuleSoft products in the future or planning to be seen as
an expert, you don’t really have a choice
● Don’t need to concern yourself with parsing and (de)serialization of data
● Less lines of code needed to perform data transformations (Less bugs)
● The programming concepts you must learn to effectively use DataWeave will make you a better
programmer. The goal of this talk is to give you an introductory understanding of these concepts.
Why not Java?
● Scope is massive (general purpose programming language)
● Object-Oriented: great for designing large-scale solutions (like the Mule runtime), not great for
designing hundreds of tiny data transformations
● Supports Functional programming (Java 8 Streams), but rather reluctantly.
● Verbose
○ When creating small hundreds of small transformation scripts, you don’t want 50% of that code to be
boilerplate like public class and public static void
Language Fundamentals
(This is the hard part)
If you’re only going to learn one thing from this
webinar is should be this...
DataWeave is a functional
programming language.
What is Functional Programming?
● Programming paradigm, like “procedural” or “object-oriented”
● Wikipedia says:
“... a programming paradigm … that treats computation as the evaluation of mathematical
functions and avoids changing-state and mutable data” (emphasis mine)
and
“It is a declarative programming paradigm, which means programming is done with expressions
and declarations instead of statements.” (emphasis mine, again)
What is meant by “mathematical functions”:
Pure functions, the same input ALWAYS yields
the same output:
fun add(x, y) = x + y
add(1, 1) // = 2
add(1, 1) // = 2
This is different from methods, which can modify
the state of the object they’re associated with:
Adder one = new Adder(1);
one.add(1) // = 2
adder.setValue(2)
one.add(1) // = 3 ???
METHODS DON’T EXIST IN DATAWEAVE
Core Concepts (Language Fundamentals)
“... a programming paradigm … that treats computation as the evaluation of mathematical
functions and avoids changing state and mutable data” (emphasis mine)
“It is a declarative programming paradigm, which means programming is done with expressions
and declarations instead of statements.” (emphasis mine, again)
1. Expressions Instead of Statements
2. Immutable Data
3. Mathematical Pure Functions
Everything is an Expression
Most programs are made up of expressions.
● An expression is something that evaluates to a value, e.g, 2 + 2
● A statement is more like a command, and represents an action, e.g., System.out.print(“Hello,
World!);
In other words: Expressions return values, statements do not.
Statements are great for doing things like printing, saving and setting values, and I/O operations.
Expressions are great for doing calculations and transformations
Everything is an Expression (cont.)
Java Statements
boolean even = true;
int x = 0;
if(even) {
x = 2;
} else {
x = 1;
}
DataWeave Expressions
var even = true
var x = if (even) 2 else 1
This returns
This does not return
Expressions Can Always Be Substituted
Example:
// Returns an array
fun complexStuff(arr) =
...
---
complexStuff(arr) map $.one
Example Pt2 (Complex Stuff is failing and
preventing you from testing the map:
// replace complexStuff with what it
// SHOULD evaluate to
...
---
[{one: 1},{one: 2}] map $.one
USE THIS WHEN YOU’RE DEBUGGING
Key Point 🔑
There are no statements in DataWeave, so you can’t perform an action that doesn’t return something
(e.g. simply writing to the console). Every logical grouping of DataWeave code returns something.
Expressions can always be replaced by the value they return. Use this property when you’re debugging!
Objects are Immutable, No Changing State
● Once an object is created, it never changes
● Typically, this is an advantage, because you have WAY less to keep track of
● How does any work get done?
Is the payload modified?
No, so what’s happening instead?
The payload “variable” is being reassigned, like this:
// Psuedo Code
payload = http.getRequestData()
// New payload created from old payload, old payload reassigned to new payload
payload = transformMessage(payload)
And this happens to the target of every DataWeave transform. If the target is a variable, and the variable is not
assigned yet, it is assigned. If the variable is already assigned, it is reassigned to the output of the DataWeave
script.
Mental Model
Key Point 🔑
Always remember that you are never modifying anything when you use DataWeave code. DataWeave
cannot modify values, it can only create new ones. It gives the “illusion” of modification through
reassignment.
Functions (cont.)
There is no return keyword (it doesn’t need one, everything is an expression).
Functions return the last expression in the body that is evaluated:
fun add(n, m) = n + m
Isn’t necessarily the same expression every time:
fun addIfOdd(n, m) = if ((n mod 2) == 0) n + m else n - m
More Functions
DataWeave Supports Higher-Order Functions
● Fancy way of saying: functions that take other functions as inputs
● You do it all the time with `map`, `filter`, `groupBy`, etc.
payload map (employee) -> employee.id
This is a function passed to the `map` function
Understanding Higher-Order Functions
Higher-order functions are very good at separating concerns. Using our map example from earlier… map
takes care of looping through the input array and applying the function to each value. The function
passed to map describes how each element in the input array should be transformed.
payload map (employee) -> employee.id
Takes care of the iteration and applying the function
Describes how each element should be transformed
Did I Mention… Functions?
Named functions:
fun greeting(name) = “Hello, $(name)”
Unnamed functions (aka “lambdas”):
(name) -> “Hello, $(name)”
Function assignment is just syntax sugar for lambdas assigned to a variable:
var greeting = (name) -> “Hello, $(name)”
greeting(“Virtual Muleys!”)
Did I Mention… Functions? (cont.)
Without Lambdas
fun mapping(obj) =
{name: obj.name, age: obj.age + 1}
---
payload map mapping($)
With Lambdas
payload map (obj) -> {
name: obj.name,
age: obj.age
}
Lambdas make it so it’s really easy to use higher order functions. We don’t need to use as much code.
Key Point 🔑
Functions are DataWeave’s most powerful and flexible tool for building transformations. Make sure you
fully understand how they’re different from Java methods (pass functions around w/o classes, assign
them to variables), and how to effectively use them (building higher-order functions, using lambdas).
Higher-order functions are great for separating concerns. Could you imagine if you had to write the code
to loop through an array and modify every value in a specific way every time? With the higher-order
function map, you don’t have to.
Practical Tips
Settle in for the journey
1. Accept that DataWeave is a complete programming language
2. Accept that DataWeave is a functional programming language, and you might not know anything
about functional programming, yet
3. Accept that there is not a lot of content out there that discusses functional programming
principles as they relate to DataWeave, or how DataWeave is a functional programming language
(except my blog 😉)
Functions are King
● Make sure you fully understand what “pure” (i.e. mathematical) functions are, and how they’re
beneficial to this style of programming (transformation)
● Make sure you understand that when you’re using `map`, `filter`, etc, that you’re using higher-
order functions
● Finally, try writing some higher order functions of your own
Don’t Give Up!
● The paradigm shift from object-oriented to functional is difficult
● Takes time and deliberate practice
● If it doesn’t work with DataWeave, try practicing with a different language:
○ Scala
○ JavaScript (some parts, search “Functional JavaScript”)
○ Clojure (if you want to get weird)
○ Haskell (if you want to get even weirder)
● Ask questions on the Mule Forums (to get help from everyone besides me), Stack Overflow (to get
help from me and some other people)
● Reach out to me if I can help
Additional Info:
Follow us on twitter: @MuleMadeEasy
Learn how to use reduce (good content on HOFs as well): https://www.jerney.io/dataweave-when-to-use-
reduce/
Looking for something more advanced? (pattern matching, recursion): https://www.jerney.io/dataweave-
applywhenkey-function/, https://www.jerney.io/how-to-modify-all-values-of-an-element/
Additional Info (Cont.)
Additional Info (Cont.)
Good Intro to Functional Programming: First 6 Chapters of “On Lisp”, by Graham:
http://ep.yimg.com/ty/cdn/paulgraham/onlisp.pdf
Another Good Intro: First 2 Chapters of “Structure and Interpretation of Computer Programs”, by
Abelson, Sussman & Sussman: http://web.mit.edu/alexmv/6.037/sicp.pdf
Something way more advanced: “Functional Programming in Scala” by Chiusana & Bjarnason:
https://github.com/awantik/scala-
programming/blob/master/Manning%20Functional%20Programming%20in%20Scala%20(2015).pdf
Questions?

Data weave 2.0 language fundamentals

  • 1.
    DataWeave 2.0 Language Fundamentals JoshuaErney ● Mountain State Software Solutions ● www.jerney.io ● www.linkedin.com/in/jerney ● @mulemadeeasy
  • 2.
    Agenda 1. Introduction 2. Whatis DataWeave? 3. Language Fundamentals 4. Practical Tips 5. Additional Info 6. Questions?
  • 3.
    Who Am I?Why Should You Listen To Me?! ● Programming since ‘11 (not bragging) ○ Java, Python, JavaScript ● Working w/ MuleSoft products for about 3 years ● Saw no DataWeave experts at my company or in the community, I LOVE learning programming languages, so decided to focus efforts there ● Started learning DataWeave about the same time Functional Programming started making sense to me (more on functional programming later)
  • 4.
    Disclaimer I don’t workfor MuleSoft, and I’ve never seen the source code for DataWeave. Because of this, I have no way of knowing with 100% certainty how the language works; DataWeave is a black box to me. What I’m going to cover today is the mental model I’ve built over the last 3 years that represents how I believe DataWeave works. In other words, what I will describe may not be how DataWeave actually works, but it will be an effective model for you to use when creating solutions using the language.
  • 5.
    What is DataWeave? ●Domain-specific Language created specifically to optimize data transformations regardless of source and target formats ● Fully-fledged programming language ● Adheres to the core tenets of functional programming
  • 6.
    Why Should YouLearn DataWeave? ● Honest answer: If you’re planning to use MuleSoft products in the future or planning to be seen as an expert, you don’t really have a choice ● Don’t need to concern yourself with parsing and (de)serialization of data ● Less lines of code needed to perform data transformations (Less bugs) ● The programming concepts you must learn to effectively use DataWeave will make you a better programmer. The goal of this talk is to give you an introductory understanding of these concepts.
  • 8.
    Why not Java? ●Scope is massive (general purpose programming language) ● Object-Oriented: great for designing large-scale solutions (like the Mule runtime), not great for designing hundreds of tiny data transformations ● Supports Functional programming (Java 8 Streams), but rather reluctantly. ● Verbose ○ When creating small hundreds of small transformation scripts, you don’t want 50% of that code to be boilerplate like public class and public static void
  • 9.
  • 10.
    If you’re onlygoing to learn one thing from this webinar is should be this...
  • 11.
    DataWeave is afunctional programming language.
  • 12.
    What is FunctionalProgramming? ● Programming paradigm, like “procedural” or “object-oriented” ● Wikipedia says: “... a programming paradigm … that treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data” (emphasis mine) and “It is a declarative programming paradigm, which means programming is done with expressions and declarations instead of statements.” (emphasis mine, again)
  • 13.
    What is meantby “mathematical functions”: Pure functions, the same input ALWAYS yields the same output: fun add(x, y) = x + y add(1, 1) // = 2 add(1, 1) // = 2 This is different from methods, which can modify the state of the object they’re associated with: Adder one = new Adder(1); one.add(1) // = 2 adder.setValue(2) one.add(1) // = 3 ??? METHODS DON’T EXIST IN DATAWEAVE
  • 14.
    Core Concepts (LanguageFundamentals) “... a programming paradigm … that treats computation as the evaluation of mathematical functions and avoids changing state and mutable data” (emphasis mine) “It is a declarative programming paradigm, which means programming is done with expressions and declarations instead of statements.” (emphasis mine, again) 1. Expressions Instead of Statements 2. Immutable Data 3. Mathematical Pure Functions
  • 15.
    Everything is anExpression Most programs are made up of expressions. ● An expression is something that evaluates to a value, e.g, 2 + 2 ● A statement is more like a command, and represents an action, e.g., System.out.print(“Hello, World!); In other words: Expressions return values, statements do not. Statements are great for doing things like printing, saving and setting values, and I/O operations. Expressions are great for doing calculations and transformations
  • 16.
    Everything is anExpression (cont.) Java Statements boolean even = true; int x = 0; if(even) { x = 2; } else { x = 1; } DataWeave Expressions var even = true var x = if (even) 2 else 1 This returns This does not return
  • 17.
    Expressions Can AlwaysBe Substituted Example: // Returns an array fun complexStuff(arr) = ... --- complexStuff(arr) map $.one Example Pt2 (Complex Stuff is failing and preventing you from testing the map: // replace complexStuff with what it // SHOULD evaluate to ... --- [{one: 1},{one: 2}] map $.one USE THIS WHEN YOU’RE DEBUGGING
  • 18.
    Key Point 🔑 Thereare no statements in DataWeave, so you can’t perform an action that doesn’t return something (e.g. simply writing to the console). Every logical grouping of DataWeave code returns something. Expressions can always be replaced by the value they return. Use this property when you’re debugging!
  • 19.
    Objects are Immutable,No Changing State ● Once an object is created, it never changes ● Typically, this is an advantage, because you have WAY less to keep track of ● How does any work get done?
  • 20.
    Is the payloadmodified?
  • 21.
    No, so what’shappening instead? The payload “variable” is being reassigned, like this: // Psuedo Code payload = http.getRequestData() // New payload created from old payload, old payload reassigned to new payload payload = transformMessage(payload) And this happens to the target of every DataWeave transform. If the target is a variable, and the variable is not assigned yet, it is assigned. If the variable is already assigned, it is reassigned to the output of the DataWeave script.
  • 22.
  • 23.
    Key Point 🔑 Alwaysremember that you are never modifying anything when you use DataWeave code. DataWeave cannot modify values, it can only create new ones. It gives the “illusion” of modification through reassignment.
  • 24.
    Functions (cont.) There isno return keyword (it doesn’t need one, everything is an expression). Functions return the last expression in the body that is evaluated: fun add(n, m) = n + m Isn’t necessarily the same expression every time: fun addIfOdd(n, m) = if ((n mod 2) == 0) n + m else n - m
  • 25.
    More Functions DataWeave SupportsHigher-Order Functions ● Fancy way of saying: functions that take other functions as inputs ● You do it all the time with `map`, `filter`, `groupBy`, etc. payload map (employee) -> employee.id This is a function passed to the `map` function
  • 26.
    Understanding Higher-Order Functions Higher-orderfunctions are very good at separating concerns. Using our map example from earlier… map takes care of looping through the input array and applying the function to each value. The function passed to map describes how each element in the input array should be transformed. payload map (employee) -> employee.id Takes care of the iteration and applying the function Describes how each element should be transformed
  • 27.
    Did I Mention…Functions? Named functions: fun greeting(name) = “Hello, $(name)” Unnamed functions (aka “lambdas”): (name) -> “Hello, $(name)” Function assignment is just syntax sugar for lambdas assigned to a variable: var greeting = (name) -> “Hello, $(name)” greeting(“Virtual Muleys!”)
  • 28.
    Did I Mention…Functions? (cont.) Without Lambdas fun mapping(obj) = {name: obj.name, age: obj.age + 1} --- payload map mapping($) With Lambdas payload map (obj) -> { name: obj.name, age: obj.age } Lambdas make it so it’s really easy to use higher order functions. We don’t need to use as much code.
  • 29.
    Key Point 🔑 Functionsare DataWeave’s most powerful and flexible tool for building transformations. Make sure you fully understand how they’re different from Java methods (pass functions around w/o classes, assign them to variables), and how to effectively use them (building higher-order functions, using lambdas). Higher-order functions are great for separating concerns. Could you imagine if you had to write the code to loop through an array and modify every value in a specific way every time? With the higher-order function map, you don’t have to.
  • 30.
  • 31.
    Settle in forthe journey 1. Accept that DataWeave is a complete programming language 2. Accept that DataWeave is a functional programming language, and you might not know anything about functional programming, yet 3. Accept that there is not a lot of content out there that discusses functional programming principles as they relate to DataWeave, or how DataWeave is a functional programming language (except my blog 😉)
  • 32.
    Functions are King ●Make sure you fully understand what “pure” (i.e. mathematical) functions are, and how they’re beneficial to this style of programming (transformation) ● Make sure you understand that when you’re using `map`, `filter`, etc, that you’re using higher- order functions ● Finally, try writing some higher order functions of your own
  • 33.
    Don’t Give Up! ●The paradigm shift from object-oriented to functional is difficult ● Takes time and deliberate practice ● If it doesn’t work with DataWeave, try practicing with a different language: ○ Scala ○ JavaScript (some parts, search “Functional JavaScript”) ○ Clojure (if you want to get weird) ○ Haskell (if you want to get even weirder) ● Ask questions on the Mule Forums (to get help from everyone besides me), Stack Overflow (to get help from me and some other people) ● Reach out to me if I can help
  • 34.
    Additional Info: Follow uson twitter: @MuleMadeEasy Learn how to use reduce (good content on HOFs as well): https://www.jerney.io/dataweave-when-to-use- reduce/ Looking for something more advanced? (pattern matching, recursion): https://www.jerney.io/dataweave- applywhenkey-function/, https://www.jerney.io/how-to-modify-all-values-of-an-element/
  • 35.
  • 36.
    Additional Info (Cont.) GoodIntro to Functional Programming: First 6 Chapters of “On Lisp”, by Graham: http://ep.yimg.com/ty/cdn/paulgraham/onlisp.pdf Another Good Intro: First 2 Chapters of “Structure and Interpretation of Computer Programs”, by Abelson, Sussman & Sussman: http://web.mit.edu/alexmv/6.037/sicp.pdf Something way more advanced: “Functional Programming in Scala” by Chiusana & Bjarnason: https://github.com/awantik/scala- programming/blob/master/Manning%20Functional%20Programming%20in%20Scala%20(2015).pdf
  • 37.

Editor's Notes

  • #4 More esoteric languages like Clojure, Racket, Scala, Ruby Saw no DataWeave experts: when I started using Mule dataweave was just coming out, people were skeptical to learn it because they were afraid it would get replaced again like Data Mapper did. After using it for a few months, I thought MuleSoft nailed it, didn’t see why it would ever be replaced. Dove in.
  • #6 What problem is it solving?
  • #8 Didn’t find any information regarding the core language concepts behindDataWeave programming, so I found languages that exhibited similar traits, and learned them. If a 9 year old can do it, so can I! Most of what I learned about how to effectively use DataWeave came from learning the functional concepts that more prominent languages, like JavaScript, Python, Scala, and Clojure. Once I had a handle on these concepts, I started sharing what I learned with my company, and later, the public.
  • #9 What problem is it solving?
  • #10 Might not get it the first time around Jot down any questions or doubts you have! We will have plenty of time to answer them at the end
  • #13 Evaluation of mathematica functions vs using statements to call methods
  • #14 Functions are completely deterministic
  • #16 An expression is something that always evaluates to a value. Expressions can always be replaced by the value that they evaluate to. This means everything except variable and function declarations return something in DataWeave. This has a profound affect on the way that you write your code.
  • #19 Think of logical groupings as any piece of code you could put a set of parenthesis around
  • #20 Objects in the object oriented sense, not the JavaScript sense This is great because when you call a function in dataweave you never need to worry about if that function is going to change anything you passed into it.