You know how to use Power BI or Power Query to load data from different sources but you want to go further than limiting yourself to the UI? You’re tired to look at auto-generated code and don’t understand the key concepts of this powerful ETL language? Then, this session is for you. During 60 minutes, you’ll learn how to write code in this functional language. Starting with syntax, primitive and structured values; we’ll quickly jump to the creation of expressions and functions and the usage of some recurrent patterns. This session contains many demos and real-life use-cases that you’ll be able to transpose to your own projects.

1. It’s Power Query M
Let’s have some fun!
Cédric L. Charlier
@Seddryck

2. Sponsors help us to run this event! THX!
You Rock! Sponsor
Gold Sponsor
Silver Sponsor
Bronze Sponsor

3. You Rock! Sponsor Session
13:45 Track 1
„Das super nerdige Solisyon Film- und Serienquiz“

4. Save the date for exiting upcoming events
PASS Camp 2017
Main Camp 05.12. – 07.12.2017 (04.12. Kick-Off abends)
Lufthansa Training & Conference Center, Seeheim
SQL Konferenz 2018
PreCon: 26.02.2018
MainCon: 27.02. – 28.02.2018
Darmstadtium, Darmstadt
More information at PASS booth

5. In a nutshell
Open-sources
contributions
Skills &
Interests
Recognitions
 Agile data warehouse
 Quality-oriented
 Automation fanboy
 Microsoft MVP Data
Platform award
 Agile leader of the month
 Database architecture
 Data modeling
 Business intelligence
 Data warehousing
 ETL
 Olap/Tabular
 No SQL
 Big data
 Data analysis
 Machine learning
 Testing framework for BI
 www.nbi.io
 Other projects on github
 Deployment SSRS – RsPackage
 TmBundle for a few languages
 Starting SSIS from .Net - Cassis
 Toolset for special cases of modeling:
ERMine et Tibre
 And much more
Experience
 15 years with SQL Server
 Former consultant, working
internally for Elia
Cédric L. Charlier
@Seddryck
seddryck.wordpress.com
Data & BI architect

6. Agenda
• How to be boring …
• Decode the fun with the html characters
• Error handling … keep smiling!
• “Do something while” does not exist
• Doing the L(augh) of ETL (no kidding)
• Metadata freedom !!!
Frankly, I can’t remember
why I selected this session

7. Programming paradigm
Functional language
• Declarative (expressions not statements)
• Languages: Lisp, Scheme, Clojure, Erlang, Haskell, F#
• DSL: R, XQuery/XSLT, (SQL and Lex/Yacc)
• Extensions: Java, C#, PHP, Julia

8. Benefits of functional languages
Evaluation of mathematical functions
• Avoids changing-state and mutable data
• No local or global state
• No side effect
• Output only depends on the arguments passed to this function

9. Expressiveness of a functional language
C# (imperative way)
static int Fibonacci(int n)
{
int a = 0;
int b = 1;
int c = 0;
if (n != 1) {
for (int i = 1; i <= n; i++){
c = a + b;
a = b;
b = c;
}
return a;
}
else {
return 1;
}
}
Elixir (functional way)
defmodule Fibonacci do
def fib(0), do: 0
def fib(1), do: 1
def fib(n), do: fib(n-1) + fib(n-2)
end

10. Values, list, records and tables
123 // A number
true // A logical
"abc" // A text
null // null value
#duration (0,1,30,0)
{123, true, "A"} // list
{1, 2, 3}
{1, 3..7, 9} //list with
values 1, 3, 4, 5, 6, 7, 9
[
A = 1,
B = 2,
C = 3
] // a record
#table(
{"A", "B"},
{
{1, 2},
{3, 4}
}
)

11. Nested records
[
Sales =
[ FirstHalf = 1000,
SecondHalf = 1100
],
Total =
Sales[FirstHalf] +
Sales[SecondHalf]
]
[ Sales =
{ [ Year = 2007,
Total = 2100
],
[ Year = 2008,
Total = 2500
]
}, TotalSales =
Sales{0}[Total] +
Sales{1}[Total] // 4600
]
Lookup
operator
Positional
index
operator

12. Function and evaluation
• Parameters between
brackets
• Symbol => to separate
declaration and
implementation
• Lazy evaluation of function
• Persistence of evaluation
(x, y) => (x + y) / 2
[
A1 = A2 * 2,
A2 = A3 + 1,
A3 = 1
]
Equivalent to
[
A1 = 4,
A2 = 2,
A3 = 1
]

13. The standard library
More than 500 predefined functions:
• Manipulate values
• Number.E,
• TextPositionOf(“Hello”, “ll”)
• Modify lists, records and tables
• List.Transform(…)
• Record.FieldCount()
• Table.AddColumn(…)
• Access data sources
• Sql.Database(…)
• Web.Contents(…)

14. each
Simplified declaration – function taking a single
parameter “_” (underscore)
each _+1  (_) => _ + 1
each [A]  (_) => _[A]
Table.SelectRows(t, each [Weight]>12)

Table.SelectRows(t, (x) => x[Weight]>12)

15. Power Query editor
• “Advanced editor”
• Notepad++
• Visual Studio and
Power Query SDK

16. The secret of
being boring is to
say everything
Voltaire

17. Demo (live coding)
• Create a list with 5000
elements from 1 to 5000
• Create a record by adding a
second field with random
value
• How-to
• Each keyword
• Persistence of evaluation

18. Agenda
• How to be boring …
• Decode the fun with the html
characters
• Error handling … keep smiling!
• “Do something while” does not exist
• Doing the L(augh) of ETL (no kidding)
• Metadata freedom !!!

19. If … then … else
• Select between two expressions
• Based on a logical condition
• else is not facultative
• But you can explicitly inform that this
path is not implemented by using an
ellipsis (…)
if a<b then a+2 else …

20. Demo: parse some html characters
• Encoded with pattern &#number;
• First iteration:
• Only 1 special char by word
• No error
• Expl:
C&#233;dric is translated to Cédric

21. Agenda
• How to be boring …
• Decode the fun with the html
characters
• Error handling … keep smiling!
• “Do something while” does not exist
• Doing the L(augh) of ETL (no kidding)
• Metadata, freedom !!!
I’ll introduce you to the Power
Query M language.
But first take a minute to say
goodbye to your friends and
family!

22. Handling errors
• Functions can return errors
• 0/0
• Use try … otherwise … to
handle them
• You can create your own errors
with error “my message”
• And check them with
x[HasError] and x[Error]

23. A challenge
Parse movie title and extract
production’s year
• The Shawshank Redemption (1994)
• My Dad Says (2010) {Dog Ed (#1.7)}
• Years Younger (2004/I)
• abcd(19xy) hqskj dhq (2020)
• fdsdf (1995) sdfsdf (19)

24. Agenda
• How to be boring …
• Decode the fun with the html
characters
• Error handling … keep smiling!
• “Do something while” does not exist
• Doing the L(augh) of ETL (no kidding)
• Metadata freedom !!!

25. Recursive function
Scoping operator “@” (arrobas)
[
Factorial = (x) =>
if x = 0
then 1
else x * @Factorial(x - 1),
Result = Factorial(3) // 6
]

26. Demo: back to the html characters
• Encoded with pattern &#number;
• Second iteration:
• None or many special chars in a word
• Expl:
H&#233;l&#232;ne is translated to Hélène
Alice is translated to Alice
Cl&#233;mence is translated to Clémence

27. Agenda
• How to be boring …
• Decode the fun with the html
characters
• Error handling … keep smiling!
• “Do something while” does not exist
• Doing the L(augh) of ETL (no kidding)
• Metadata freedom !!!

28. Using data source access
• Standard library includes methods to access data
sources
• Sql.Database lets you run a query on a database
• Usually it’s a SELECT but could be INSERT or
UPDATE

29. Demo
Manage customers:
• Update “Age”
• Insert new customers

30. Agenda
• How to be boring …
• Decode the fun with the html
characters
• Error handling … keep smiling!
• “Do something while” does not exist
• Doing the L(augh) of ETL (no kidding)
• Metadata, freedom !!!

31. Metadata
Information associated to a value
Presented as a record
• Assign
"Mozart" meta [
Rating = 5,
Tags = {"Classical"}
]
• Retrieve
Value.Metadata( "Mozart" )

32. Demo
• Retrieve content of a website
• Handling not existing/available
website

33. Don‘t forget ... After-Show-Party!!!
5 Jahre SQL Saturday
an der Hochschule Bonn-Rhein-Sieg
SQLSat Bruzzler - Grillparty
Würstchen & Bier ab ca. 19.00 Uhr
am Ende der Hochschulstraße

34. Sponsors
You Rock! Sponsor
Gold Sponsor
Silver Sponsor
Bronze Sponsor