2. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Session Description
"Wait, what? Biml is not just for generating SSIS packages?"
Absolutely not! Come and see how you can use Biml (Business
Intelligence Markup Language) to save time and speed up
other Data Warehouse development tasks. You can generate
complex T-SQL statements with Biml instead of using dynamic
SQL, create test data, and even populate static dimensions.
Don't Repeat Yourself, start automating those boring, manual
tasks today!

3. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
@cathrinew
cathrinew.net

4. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)

5. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
…the next 50 minutes…
T-SQL Test Data Dimensions

6. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Wait…
Can't I use
<something else>?

7. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Of course!
But Biml works well
with source metadata

8. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
And…
Biml is fun!

9. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
But... Biml? How?

10. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Traditional Biml

11. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Crazy Fun Biml

12. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
What do you need?

13. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Preview Pane
The Power is in the...

14. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Traditional BimlScript

15. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Traditional BimlScript
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load_<#=table.Name#>" />
<# } #>
<Package Name="Load_Customer" />
<Package Name="Load_Product" />
<Package Name="Load_Sales" />

16. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
BimlScript to Biml
BimlExpress Preview Pane

17. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
BimlScript to... ???
BimlExpress Preview Pane

18. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Crazy Fun BimlScript
<# foreach (var table in RootNode.Tables) { #>
Yay, a package! <#=table.Name#> :)
<# } #>
Yay, a package! Load_Customer :)
Yay, a package! Load_Product :)
Yay, a package! Load_Sales :)

19. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)

20. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
T-SQL from Biml

21. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnList()
GetColumnAssignmentList()
GetColumnComparisonList()

22. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Column Methods
Return code fragments
Use as building blocks in custom T-SQL
Customize output by passing parameters
Filter columns by using lambda expressions

23. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Column Methods
Return code fragments
Use as building blocks in custom T-SQL
Customize output by passing parameters
Filter columns by using lambda expressions

24. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
"A lambda expression is an
anonymous function that you can
use to create delegates or
expression tree types"

25. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
"A lambda expression is an
anonymous function that you can
use to create delegates or
expression tree types"

26. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"

27. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
The arrow is the lambda operator

28. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
Input parameter is on the left side

29. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
Expression is on the right side

30. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"

31. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnList()
Get columns for SELECT:

32. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnList()
[Col1], [Col2], [Col3]

33. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnList("src")
[src].[Col1], [src].[Col2], [src].[Col3]

34. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList()
Get columns for JOIN … ON:

35. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList()
[l].[Col1] = [r].[Col1]
AND [l].[Col2] = [r].[Col2]
AND [l].[Col3] = [r].[Col3]

36. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList("!=")
[l].[Col1] != [r].[Col1]
AND [l].[Col2] != [r].[Col2]
AND [l].[Col3] != [r].[Col3]

37. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList("src", "dst")
[src].[Col1] = [dst].[Col1]
AND [src].[Col2] = [dst].[Col2]
AND [src].[Col3] = [dst].[Col3]

38. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList
("!=", "src", "dst")
[src].[Col1] != [dst].[Col1]
AND [src].[Col2] != [dst].[Col2]
AND [src].[Col3] != [dst].[Col3]

39. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList
("!=", "src", "dst", " OR ",)
[src].[Col1] != [dst].[Col1]
OR [src].[Col2] != [dst].[Col2]
OR [src].[Col3] != [dst].[Col3]

40. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList
GetColumnComparisonList()
GetColumnComparisonList("!=")
GetColumnComparisonList("dst", "src")
GetColumnComparisonList(c => c.IsUsedInPrimaryKey)
GetColumnComparisonList(c => c.IsUsedInPrimaryKey, "dst", "src")
GetColumnComparisonList(c => c.IsUsedInPrimaryKey, "!=", "dst", "src")

41. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnAssignmentList()
Get columns for UPDATE … SET:

42. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnAssignmentList()
[l].[Col1] = [r].[Col1]
,[l].[Col2] = [r].[Col2]
,[l].[Col3] = [r].[Col3]

43. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnAssignmentList("src", "dst")
[src].[Col1] = [dst].[Col1]
,[src].[Col2] = [dst].[Col2]
,[src].[Col3] = [dst].[Col3]

44. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
T-SQL from Biml

45. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Test Data

46. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml and Bogus
Create Random and Specific Test Data

47. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Bogus Project
Simple and sane fake data generator for .NET
https://github.com/bchavez/Bogus
Advanced functionality for custom objects
…or simple functionality for Biml hacks :)

48. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Address
Commerce
Company
Database
Date
Finance
Internet
Lorem
Name
Person
Phone
System
Bogus API
Supports all data types plus built-in test data, including:

49. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Bogus Methods
<#@ assembly name="Bogus.dll" #>
<#@ import namespace="Bogus" #>
<# var f = new Faker(); #>
<# Person p = new Person(); #>
INSERT INTO dbo.Employee(FirstName, LastName, Birthday) VALUES
(
'<#=p.FirstName#>',
'<#=p.LastName#>',
'<#=p.DateOfBirth.ToString("yyyyMMdd")#>'
)

50. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
BOgus Installation
Install via Nuget:
Install-Package Bogus
Or download latest release .zip:
https://github.com/bchavez/Bogus/releases

51. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
Test Data
with Bogus

52. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Static Dimensions

53. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Static Dimensions
SCD Type 0: Not changing
• Unknown dimension members
• Date dimension
• Code tables

54. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Static Dimension Creation
1. Create table definition
2. Add static source rows
3. Generate INSERT statements

55. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Sources
Part of <Table> objects
Defines rows to be inserted when table is created

56. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Source
<Table Name="Table" SchemaName="Database.Schema">
<Columns>
<Column Name="Column1" DataType="Int32" />
<Column Name="Column2" DataType="String" Length="10" />
</Columns>
<Sources>
<StaticSource Name="TableRows">
...
</StaticSource>
</Sources>
</Table>

57. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Source
<StaticSource Name="TableRows">
<Rows>
<Row>
<ColumnValues>
<ColumnValue ColumnName="Col1" Value="-1" />
<ColumnValue ColumnName="Col2" Value="'N/A'" />
</ColumnValues>
</Row>
</Rows>
</StaticSource>

58. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Source
<# if (table.Sources.Any()) { #>
<#=TableToPackageLowerer.GetStaticSourceInsertStatements(
table.SchemaQualifiedName,
table.HasIdentity,
(AstTableStaticSourceNode)table.Sources.FirstOrDefault()
)#>
<# } #>

59. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
Static Dimensions

60. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
…the Past 50 minutes…
T-SQL Test Data Dimensions

61. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Is this useful?

62. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
What other
ideas do you
have?

63. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Have fun!

64. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
@cathrinew
cathrinew.net
hi@cathrinew.net
Biml resources and demo files:
cathrinew.net/biml