Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (Presented at SQL Server Days Belgium on September 29th 2015)

1. Don't Repeat Yourself -
Agile SSIS development with
Biml and BimlScript
Cathrine Wilhelmsen
SQL Server Days 2015

2. SQL Server Days would like to thank all of our sponsors!

3. Session Description

4. Cathrine Wilhelmsen
@cathrinew
cathrinewilhelmsen.net
Data Warehouse Architect
Business Intelligence Developer

5. Who are you? *
( * Probably not a cat)

6. Your work day?
(Drag, drop, drag, drop, connect, drag, drop, connect,
resize, align, drag, drop, resize, connect, align…)

7. job done!
new standards
...yay
Ever experienced this?

8. Ready for a change?

9. From Traditional SSIS to Agile SSIS

10. Traditional SSIS: Plumbing

11. Agile SSIS: Business Logic

13. Will Biml solve all your problems?

14. How can Biml help you?

15. What is Business Intelligence Markup Language?

16. What do you need?

17. ...or you can invest in

18. How does it work?

19. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax

20. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Root Element

21. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Collection of Elements

22. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Elements

23. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Attributes

24. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Full vs. Shorthand Syntax

25. Let's generate some packages

26. Right-click on SSIS project and click Add New Biml File

27. Biml files are placed in the Miscellaneous folder
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>

28. Right-click on the Biml file to Generate SSIS Packages
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>

29. From Biml to SSIS

30. From Biml to SSIS

31. .biml vs .dtsx: human-readable vs ALL THE CODE!
(150% zoom) (20% zoom)

32. I create SSIS packages faster than that

33. But wait!

34. The magic is in the

35. Import
Loop
Add expressions
What is ?

36. BimlScript code blocks
<#@ … #>
<# … #>
<#= … #>
<#+ … #>

37. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
BimlScript syntax

38. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
BimlScript syntax: Control Blocks

39. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
BimlScript syntax: Expression Control Block

40. How does it work?

41. Yes, but how does it work?
41

42. Yes, but how does it actually work?
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="LoadCustomer"></Package>
<Package Name="LoadProduct"></Package>
<Package Name="LoadSales"></Package>
</Packages>
</Biml>

43. Don't Repeat Yourself

44. Don't Repeat Yourself

45. Don't Repeat Yourself

46. <#@ template tier="1" #>
Don't Repeat Yourself: Split and combine Biml files

47. Split and combine multiple Biml files
1
2
3

48. <#@ include file="CommonCode.biml" #>
Don't Repeat Yourself: Include files

49. Don't Repeat Yourself: Include files

50. Don't Repeat Yourself: Include files

51. Don't Repeat Yourself: Include files

52. <#@ property name="Table" type="AstTableNode" #>
<#=CallBimlScript("CommonCode.biml", Table)#>
Don't Repeat Yourself: CallBimlScript with parameters

53. Don't Repeat Yourself: CallBimlScript with parameters

54. Don't Repeat Yourself: CallBimlScript with parameters

55. Don't Repeat Yourself: CallBimlScript with parameters

56. Don't Repeat Yourself: CallBimlScript with parameters

57. Don't Repeat Yourself: CallBimlScript with parameters

58. C# Classes and Methods

59. <#+
public class HelperClass {
public static string HelperMethod(string InputParameter) {
return string.Concat("InputParameter: ", InputParameter);
}
}
#>
C# Classes and Methods

60. <Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#=HelperClass.HelperMethod("Cats")#>
</Biml>
<#+
public class HelperClass {
public static string HelperMethod(string InputParameter) {
return string.Concat("InputParameter: ", InputParameter);
}
}
#>
C# Classes and Methods: Inline

61. <#@ include file="HelperClass.biml" #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#=HelperClass.HelperMethod("Cats")#>
</Biml>
C# Classes and Methods: Included
<#+
public class HelperClass {
public static string HelperMethod(string InputParameter) {
return string.Concat("InputParameter: ", InputParameter);
}
}
#>

62. <#@ code file="..CodeHelperClass.cs" #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#=HelperClass.HelperMethod("Cats")#>
</Biml>
C# Classes and Methods: Mist / BimlStudio
using System.Data;
public class HelperClass {
public static string HelperMethod(string InputParameter) {
return string.Concat("InputParameter: ", InputParameter);
}
}

63. Transformers
(No, not those transformers, but still awesome!)

64. Transformers: Modify existing Biml

65. <#@ target type="Package" mergemode="LocalMerge" #>
<!–- <Node> = Target Type = <Package> -->
<Node>
<Variables>
<Variable Name="NewRows" DataType="Int32">0</Variable>
</Variables>
</Node>
Transformers syntax

66. <#@ target type="Package" mergemode="LocalMerge" #>
<!–- <Node> = Target Type = <Package> -->
<Node>
<Variables>
<Variable Name="NewRows" DataType="Int32">0</Variable>
</Variables>
</Node>
Transformers syntax: Target Type

67. <#@ target type="Package" mergemode="LocalMerge" #>
<!–- <Node> = Target Type = <Package> -->
<Node>
<Variables>
<Variable Name="NewRows" DataType="Int32">0</Variable>
</Variables>
</Node>
Transformers syntax: Merge Mode

68. Demo

69. GET THINGS DONE KEEP GOING

70. What do you do next?

71. Biml on Monday...

72. @cathrinew
cathrinewilhelmsen.net
no.linkedin.com/in/cathrinewilhelmsen
contact@cathrinewilhelmsen.net
slideshare.net/cathrinewilhelmsen
Biml resources and references:
cathrinewilhelmsen.net/biml