Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL Server Days)

3,978 views

Published on

Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (Presented at SQL Server Days Belgium on September 29th 2015)

Published in: Technology
  • Be the first to comment

Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL Server Days)

  1. 1. Don't Repeat Yourself - Agile SSIS development with Biml and BimlScript Cathrine Wilhelmsen SQL Server Days 2015
  2. 2. SQL Server Days would like to thank all of our sponsors!
  3. 3. Session Description
  4. 4. Cathrine Wilhelmsen @cathrinew cathrinewilhelmsen.net Data Warehouse Architect Business Intelligence Developer
  5. 5. Who are you? * ( * Probably not a cat)
  6. 6. Your work day? (Drag, drop, drag, drop, connect, drag, drop, connect, resize, align, drag, drop, resize, connect, align…)
  7. 7. job done! new standards ...yay Ever experienced this?
  8. 8. Ready for a change?
  9. 9. From Traditional SSIS to Agile SSIS
  10. 10. Traditional SSIS: Plumbing
  11. 11. Agile SSIS: Business Logic
  12. 12. Will Biml solve all your problems?
  13. 13. How can Biml help you?
  14. 14. What is Business Intelligence Markup Language?
  15. 15. What do you need?
  16. 16. ...or you can invest in
  17. 17. How does it work?
  18. 18. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml> Biml syntax
  19. 19. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml> Biml syntax: Root Element
  20. 20. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml> Biml syntax: Collection of Elements
  21. 21. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml> Biml syntax: Elements
  22. 22. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml> Biml syntax: Attributes
  23. 23. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml> Biml syntax: Full vs. Shorthand Syntax
  24. 24. Let's generate some packages
  25. 25. Right-click on SSIS project and click Add New Biml File
  26. 26. Biml files are placed in the Miscellaneous folder <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  27. 27. Right-click on the Biml file to Generate SSIS Packages <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  28. 28. From Biml to SSIS
  29. 29. From Biml to SSIS
  30. 30. .biml vs .dtsx: human-readable vs ALL THE CODE! (150% zoom) (20% zoom)
  31. 31. I create SSIS packages faster than that
  32. 32. But wait!
  33. 33. The magic is in the
  34. 34. Import Loop Add expressions What is ?
  35. 35. BimlScript code blocks <#@ … #> <# … #> <#= … #> <#+ … #>
  36. 36. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml> BimlScript syntax
  37. 37. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml> BimlScript syntax: Control Blocks
  38. 38. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml> BimlScript syntax: Expression Control Block
  39. 39. How does it work?
  40. 40. Yes, but how does it work? 41
  41. 41. Yes, but how does it actually work? <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="LoadCustomer"></Package> <Package Name="LoadProduct"></Package> <Package Name="LoadSales"></Package> </Packages> </Biml>
  42. 42. Don't Repeat Yourself
  43. 43. Don't Repeat Yourself
  44. 44. Don't Repeat Yourself
  45. 45. <#@ template tier="1" #> Don't Repeat Yourself: Split and combine Biml files
  46. 46. Split and combine multiple Biml files 1 2 3
  47. 47. <#@ include file="CommonCode.biml" #> Don't Repeat Yourself: Include files
  48. 48. Don't Repeat Yourself: Include files
  49. 49. Don't Repeat Yourself: Include files
  50. 50. Don't Repeat Yourself: Include files
  51. 51. <#@ property name="Table" type="AstTableNode" #> <#=CallBimlScript("CommonCode.biml", Table)#> Don't Repeat Yourself: CallBimlScript with parameters
  52. 52. Don't Repeat Yourself: CallBimlScript with parameters
  53. 53. Don't Repeat Yourself: CallBimlScript with parameters
  54. 54. Don't Repeat Yourself: CallBimlScript with parameters
  55. 55. Don't Repeat Yourself: CallBimlScript with parameters
  56. 56. Don't Repeat Yourself: CallBimlScript with parameters
  57. 57. C# Classes and Methods
  58. 58. <#+ public class HelperClass { public static string HelperMethod(string InputParameter) { return string.Concat("InputParameter: ", InputParameter); } } #> C# Classes and Methods
  59. 59. <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <#=HelperClass.HelperMethod("Cats")#> </Biml> <#+ public class HelperClass { public static string HelperMethod(string InputParameter) { return string.Concat("InputParameter: ", InputParameter); } } #> C# Classes and Methods: Inline
  60. 60. <#@ include file="HelperClass.biml" #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <#=HelperClass.HelperMethod("Cats")#> </Biml> C# Classes and Methods: Included <#+ public class HelperClass { public static string HelperMethod(string InputParameter) { return string.Concat("InputParameter: ", InputParameter); } } #>
  61. 61. <#@ code file="..CodeHelperClass.cs" #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <#=HelperClass.HelperMethod("Cats")#> </Biml> C# Classes and Methods: Mist / BimlStudio using System.Data; public class HelperClass { public static string HelperMethod(string InputParameter) { return string.Concat("InputParameter: ", InputParameter); } }
  62. 62. Transformers (No, not those transformers, but still awesome!)
  63. 63. Transformers: Modify existing Biml
  64. 64. <#@ target type="Package" mergemode="LocalMerge" #> <!–- <Node> = Target Type = <Package> --> <Node> <Variables> <Variable Name="NewRows" DataType="Int32">0</Variable> </Variables> </Node> Transformers syntax
  65. 65. <#@ target type="Package" mergemode="LocalMerge" #> <!–- <Node> = Target Type = <Package> --> <Node> <Variables> <Variable Name="NewRows" DataType="Int32">0</Variable> </Variables> </Node> Transformers syntax: Target Type
  66. 66. <#@ target type="Package" mergemode="LocalMerge" #> <!–- <Node> = Target Type = <Package> --> <Node> <Variables> <Variable Name="NewRows" DataType="Int32">0</Variable> </Variables> </Node> Transformers syntax: Merge Mode
  67. 67. Demo
  68. 68. GET THINGS DONE KEEP GOING
  69. 69. What do you do next?
  70. 70. Biml on Monday...
  71. 71. @cathrinew cathrinewilhelmsen.net no.linkedin.com/in/cathrinewilhelmsen contact@cathrinewilhelmsen.net slideshare.net/cathrinewilhelmsen Biml resources and references: cathrinewilhelmsen.net/biml

×