Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
SQLDay 2018
Level Up Your Biml
Best Practices and Coding Techniques
Cathrine Wilhelmsen
@cathrinew
SQLDay 2018
GOLD SPONSORS
SILVER SPONSORS
BRONZE SPONSOR
STRATEGIC PARTNER
SQLDay 2018
Session Description
Is your Biml solution starting to remind you of a bowl of tangled spaghetti
code? Good! Th...
SQLDay 2018
Cathrine Wilhelmsen
@cathrinew cathrinew.net
Business Intelligence Consultant
SQLDay 2018
You…
…?
Know basic Biml and BimlScript
Created a staging environment using Biml
Created a metadata-driven Biml...
SQLDay 2018
…the next 60 minutes…
Practical Biml Coding C# Classes and Methods
Code Management
SQLDay 2018
Biml Tools
SQLDay 2018
Code
Management
SQLDay 2018
Don't Repeat Yourself
Centralize and reuse code
Update once in one file
1. Tiered Biml Files
2. Include Files
...
SQLDay 2018
Tiered Biml Files
Multiple Biml Files Working Together
SQLDay 2018
What are Tiered Biml Files?
Think of tiers as stacked layers or sequential steps
Tier (Layer) 1
Tier 0
Tier (L...
SQLDay 2018
Tiered Biml Files
Split Biml code in multiple files to:
• Solve logical dependencies
• Build solutions in mult...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Databases
Schemas
Ta...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Databases
Schemas
Tables
Packages
Connections
Load Packages
M...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Co...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Co...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Co...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Lo...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Lo...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Lo...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Lo...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Lo...
SQLDay 2018
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Lo...
SQLDay 2018
How do you use Tiered Biml Files?
1. Create Biml files with specified tiers
2. Select all the tiered Biml file...
SQLDay 2018
Annotations and ObjectTags
Store and Pass Metadata Between Biml Files
SQLDay 2018
Annotations and ObjectTags
Annotations and ObjectTags are Key / Value pairs
Annotations: String / String
Objec...
SQLDay 2018
Annotations
Create annotations:
<OleDbConnection Name="Dest">
<Annotations>
<Annotation Tag="Schema">stg</Anno...
SQLDay 2018
ObjectTags
Create ObjectTags:
RootNode.OleDbConnections["Dest"].ObjectTag["Filter"]
= new List<string>{"Produc...
SQLDay 2018
Include Files
Automated Copy & Paste
SQLDay 2018
Include Files
Include common code in multiple files and projects
Can include many file types: .biml .txt .sql ...
SQLDay 2018
Include Files
SQLDay 2018
Include Files
SQLDay 2018
Include Files
SQLDay 2018
Global Include Files
Automated Copy & Paste Everywhere
SQLDay 2018
Global Include Files
Use the global directive
<#@ global #>
Use instead of adding include directive to all fil...
SQLDay 2018
Global Include Files
Specify attributes to control global include files:
<#@ global
order="1"
location="top"
a...
SQLDay 2018
Global Include Files
SQLDay 2018
Global Include Files
SQLDay 2018
How do you use Global Include Files?
1. Create Global Include File
2. Select Biml files and Global Include Fil...
SQLDay 2018
CallBimlScript
Parameterized Control over Returned Code
SQLDay 2018
CallBimlScript
Control and limit the code returned
Works like a parameterized include (or stored procedure)
Ca...
SQLDay 2018
CallBimlScript
SQLDay 2018
CallBimlScript
SQLDay 2018
CallBimlScript
SQLDay 2018
CallBimlScript
SQLDay 2018
CallBimlScript
SQLDay 2018
CallBimlScriptWithOutput
Parameterized Control over Returned Code and Objects
SQLDay 2018
CallBimlScriptWithOutput
CallBimlScript file specifies custom output object:
<# CustomOutput.BimlComment = "Co...
SQLDay 2018
CallBimlScriptWithOutput
SQLDay 2018
CallBimlScriptWithOutput
SQLDay 2018
CallBimlScriptWithOutput
SQLDay 2018
CallBimlScriptWithOutput
SQLDay 2018
CallBimlScriptWithOutput
use CustomOutput
SQLDay 2018
Wait! When do you use what?
In larger projects, you often see a combination of
Tiered Biml Files, Include File...
SQLDay 2018
Include Files
Automated Copy & Paste
Code Included As-Is
CallBimlScript
Parameterized
Control Returned Code
Ti...
SQLDay 2018
DEMO
Code
Management
SQLDay 2018
Practical
Biml Coding
SQLDay 2018
LINQ
SQLDay 2018
LINQ (Language-Integrated Query)
One language to query:
SQL Server Databases
Datasets
Collections
Two ways to ...
SQLDay 2018
LINQ Extension Methods
Sort
OrderBy, ThenBy
Filter
Where, OfType
Group
GroupBy
Aggregate
Count, Sum
Check Coll...
SQLDay 2018
LINQ Extension Methods Example
var numConnections = RootNode.Connections.Count()
foreach (var table in RootNod...
SQLDay 2018
LINQ Extension Methods Example
foreach (var table in RootNode.Tables.Where(…))
if (RootNode.Packages.Any(…))
B...
SQLDay 2018
Lambda Expressions
"A lambda expression is an anonymous
function that you can use to create
delegates or expre...
SQLDay 2018
Lambda Expressions
…huh? o_O
SQLDay 2018
Lambda Expressions
table => table.Name == "Product"
SQLDay 2018
Lambda Expressions
table => table.Name == "Product"
The arrow is the lambda operator
SQLDay 2018
Lambda Expressions
table => table.Name == "Product"
Input parameter is on the left side
SQLDay 2018
Lambda Expressions
table => table.Name == "Product"
Expression is on the right side
SQLDay 2018
LINQ and Lambda
Chain LINQ Methods and use Lambda Expressions
for simple and powerful querying of collections:...
SQLDay 2018
LINQ: Filter collections
Where()
Returns the filtered collection with all elements that meet the criteria
Root...
SQLDay 2018
LINQ: Sort collections
OrderBy()
Returns the collection sorted by key…
RootNode.Tables.OrderBy(t => t.Name)
Th...
SQLDay 2018
LINQ: Sort collections
OrderByDescending()
Returns the collection sorted by key…
RootNode.Tables.OrderByDescen...
SQLDay 2018
LINQ: Sort collections
Reverse()
Returns the collection sorted in reverse order
RootNode.Tables.Reverse()
SQLDay 2018
LINQ: Group collections
GroupBy()
Returns a collection of key-value pairs where each value is a new collection...
SQLDay 2018
LINQ: Aggregate collections
Count()
Returns the number of elements in the collection
RootNode.Tables.Count()
R...
SQLDay 2018
LINQ: Aggregate collections
Sum()
Returns the sum of the (numeric) values in the collection
RootNode.Tables.Su...
SQLDay 2018
LINQ: Aggregate collections
Min()
Returns the minimum value of the (numeric) values in the collection
RootNode...
SQLDay 2018
LINQ: Check collections
All()
Returns true if all elements in the collection meet the criteria
RootNode.Databa...
SQLDay 2018
LINQ: Check collections
Contains()
Returns true if collection contains element
RootNode.Databases.Contains(Adv...
SQLDay 2018
LINQ: Get elements
First()
Returns the first element in the collection (that meets the criteria)
RootNode.Tabl...
SQLDay 2018
LINQ: Get elements
Last()
Returns the last element in the collection (that meets the criteria)
RootNode.Tables...
SQLDay 2018
LINQ: Get elements
ElementAt()
Returns the element in the collection at the specified index
RootNode.Tables.El...
SQLDay 2018
LINQ: Project collections
Select()
Creates a new collection from one collection
A list of table names:
RootNod...
SQLDay 2018
LINQ: Project collections
SelectMany()
Creates a new collection from many collections and merges the collectio...
SQLDay 2018
Preview Pane
The power is in the…
SQLDay 2018
BimlExpress Preview Pane
SQLDay 2018
BimlScript to Biml to SSIS
SQLDay 2018
BimlScript to Biml
Preview Pane
SQLDay 2018
BimlScript to …?
Preview Pane
SQLDay 2018
SQLDay 2018
DEMO
Practical
Biml Coding
SQLDay 2018
C# Classes
and Methods
SQLDay 2018
C# Classes and Methods
BimlScript and LINQ not enough?
Need to reuse C# code?
Create your own classes and meth...
SQLDay 2018
C# Classes and Methods
public static class Utilities {
public static string GetName(AstTableNode table) {
retu...
SQLDay 2018
C# Classes and Methods: Class
public static class Utilities {
public static string GetName(AstTableNode table)...
SQLDay 2018
C# Classes and Methods: Method
public static class Utilities {
public static string GetName(AstTableNode table...
SQLDay 2018
C# Classes and Methods: Logic
public static class Utilities {
public static string GetName(AstTableNode table)...
SQLDay 2018
C# Code Management
SQLDay 2018
Inline Code Blocks
<#+ ... #>
<#+ ... #>
SQLDay 2018
Included Biml Files with Code Blocks
<#@ include file="CodeBlock.biml" #>
<#+ ... #>
SQLDay 2018
Code Files
<#@ code file="Code.cs" #>
SQLDay 2018
Code Blocks vs. Code Files
Code Blocks
• Used when logic is specific to one file
• Reuse by including files wi...
SQLDay 2018
C# Extension Methods
SQLDay 2018
"Make it look like the method belongs
to an object instead of a helper class"
SQLDay 2018
Extension Methods: From this…
<#@ code file="HelperClass.cs" #>
<Biml xmlns="http://schemas.varigence.com/biml...
SQLDay 2018
Extension Methods: …to this
<#@ code file="HelperClass.cs" #>
<Biml xmlns="http://schemas.varigence.com/biml.x...
SQLDay 2018
Extension Methods: …to this
<#@ code file="HelperClass.cs" #>
<Biml xmlns="http://schemas.varigence.com/biml.x...
SQLDay 2018
Extension Methods: …to this :)
<#@ code file="HelperClass.cs" #>
<Biml xmlns="http://schemas.varigence.com/bim...
SQLDay 2018
DEMO
C# Classes
and Methods
SQLDay 2018
…the past 60 minutes…
Practical Biml Coding C# Classes and Methods
Code Management
SQLDay 2018
Where can you learn more?
cathrinew.net/BimlBook
SQLDay 2018
Evaluations
What can we do better? How can we improve?
:):( :|
SQLDay 2018
Get things done
Start small
Start simple
Start with ugly code
Keep going
Expand
Improve
Deliver often
SQLDay 2018
@cathrinew
cathrinew.net
hi@cathrinew.net
Biml resources and demo files:
cathrinew.net/biml
Upcoming SlideShare
Loading in …5
×

of

Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 1 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 2 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 3 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 4 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 5 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 6 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 7 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 8 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 9 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 10 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 11 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 12 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 13 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 14 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 15 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 16 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 17 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 18 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 19 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 20 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 21 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 22 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 23 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 24 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 25 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 26 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 27 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 28 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 29 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 30 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 31 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 32 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 33 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 34 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 35 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 36 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 37 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 38 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 39 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 40 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 41 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 42 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 43 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 44 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 45 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 46 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 47 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 48 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 49 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 50 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 51 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 52 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 53 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 54 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 55 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 56 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 57 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 58 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 59 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 60 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 61 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 62 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 63 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 64 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 65 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 66 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 67 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 68 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 69 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 70 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 71 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 72 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 73 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 74 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 75 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 76 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 77 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 78 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 79 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 80 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 81 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 82 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 83 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 84 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 85 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 86 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 87 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 88 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 89 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 90 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 91 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 92 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 93 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 94 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 95 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 96 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 97 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 98 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 99 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 100 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 101 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 102 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 103 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 104 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 105 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 106 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 107 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 108 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 109 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 110 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 111 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 112 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 113 Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018) Slide 114
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)

Download to read offline

Level Up Your Biml: Best Practices and Coding Techniques (Presented at SQLDay in Poland on May 16th 2018)

  • Be the first to like this

Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)

  1. 1. SQLDay 2018 Level Up Your Biml Best Practices and Coding Techniques Cathrine Wilhelmsen @cathrinew
  2. 2. SQLDay 2018 GOLD SPONSORS SILVER SPONSORS BRONZE SPONSOR STRATEGIC PARTNER
  3. 3. SQLDay 2018 Session Description Is your Biml solution starting to remind you of a bowl of tangled spaghetti code? Good! That means you are solving real problems while saving a lot of time. The next step is to make sure that your solution does not grow too complex and confusing – you do not want to waste all that saved time on future maintenance! Attend this session for an overview of Biml best practices and coding techniques. Learn how to centralize and reuse code with Include files and the CallBimlScript method. Make your code easier to read and write by utilizing LINQ (Language-Integrated Queries). Share code between files by using Annotations and ObjectTags. And finally, if standard Biml is not enough to solve your problems, you can create your own C# helper classes and extension methods to implement custom logic. Start improving your code today and level up your Biml in no time!
  4. 4. SQLDay 2018 Cathrine Wilhelmsen @cathrinew cathrinew.net Business Intelligence Consultant
  5. 5. SQLDay 2018 You… …? Know basic Biml and BimlScript Created a staging environment using Biml Created a metadata-driven Biml framework
  6. 6. SQLDay 2018 …the next 60 minutes… Practical Biml Coding C# Classes and Methods Code Management
  7. 7. SQLDay 2018 Biml Tools
  8. 8. SQLDay 2018 Code Management
  9. 9. SQLDay 2018 Don't Repeat Yourself Centralize and reuse code Update once in one file 1. Tiered Biml Files 2. Include Files 3. CallBimlScript
  10. 10. SQLDay 2018 Tiered Biml Files Multiple Biml Files Working Together
  11. 11. SQLDay 2018 What are Tiered Biml Files? Think of tiers as stacked layers or sequential steps Tier (Layer) 1 Tier 0 Tier (Layer) 2 Tier (Step) 1 Tier (Step) 2
  12. 12. SQLDay 2018 Tiered Biml Files Split Biml code in multiple files to: • Solve logical dependencies • Build solutions in multiple steps behind the scenes Specify the tier per file by using the template directive: <#@ template tier="2" #>
  13. 13. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Databases Schemas Tables Packages
  14. 14. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Databases Schemas Tables Packages Connections Load Packages Master Package
  15. 15. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Connections Load Packages Master Package
  16. 16. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Connections Load Packages Master Package
  17. 17. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Connections Load Packages Master Package
  18. 18. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Connections Load Packages Master Package
  19. 19. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Connections Load Packages Master Package
  20. 20. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Connections Load Packages Master Package
  21. 21. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Master Connections Load Packages Master Package
  22. 22. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Master Connections Load Packages Master Package
  23. 23. SQLDay 2018 Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Master
  24. 24. SQLDay 2018 How do you use Tiered Biml Files? 1. Create Biml files with specified tiers 2. Select all the tiered Biml files 3. Right-click and click Generate SSIS Packages 1 2 3
  25. 25. SQLDay 2018 Annotations and ObjectTags Store and Pass Metadata Between Biml Files
  26. 26. SQLDay 2018 Annotations and ObjectTags Annotations and ObjectTags are Key / Value pairs Annotations: String / String ObjectTags: String / Object Store metadata by attaching tags to Biml objects Higher tier files can get tags from lower tier files
  27. 27. SQLDay 2018 Annotations Create annotations: <OleDbConnection Name="Dest"> <Annotations> <Annotation Tag="Schema">stg</Annotation> </Annotations> </OleDbConnection> Use annotations: RootNode.OleDbConnections["Dest"].GetTag("Schema");
  28. 28. SQLDay 2018 ObjectTags Create ObjectTags: RootNode.OleDbConnections["Dest"].ObjectTag["Filter"] = new List<string>{"Product","ProductCategory"}; Use ObjectTags: RootNode.OleDbConnections["Dest"].ObjectTag["Filter"];
  29. 29. SQLDay 2018 Include Files Automated Copy & Paste
  30. 30. SQLDay 2018 Include Files Include common code in multiple files and projects Can include many file types: .biml .txt .sql .cs Use the include directive <#@ include file="CommonCode.biml" #> The directive will be replaced by the included file Works like an automated Copy & Paste
  31. 31. SQLDay 2018 Include Files
  32. 32. SQLDay 2018 Include Files
  33. 33. SQLDay 2018 Include Files
  34. 34. SQLDay 2018 Global Include Files Automated Copy & Paste Everywhere
  35. 35. SQLDay 2018 Global Include Files Use the global directive <#@ global #> Use instead of adding include directive to all files to • Create global variables • Include code files • Make VB default language
  36. 36. SQLDay 2018 Global Include Files Specify attributes to control global include files: <#@ global order="1" location="top" active="true" #>
  37. 37. SQLDay 2018 Global Include Files
  38. 38. SQLDay 2018 Global Include Files
  39. 39. SQLDay 2018 How do you use Global Include Files? 1. Create Global Include File 2. Select Biml files and Global Include File 3. Right-click and click Generate SSIS Packages 1 2 3
  40. 40. SQLDay 2018 CallBimlScript Parameterized Control over Returned Code
  41. 41. SQLDay 2018 CallBimlScript Control and limit the code returned Works like a parameterized include (or stored procedure) CallBimlScript file specifies accepted parameters: <#@ property name="Parameter" type="String" #> Main file calls and passes parameters: <#=CallBimlScript("CommonCode.biml", Parameter)#>
  42. 42. SQLDay 2018 CallBimlScript
  43. 43. SQLDay 2018 CallBimlScript
  44. 44. SQLDay 2018 CallBimlScript
  45. 45. SQLDay 2018 CallBimlScript
  46. 46. SQLDay 2018 CallBimlScript
  47. 47. SQLDay 2018 CallBimlScriptWithOutput Parameterized Control over Returned Code and Objects
  48. 48. SQLDay 2018 CallBimlScriptWithOutput CallBimlScript file specifies custom output object: <# CustomOutput.BimlComment = "Comment"; #> Main file defines, passes and uses output object: <# dynamic outObj; #> <#=CallBimlScriptWithOutput("CommonCode.biml", out outObj)#> <#=outObj.BimlComment#>
  49. 49. SQLDay 2018 CallBimlScriptWithOutput
  50. 50. SQLDay 2018 CallBimlScriptWithOutput
  51. 51. SQLDay 2018 CallBimlScriptWithOutput
  52. 52. SQLDay 2018 CallBimlScriptWithOutput
  53. 53. SQLDay 2018 CallBimlScriptWithOutput use CustomOutput
  54. 54. SQLDay 2018 Wait! When do you use what? In larger projects, you often see a combination of Tiered Biml Files, Include Files, and CallBimlScript Rule of Thumb: If you reuse code more than 3 times, refactor so you Don't Repeat Yourself
  55. 55. SQLDay 2018 Include Files Automated Copy & Paste Code Included As-Is CallBimlScript Parameterized Control Returned Code Tiered Biml Files Solve Dependencies Multi-Step Builds
  56. 56. SQLDay 2018 DEMO Code Management
  57. 57. SQLDay 2018 Practical Biml Coding
  58. 58. SQLDay 2018 LINQ
  59. 59. SQLDay 2018 LINQ (Language-Integrated Query) One language to query: SQL Server Databases Datasets Collections Two ways to write queries: SQL-ish Syntax Extension Methods
  60. 60. SQLDay 2018 LINQ Extension Methods Sort OrderBy, ThenBy Filter Where, OfType Group GroupBy Aggregate Count, Sum Check Collections All, Any, Contains Get Elements First, Last, ElementAt Project Collections Select, SelectMany
  61. 61. SQLDay 2018 LINQ Extension Methods Example var numConnections = RootNode.Connections.Count() foreach (var table in RootNode.Tables.Where(…)) if (RootNode.Packages.Any(…))
  62. 62. SQLDay 2018 LINQ Extension Methods Example foreach (var table in RootNode.Tables.Where(…)) if (RootNode.Packages.Any(…)) But what do you put in here?
  63. 63. SQLDay 2018 Lambda Expressions "A lambda expression is an anonymous function that you can use to create delegates or expression tree types"
  64. 64. SQLDay 2018 Lambda Expressions …huh? o_O
  65. 65. SQLDay 2018 Lambda Expressions table => table.Name == "Product"
  66. 66. SQLDay 2018 Lambda Expressions table => table.Name == "Product" The arrow is the lambda operator
  67. 67. SQLDay 2018 Lambda Expressions table => table.Name == "Product" Input parameter is on the left side
  68. 68. SQLDay 2018 Lambda Expressions table => table.Name == "Product" Expression is on the right side
  69. 69. SQLDay 2018 LINQ and Lambda Chain LINQ Methods and use Lambda Expressions for simple and powerful querying of collections: .Where(table => table.Schema.Name == "Production") .OrderBy(table => table.Name)
  70. 70. SQLDay 2018 LINQ: Filter collections Where() Returns the filtered collection with all elements that meet the criteria RootNode.Tables.Where(t => t.Schema.Name == "Production") OfType() Returns the filtered collection with all elements of the specified type RootNode.Connections.OfType<AstExcelOleDbConnectionNode>()
  71. 71. SQLDay 2018 LINQ: Sort collections OrderBy() Returns the collection sorted by key… RootNode.Tables.OrderBy(t => t.Name) ThenBy() …then sorted by secondary key RootNode.Tables.OrderBy(t => t.Schema.Name) .ThenBy(t => t.Name)
  72. 72. SQLDay 2018 LINQ: Sort collections OrderByDescending() Returns the collection sorted by key… RootNode.Tables.OrderByDescending(t => t.Name) ThenByDescending() …then sorted by secondary key RootNode.Tables.OrderBy(t => t.Schema.Name) .ThenByDescending(t => t.Name)
  73. 73. SQLDay 2018 LINQ: Sort collections Reverse() Returns the collection sorted in reverse order RootNode.Tables.Reverse()
  74. 74. SQLDay 2018 LINQ: Group collections GroupBy() Returns a collection of key-value pairs where each value is a new collection RootNode.Tables.GroupBy(t => t.Schema.Name)
  75. 75. SQLDay 2018 LINQ: Aggregate collections Count() Returns the number of elements in the collection RootNode.Tables.Count() RootNode.Tables.Count(t => t.Schema.Name == "Production")
  76. 76. SQLDay 2018 LINQ: Aggregate collections Sum() Returns the sum of the (numeric) values in the collection RootNode.Tables.Sum(t => t.Columns.Count) Average() Returns the average value of the (numeric) values in the collection RootNode.Tables.Average(t => t.Columns.Count)
  77. 77. SQLDay 2018 LINQ: Aggregate collections Min() Returns the minimum value of the (numeric) values in the collection RootNode.Tables.Min(t => t.Columns.Count) Max() Returns the maximum value of the (numeric) values in the collection RootNode.Tables.Max(t => t.Columns.Count)
  78. 78. SQLDay 2018 LINQ: Check collections All() Returns true if all elements in the collection meet the criteria RootNode.Databases.All(d => d.Name.StartsWith("A")) Any() Returns true if any element in the collection meets the criteria RootNode.Databases.Any(d => d.Name.Contains("DW"))
  79. 79. SQLDay 2018 LINQ: Check collections Contains() Returns true if collection contains element RootNode.Databases.Contains(AdventureWorks2014)
  80. 80. SQLDay 2018 LINQ: Get elements First() Returns the first element in the collection (that meets the criteria) RootNode.Tables.First() RootNode.Tables.First(t => t.Schema.Name == "Production") FirstOrDefault() Returns the first element in the collection or default value (that meets the criteria) RootNode.Tables.FirstOrDefault() RootNode.Tables.FirstOrDefault(t => t.Schema.Name == "Production")
  81. 81. SQLDay 2018 LINQ: Get elements Last() Returns the last element in the collection (that meets the criteria) RootNode.Tables.Last() RootNode.Tables.Last(t => t.Schema.Name == "Production") LastOrDefault() Returns the last element in the collection or default value (that meets the criteria) RootNode.Tables.LastOrDefault() RootNode.Tables.LastOrDefault(t => t.Schema.Name == "Production")
  82. 82. SQLDay 2018 LINQ: Get elements ElementAt() Returns the element in the collection at the specified index RootNode.Tables.ElementAt(42) ElementAtOrDefault() Returns the element in the collection or default value at the specified index RootNode.Tables.ElementAtOrDefault(42)
  83. 83. SQLDay 2018 LINQ: Project collections Select() Creates a new collection from one collection A list of table names: RootNode.Tables.Select(t => t.Name) A list of table and schema names: RootNode.Tables.Select(t => new {t.Name, t.Schema.Name})
  84. 84. SQLDay 2018 LINQ: Project collections SelectMany() Creates a new collection from many collections and merges the collections A list of all columns from all tables: RootNode.Tables.SelectMany(t => t.Columns)
  85. 85. SQLDay 2018 Preview Pane The power is in the…
  86. 86. SQLDay 2018 BimlExpress Preview Pane
  87. 87. SQLDay 2018 BimlScript to Biml to SSIS
  88. 88. SQLDay 2018 BimlScript to Biml Preview Pane
  89. 89. SQLDay 2018 BimlScript to …? Preview Pane
  90. 90. SQLDay 2018
  91. 91. SQLDay 2018 DEMO Practical Biml Coding
  92. 92. SQLDay 2018 C# Classes and Methods
  93. 93. SQLDay 2018 C# Classes and Methods BimlScript and LINQ not enough? Need to reuse C# code? Create your own classes and methods!
  94. 94. SQLDay 2018 C# Classes and Methods public static class Utilities { public static string GetName(AstTableNode table) { return table.Schema.Name.ToUpper() + "_" + table.Name.ToUpper(); } }
  95. 95. SQLDay 2018 C# Classes and Methods: Class public static class Utilities { public static string GetName(AstTableNode table) { return table.Schema.Name.ToUpper() + "_" + table.Name.ToUpper(); } }
  96. 96. SQLDay 2018 C# Classes and Methods: Method public static class Utilities { public static string GetName(AstTableNode table) { return table.Schema.Name.ToUpper() + "_" + table.Name.ToUpper(); } }
  97. 97. SQLDay 2018 C# Classes and Methods: Logic public static class Utilities { public static string GetName(AstTableNode table) { return table.Schema.Name.ToUpper() + "_" + table.Name.ToUpper(); } }
  98. 98. SQLDay 2018 C# Code Management
  99. 99. SQLDay 2018 Inline Code Blocks <#+ ... #> <#+ ... #>
  100. 100. SQLDay 2018 Included Biml Files with Code Blocks <#@ include file="CodeBlock.biml" #> <#+ ... #>
  101. 101. SQLDay 2018 Code Files <#@ code file="Code.cs" #>
  102. 102. SQLDay 2018 Code Blocks vs. Code Files Code Blocks • Used when logic is specific to one file • Reuse by including files with code blocks Code Files • Use code editor of choice • Create extension methods
  103. 103. SQLDay 2018 C# Extension Methods
  104. 104. SQLDay 2018 "Make it look like the method belongs to an object instead of a helper class"
  105. 105. SQLDay 2018 Extension Methods: From this… <#@ code file="HelperClass.cs" #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <# foreach (var table in RootNode.Tables) { #> <# if (HelperClass.AnnotationTagExists(table, "SourceSchema")) { #> ... <# } #> <# } #> </Biml> public static class HelperClass { public static bool AnnotationTagExists(AstNode node, string tag) { return (node.GetTag(tag) != "") ? true : false; } }
  106. 106. SQLDay 2018 Extension Methods: …to this <#@ code file="HelperClass.cs" #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <# foreach (var table in RootNode.Tables) { #> <# if (HelperClass.AnnotationTagExists(table, "SourceSchema")) { #> ... <# } #> <# } #> </Biml> public static class HelperClass { public static bool AnnotationTagExists(this AstNode node, string tag) { return (node.GetTag(tag) != "") ? true : false; } }
  107. 107. SQLDay 2018 Extension Methods: …to this <#@ code file="HelperClass.cs" #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <# foreach (var table in RootNode.Tables) { #> <# if (table.AnnotationTagExists("SourceSchema")) { #> ... <# } #> <# } #> </Biml> public static class HelperClass { public static bool AnnotationTagExists(this AstNode node, string tag) { return (node.GetTag(tag) != "") ? true : false; } }
  108. 108. SQLDay 2018 Extension Methods: …to this :) <#@ code file="HelperClass.cs" #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <# foreach (var table in RootNode.Tables.Where(t => t.AnnotationTagExists("SourceSchema")) { #> ... <# } #> <# } #> </Biml> public static class HelperClass { public static bool AnnotationTagExists(this AstNode node, string tag) { return (node.GetTag(tag) != "") ? true : false; } }
  109. 109. SQLDay 2018 DEMO C# Classes and Methods
  110. 110. SQLDay 2018 …the past 60 minutes… Practical Biml Coding C# Classes and Methods Code Management
  111. 111. SQLDay 2018 Where can you learn more? cathrinew.net/BimlBook
  112. 112. SQLDay 2018 Evaluations What can we do better? How can we improve? :):( :|
  113. 113. SQLDay 2018 Get things done Start small Start simple Start with ugly code Keep going Expand Improve Deliver often
  114. 114. SQLDay 2018 @cathrinew cathrinew.net hi@cathrinew.net Biml resources and demo files: cathrinew.net/biml

Level Up Your Biml: Best Practices and Coding Techniques (Presented at SQLDay in Poland on May 16th 2018)

Views

Total views

842

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

8

Shares

0

Comments

0

Likes

0

×