SlideShare a Scribd company logo
1 of 98
Download to read offline
Biml for Beginners:
Script and Automate
SSIS Development
Cathrine Wilhelmsen
SQLSaturday Chicago · March 17th 2018
Session Description
Are you tired of creating and updating the same SSIS packages again and again? Is your wrist hurting
from all that clicking, dragging, dropping, connecting and aligning? Do you want to take the next step and
really speed up your SSIS development?
Say goodbye to repetitive work and hello to Biml, the markup language for Business Intelligence projects.
In this session we will look at the basics of Biml. First learn how to use Biml to generate SSIS packages
from database metadata. Then see how you can reuse code to implement changes in multiple SSIS
packages and projects with just a few clicks. Finally, we will create an example project that you can
download and start with to speed up your SSIS development from day one.
Stop wasting your valuable time on doing the same things over and over and over again, and see how
you can complete in a day what once took more than a week!
Biml Basics Tools & Code
SQL and SSIS from Metadata
…the next 60 minutes…
Cathrine Wilhelmsen
@cathrinew cathrinew.net
Data Warehouse & Business Intelligence Consultant
You…
…?
SSIS developer
Easily bored
Tired of repetitive work
Work…
…?
Long development time
Many SSIS packages
Frequent requirement changes
job done!
new standards
…yay
Ever experienced this?
It's time for a change!
What is Biml?
Business Intelligence Markup Language
Easy to read and write XML language
Describes business intelligence objects:
• Databases, Schemas, Tables, Views, Columns
• SSIS Packages
• SSAS Cubes
Why use Biml?
SSIS: Plumbing Biml: Business Logic
SSIS: Plumbing
Spend time on dragging, dropping, connecting, aligning
Create the same package over and over again with minor changes
Standards, patterns and templates must be defined up-front
Changes must be done in every single package
Higher risk of manual errors
More packages, more time
Biml: Business Logic
Spend time on what is unique in a package
Create a pattern once and reuse for all similar packages
Handle scope and requirement changes quickly and easily
Changes can be applied to all packages at once
Lower risk of manual errors
Longer time to start, but then automate and reuse
Will Biml solve all your challenges?
Probably not...
Biml is a tool for generating SSIS packages
Biml is not a pre-defined ETL framework
Biml is not a tool for automated deployment
...but it will solve many challenges!
How can Biml help you?
Biml is great for large projects with common patterns…
Timesaving: Many SSIS packages from one Biml file
Reusable: Write once and run on any platform
Flexible: Start simple, expand as you learn
…but is also useful for smaller projects!
What do you need?
BimlExpress
Free add-in for Visual Studio
Code editor with syntax highlighting and Biml Intellisense
More frequent updates than BIDS Helper
varigence.com/bimlexpress
BimlOnline
Free browser-based Biml editor
Code editor with Biml and C# Intellisense
Reverse-engineer from SSIS to Biml
bimlonline.com
BimlStudio
Licensed full-featured development environment for Biml
Visual designer and metadata modeling
Full-stack automation and transformers
varigence.com/bimlstudio
How does it work?
…generated packages look exactly like manually created packages
Biml syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Root Element
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Collections of Elements
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Elements
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Attributes
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Biml syntax: Full vs. Shorthand Syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
Let's generate
some packages!
Goal: Load from source to staging
Add New Biml File from BimlExpress menu…
…or right-click on SSIS project to Add New Biml File
Biml files are placed under Miscellaneous
Check Biml For Errors from BimlExpress menu…
…or right-click on file to Check Biml For Errors
Generate SSIS Packages from BimlExpress menu…
…or right-click on file to Generate SSIS Packages
From Biml to SSIS: Control Flow
<Package Name="TruncateLoad" ConstraintMode="Linear">
<Tasks>
<ExecuteSQL Name="Truncate Table" ConnectionName="Staging">
<DirectInput>TRUNCATE TABLE DestinationTable</DirectInput>
</ExecuteSQL>
<Dataflow Name="Load Table">
<Transformations>...</Transformations>
</Dataflow>
</Tasks>
</Package>
From Biml to SSIS: Data Flow
<Transformations>
<OleDbSource Name="Source" ConnectionName="AW2014">
<ExternalTableInput Table="SourceTable" />
</OleDbSource>
<DerivedColumns Name="Add LoadDate">
<Columns>
<Column Name="LoadDate" DataType="DateTime">
@[System::StartTime]
</Column>
</Columns>
</DerivedColumns>
<OleDbDestination Name="Destination" ConnectionName="Staging">
<ExternalTableOutput Table="DestinationTable" />
</OleDbDestination>
</Transformations>
.biml vs .dtsx: human-readable vs ALL THE CODE!
(20% zoom)(150% zoom)
Ok, so we can go from Biml to SSIS…
…can we go from SSIS to Biml?
Yes!
Let's convert some packages
from SSIS to Biml!
Convert SSIS to Biml in BimlExpress 2018
Select Package or project…
…import, filter and add to project
The magic is in the…
What is BimlScript?
Extend Biml with C# or VB code blocks
Import database structure and metadata
Loop over tables and columns
Expressions replace static values
Generate, control and manipulate Biml code
BimlScript Code Blocks
<# … #> Control Block (Control logic)
<#= … #> Text Block (Returns string)
<#@ … #> Directive (Compiler instructions)
<#+ … #> Class Block (Create C# classes)
<#* … *#> Comment Block (Comments)
BimlScript Syntax
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetDatabaseSchema(); #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in metadata.TableNodes) { #>
<Package Name="Load_<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
BimlScript Syntax: Import metadata
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetDatabaseSchema(); #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in metadata.TableNodes) { #>
<Package Name="Load_<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
BimlScript Syntax: Loop over tables
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetDatabaseSchema(); #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in metadata.TableNodes) { #>
<Package Name="Load_<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
BimlScript Syntax: Replace static values
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetDatabaseSchema(); #>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in metadata.TableNodes) { #>
<Package Name="Load_<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
Biml vs. BimlScript
Generate, control and
manipulate Biml with C#
XML Language
"Just plain text"
How does it work?
Yes, but how does it work?
Yes, but how does it actually work?
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load_<#=table.Name#>" />
<# } #>
<Package Name="Load_Customer" />
<Package Name="Load_Product" />
<Package Name="Load_Sales" />
Let's generate
many packages!
Of course I can create 200 SSIS Packages!
…what do you need me to do after lunch?
Don't Repeat Yourself
Move code to separate files
Centralize and reuse
Update once for all projects
1. Tiered Biml Files
2. Include Files
3. CallBimlScript
Tiered Biml Files
What are Tiered Biml Files?
Think of tiers as stacked layers or sequential steps
Tier (Layer) 1
Tier 0
Tier (Layer) 2
Tier (Step) 1 Tier (Step) 2
Tiered Biml Files
Split Biml code in multiple files to:
• Solve logical dependencies
• Build solutions in multiple steps behind the scenes
Specify the tier per file by using the template directive:
<#@ template tier="2" #>
Tiered Biml Files
Biml is compiled step-by-step from lowest to highest tier
• Biml files are implicitly tier 0
• BimlScript files are implicitly tier 1
In each tier, objects are added in-memory to RootNode
Higher tiers can call RootNode to use objects from lower tiers
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Databases
Schemas
Tables
Packages
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Databases
Schemas
Tables
Packages
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Load_Customer
Load_Product
Load_Sales
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Load_Customer
Load_Product
Load_Sales
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Load_Customer
Load_Product
Load_Sales
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Load_Customer
Load_Product
Load_Sales
Master
Tiered Biml Files: Behind the Scenes
Connections
Load Packages
Master Package
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Load_Customer
Load_Product
Load_Sales
Master
Tiered Biml Files: Behind the Scenes
Connections
Admin
Source
Destination
Databases
Schemas
Tables
Packages
Load_Customer
Load_Product
Load_Sales
Master
How do you use Tiered Biml Files?
1. Create Biml files with specified tiers
2. Select all the tiered Biml files
3. Right-click and click Generate SSIS Packages
1
2
3
Annotations
and ObjectTags
Annotations and ObjectTags
Annotations and ObjectTags are Key / Value pairs
Use them to store and pass code between tiers
Annotations: String / String
ObjectTags: String / Object
Annotations
Create annotations:
<OleDbConnection Name="Dest">
<Annotations>
<Annotation Tag="Schema">aw</Annotation>
</Annotations>
</OleDbConnection>
Use annotations:
RootNode.OleDbConnections["Dest"].GetTag("Schema");
ObjectTags
Create ObjectTags:
RootNode.ObjectTag["Filter"] =
new List<string>{"Product","ProductCategory"};
Use ObjectTags:
RootNode.ObjectTag["Filter"];
Include Files
Include Files
Include common code in multiple files and projects
Can include many file types: .biml .txt .sql .cs
Use the include directive
<#@ include file="CommonCode.biml" #>
The directive will be replaced by the included file
Works like an automated Copy & Paste
Include Files
Include Files
Include Files
CallBimlScript
CallBimlScript
Works like a parameterized include (or stored procedure)
File to be called (callee) specifies accepted parameters:
<#@ property name="Parameter" type="String" #>
File that calls (caller) passes parameters:
<#=CallBimlScript("Common.biml", Parameter)#>
CallBimlScript
CallBimlScript
CallBimlScript
CallBimlScript
CallBimlScript
How does all this work?
Biml Basics Tools & Code
SQL and SSIS from Metadata
…the past 60 minutes…
Where can I learn more?
Free online training
bimlscript.com
Where can I learn more?
cathrinew.net/BimlBook
Get things done
Start small
Start simple
Start with ugly code
Keep going
Expand
Improve
Deliver often
Biml on Monday…
…BimlBreak the rest of the week ☺
@cathrinew
cathrinew.net
linkedin.com/in/cathrinewilhelmsen
hi@cathrinew.net
slideshare.net/cathrinewilhelmsen
Biml resources and references:
cathrinew.net/biml

More Related Content

What's hot

Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)
Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)
Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)Cathrine Wilhelmsen
 
Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...
Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...
Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...Cathrine Wilhelmsen
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)Cathrine Wilhelmsen
 
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Cathrine Wilhelmsen
 
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...Cathrine Wilhelmsen
 
Biml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into BimlBiml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into BimlCathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)
Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)
Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)Cathrine Wilhelmsen
 
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...Cathrine Wilhelmsen
 
Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)
Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)
Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...Cathrine Wilhelmsen
 

What's hot (20)

Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
 
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
 
Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)
Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)
Level Up Your Biml: Best Practices and Coding Techniques (PASS Summit 2018)
 
Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...
Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...
Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours o...
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Sacrame...
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
 
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
 
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
 
Biml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into BimlBiml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into Biml
 
Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)
Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)
Level Up Your Biml: Best Practices and Coding Techniques (NTK 2016)
 
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
 
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
 
Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)
Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)
Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Oslo)
 
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
 
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Oslo)
 
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
 

Similar to Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)

Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Cathrine Wilhelmsen
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)Cathrine Wilhelmsen
 
EnterJS 2015 - Continuous Integration for Frontend Code
EnterJS 2015 - Continuous Integration for Frontend CodeEnterJS 2015 - Continuous Integration for Frontend Code
EnterJS 2015 - Continuous Integration for Frontend CodeMarcel Birkner
 
Turku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-componentsTurku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-componentsJames Stone
 
Sql bits creating a meta data driven ssis solution with biml
Sql bits   creating a meta data driven ssis solution with bimlSql bits   creating a meta data driven ssis solution with biml
Sql bits creating a meta data driven ssis solution with bimlMarco Schreuder
 
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...Cathrine Wilhelmsen
 
MongoDB.local Dallas 2019: MongoDB Stitch Tutorial
MongoDB.local Dallas 2019: MongoDB Stitch TutorialMongoDB.local Dallas 2019: MongoDB Stitch Tutorial
MongoDB.local Dallas 2019: MongoDB Stitch TutorialMongoDB
 
MongoDB.local Atlanta: MongoDB Stitch Tutorial
MongoDB.local Atlanta: MongoDB Stitch TutorialMongoDB.local Atlanta: MongoDB Stitch Tutorial
MongoDB.local Atlanta: MongoDB Stitch TutorialMongoDB
 
Stencil the time for vanilla web components has arrived
Stencil the time for vanilla web components has arrivedStencil the time for vanilla web components has arrived
Stencil the time for vanilla web components has arrivedGil Fink
 
MongoDB.local Seattle 2019: MongoDB Stitch Tutorial
MongoDB.local Seattle 2019: MongoDB Stitch TutorialMongoDB.local Seattle 2019: MongoDB Stitch Tutorial
MongoDB.local Seattle 2019: MongoDB Stitch TutorialMongoDB
 
Build Your Own Instagram Filters
Build Your Own Instagram FiltersBuild Your Own Instagram Filters
Build Your Own Instagram FiltersTJ Stalcup
 

Similar to Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago) (15)

Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
 
Dbi h315
Dbi h315Dbi h315
Dbi h315
 
EnterJS 2015 - Continuous Integration for Frontend Code
EnterJS 2015 - Continuous Integration for Frontend CodeEnterJS 2015 - Continuous Integration for Frontend Code
EnterJS 2015 - Continuous Integration for Frontend Code
 
Turku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-componentsTurku loves-storybook-styleguidist-styled-components
Turku loves-storybook-styleguidist-styled-components
 
Sql bits creating a meta data driven ssis solution with biml
Sql bits   creating a meta data driven ssis solution with bimlSql bits   creating a meta data driven ssis solution with biml
Sql bits creating a meta data driven ssis solution with biml
 
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
 
MongoDB.local Dallas 2019: MongoDB Stitch Tutorial
MongoDB.local Dallas 2019: MongoDB Stitch TutorialMongoDB.local Dallas 2019: MongoDB Stitch Tutorial
MongoDB.local Dallas 2019: MongoDB Stitch Tutorial
 
MongoDB.local Atlanta: MongoDB Stitch Tutorial
MongoDB.local Atlanta: MongoDB Stitch TutorialMongoDB.local Atlanta: MongoDB Stitch Tutorial
MongoDB.local Atlanta: MongoDB Stitch Tutorial
 
Stencil the time for vanilla web components has arrived
Stencil the time for vanilla web components has arrivedStencil the time for vanilla web components has arrived
Stencil the time for vanilla web components has arrived
 
MongoDB.local Seattle 2019: MongoDB Stitch Tutorial
MongoDB.local Seattle 2019: MongoDB Stitch TutorialMongoDB.local Seattle 2019: MongoDB Stitch Tutorial
MongoDB.local Seattle 2019: MongoDB Stitch Tutorial
 
Ad505 dev blast
Ad505 dev blastAd505 dev blast
Ad505 dev blast
 
Build Your Own Instagram Filters
Build Your Own Instagram FiltersBuild Your Own Instagram Filters
Build Your Own Instagram Filters
 

More from Cathrine Wilhelmsen

Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Cathrine Wilhelmsen
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Cathrine Wilhelmsen
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Cathrine Wilhelmsen
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Cathrine Wilhelmsen
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)Cathrine Wilhelmsen
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Cathrine Wilhelmsen
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Cathrine Wilhelmsen
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Cathrine Wilhelmsen
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...Cathrine Wilhelmsen
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Cathrine Wilhelmsen
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)Cathrine Wilhelmsen
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
 

Recently uploaded

Role of Consumer Insights in business transformation
Role of Consumer Insights in business transformationRole of Consumer Insights in business transformation
Role of Consumer Insights in business transformationAnnie Melnic
 
Adobe Scan 06-Mar-2024 (1).pdf shavashwvw
Adobe Scan 06-Mar-2024 (1).pdf shavashwvwAdobe Scan 06-Mar-2024 (1).pdf shavashwvw
Adobe Scan 06-Mar-2024 (1).pdf shavashwvws73678sri
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelBoston Institute of Analytics
 
Adobe Scan 06-Mar-2024 (1).pdfwvsbbsbsba
Adobe Scan 06-Mar-2024 (1).pdfwvsbbsbsbaAdobe Scan 06-Mar-2024 (1).pdfwvsbbsbsba
Adobe Scan 06-Mar-2024 (1).pdfwvsbbsbsbas73678sri
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaManalVerma4
 
Digital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfDigital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfNicoChristianSunaryo
 
Film cover research.pptx for media courseowrk
Film cover research.pptx for media courseowrkFilm cover research.pptx for media courseowrk
Film cover research.pptx for media courseowrk494f574xmv
 
Rabobank_Exploring the Impact of Graph Technology on Financial Services.pdf
Rabobank_Exploring the Impact of Graph Technology on Financial Services.pdfRabobank_Exploring the Impact of Graph Technology on Financial Services.pdf
Rabobank_Exploring the Impact of Graph Technology on Financial Services.pdfNeo4j
 
Inference rules in artificial intelligence
Inference rules in artificial intelligenceInference rules in artificial intelligence
Inference rules in artificial intelligencePriyadharshiniG41
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...Jack Cole
 
prediction of default payment next month using a logistic approach
prediction of default payment next month using a logistic approachprediction of default payment next month using a logistic approach
prediction of default payment next month using a logistic approachAdekunleJoseph4
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...Dr Arash Najmaei ( Phd., MBA, BSc)
 
Statistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfStatistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfnikeshsingh56
 
DATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etcDATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etclalithasri22
 
Neo4j_Exploring the Impact of Graph Technology on Financial Services.pdf
Neo4j_Exploring the Impact of Graph Technology on Financial Services.pdfNeo4j_Exploring the Impact of Graph Technology on Financial Services.pdf
Neo4j_Exploring the Impact of Graph Technology on Financial Services.pdfNeo4j
 
Data Discovery With Power Query in excel
Data Discovery With Power Query in excelData Discovery With Power Query in excel
Data Discovery With Power Query in excelKapilSidhpuria3
 
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...ThinkInnovation
 
testingsdadadadaaddadadadadadadadaad.pdf
testingsdadadadaaddadadadadadadadaad.pdftestingsdadadadaaddadadadadadadadaad.pdf
testingsdadadadaaddadadadadadadadaad.pdfDSP Mutual Fund
 

Recently uploaded (19)

Role of Consumer Insights in business transformation
Role of Consumer Insights in business transformationRole of Consumer Insights in business transformation
Role of Consumer Insights in business transformation
 
Adobe Scan 06-Mar-2024 (1).pdf shavashwvw
Adobe Scan 06-Mar-2024 (1).pdf shavashwvwAdobe Scan 06-Mar-2024 (1).pdf shavashwvw
Adobe Scan 06-Mar-2024 (1).pdf shavashwvw
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
 
Adobe Scan 06-Mar-2024 (1).pdfwvsbbsbsba
Adobe Scan 06-Mar-2024 (1).pdfwvsbbsbsbaAdobe Scan 06-Mar-2024 (1).pdfwvsbbsbsba
Adobe Scan 06-Mar-2024 (1).pdfwvsbbsbsba
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in India
 
Digital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfDigital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdf
 
2023 Survey Shows Dip in High School E-Cigarette Use
2023 Survey Shows Dip in High School E-Cigarette Use2023 Survey Shows Dip in High School E-Cigarette Use
2023 Survey Shows Dip in High School E-Cigarette Use
 
Film cover research.pptx for media courseowrk
Film cover research.pptx for media courseowrkFilm cover research.pptx for media courseowrk
Film cover research.pptx for media courseowrk
 
Rabobank_Exploring the Impact of Graph Technology on Financial Services.pdf
Rabobank_Exploring the Impact of Graph Technology on Financial Services.pdfRabobank_Exploring the Impact of Graph Technology on Financial Services.pdf
Rabobank_Exploring the Impact of Graph Technology on Financial Services.pdf
 
Inference rules in artificial intelligence
Inference rules in artificial intelligenceInference rules in artificial intelligence
Inference rules in artificial intelligence
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
 
prediction of default payment next month using a logistic approach
prediction of default payment next month using a logistic approachprediction of default payment next month using a logistic approach
prediction of default payment next month using a logistic approach
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
 
Statistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfStatistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdf
 
DATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etcDATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etc
 
Neo4j_Exploring the Impact of Graph Technology on Financial Services.pdf
Neo4j_Exploring the Impact of Graph Technology on Financial Services.pdfNeo4j_Exploring the Impact of Graph Technology on Financial Services.pdf
Neo4j_Exploring the Impact of Graph Technology on Financial Services.pdf
 
Data Discovery With Power Query in excel
Data Discovery With Power Query in excelData Discovery With Power Query in excel
Data Discovery With Power Query in excel
 
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
 
testingsdadadadaaddadadadadadadadaad.pdf
testingsdadadadaaddadadadadadadadaad.pdftestingsdadadadaaddadadadadadadadaad.pdf
testingsdadadadaaddadadadadadadadaad.pdf
 

Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)

  • 1. Biml for Beginners: Script and Automate SSIS Development Cathrine Wilhelmsen SQLSaturday Chicago · March 17th 2018
  • 2. Session Description Are you tired of creating and updating the same SSIS packages again and again? Is your wrist hurting from all that clicking, dragging, dropping, connecting and aligning? Do you want to take the next step and really speed up your SSIS development? Say goodbye to repetitive work and hello to Biml, the markup language for Business Intelligence projects. In this session we will look at the basics of Biml. First learn how to use Biml to generate SSIS packages from database metadata. Then see how you can reuse code to implement changes in multiple SSIS packages and projects with just a few clicks. Finally, we will create an example project that you can download and start with to speed up your SSIS development from day one. Stop wasting your valuable time on doing the same things over and over and over again, and see how you can complete in a day what once took more than a week!
  • 3. Biml Basics Tools & Code SQL and SSIS from Metadata …the next 60 minutes…
  • 4. Cathrine Wilhelmsen @cathrinew cathrinew.net Data Warehouse & Business Intelligence Consultant
  • 6. Work… …? Long development time Many SSIS packages Frequent requirement changes
  • 8. It's time for a change!
  • 9.
  • 10. What is Biml? Business Intelligence Markup Language Easy to read and write XML language Describes business intelligence objects: • Databases, Schemas, Tables, Views, Columns • SSIS Packages • SSAS Cubes
  • 11. Why use Biml? SSIS: Plumbing Biml: Business Logic
  • 12. SSIS: Plumbing Spend time on dragging, dropping, connecting, aligning Create the same package over and over again with minor changes Standards, patterns and templates must be defined up-front Changes must be done in every single package Higher risk of manual errors More packages, more time
  • 13. Biml: Business Logic Spend time on what is unique in a package Create a pattern once and reuse for all similar packages Handle scope and requirement changes quickly and easily Changes can be applied to all packages at once Lower risk of manual errors Longer time to start, but then automate and reuse
  • 14. Will Biml solve all your challenges? Probably not... Biml is a tool for generating SSIS packages Biml is not a pre-defined ETL framework Biml is not a tool for automated deployment ...but it will solve many challenges!
  • 15. How can Biml help you? Biml is great for large projects with common patterns… Timesaving: Many SSIS packages from one Biml file Reusable: Write once and run on any platform Flexible: Start simple, expand as you learn …but is also useful for smaller projects!
  • 16. What do you need?
  • 17. BimlExpress Free add-in for Visual Studio Code editor with syntax highlighting and Biml Intellisense More frequent updates than BIDS Helper varigence.com/bimlexpress
  • 18. BimlOnline Free browser-based Biml editor Code editor with Biml and C# Intellisense Reverse-engineer from SSIS to Biml bimlonline.com
  • 19. BimlStudio Licensed full-featured development environment for Biml Visual designer and metadata modeling Full-stack automation and transformers varigence.com/bimlstudio
  • 20. How does it work? …generated packages look exactly like manually created packages
  • 21. Biml syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 22. Biml syntax: Root Element <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 23. Biml syntax: Collections of Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 24. Biml syntax: Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 25. Biml syntax: Attributes <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 26. Biml syntax: Full vs. Shorthand Syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 28. Goal: Load from source to staging
  • 29. Add New Biml File from BimlExpress menu…
  • 30. …or right-click on SSIS project to Add New Biml File
  • 31. Biml files are placed under Miscellaneous
  • 32. Check Biml For Errors from BimlExpress menu…
  • 33. …or right-click on file to Check Biml For Errors
  • 34. Generate SSIS Packages from BimlExpress menu…
  • 35. …or right-click on file to Generate SSIS Packages
  • 36. From Biml to SSIS: Control Flow <Package Name="TruncateLoad" ConstraintMode="Linear"> <Tasks> <ExecuteSQL Name="Truncate Table" ConnectionName="Staging"> <DirectInput>TRUNCATE TABLE DestinationTable</DirectInput> </ExecuteSQL> <Dataflow Name="Load Table"> <Transformations>...</Transformations> </Dataflow> </Tasks> </Package>
  • 37. From Biml to SSIS: Data Flow <Transformations> <OleDbSource Name="Source" ConnectionName="AW2014"> <ExternalTableInput Table="SourceTable" /> </OleDbSource> <DerivedColumns Name="Add LoadDate"> <Columns> <Column Name="LoadDate" DataType="DateTime"> @[System::StartTime] </Column> </Columns> </DerivedColumns> <OleDbDestination Name="Destination" ConnectionName="Staging"> <ExternalTableOutput Table="DestinationTable" /> </OleDbDestination> </Transformations>
  • 38. .biml vs .dtsx: human-readable vs ALL THE CODE! (20% zoom)(150% zoom)
  • 39. Ok, so we can go from Biml to SSIS…
  • 40. …can we go from SSIS to Biml?
  • 41. Yes!
  • 42. Let's convert some packages from SSIS to Biml!
  • 43. Convert SSIS to Biml in BimlExpress 2018
  • 44. Select Package or project…
  • 45. …import, filter and add to project
  • 46. The magic is in the…
  • 47. What is BimlScript? Extend Biml with C# or VB code blocks Import database structure and metadata Loop over tables and columns Expressions replace static values Generate, control and manipulate Biml code
  • 48. BimlScript Code Blocks <# … #> Control Block (Control logic) <#= … #> Text Block (Returns string) <#@ … #> Directive (Compiler instructions) <#+ … #> Class Block (Create C# classes) <#* … *#> Comment Block (Comments)
  • 49. BimlScript Syntax <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 50. BimlScript Syntax: Import metadata <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 51. BimlScript Syntax: Loop over tables <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 52. BimlScript Syntax: Replace static values <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 53. Biml vs. BimlScript Generate, control and manipulate Biml with C# XML Language "Just plain text"
  • 54. How does it work?
  • 55. Yes, but how does it work?
  • 56. Yes, but how does it actually work? <# foreach (var table in RootNode.Tables) { #> <Package Name="Load_<#=table.Name#>" /> <# } #> <Package Name="Load_Customer" /> <Package Name="Load_Product" /> <Package Name="Load_Sales" />
  • 58. Of course I can create 200 SSIS Packages! …what do you need me to do after lunch?
  • 59. Don't Repeat Yourself Move code to separate files Centralize and reuse Update once for all projects 1. Tiered Biml Files 2. Include Files 3. CallBimlScript
  • 61. What are Tiered Biml Files? Think of tiers as stacked layers or sequential steps Tier (Layer) 1 Tier 0 Tier (Layer) 2 Tier (Step) 1 Tier (Step) 2
  • 62. Tiered Biml Files Split Biml code in multiple files to: • Solve logical dependencies • Build solutions in multiple steps behind the scenes Specify the tier per file by using the template directive: <#@ template tier="2" #>
  • 63. Tiered Biml Files Biml is compiled step-by-step from lowest to highest tier • Biml files are implicitly tier 0 • BimlScript files are implicitly tier 1 In each tier, objects are added in-memory to RootNode Higher tiers can call RootNode to use objects from lower tiers
  • 64. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Databases Schemas Tables Packages
  • 65. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Databases Schemas Tables Packages
  • 66. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages
  • 67. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages
  • 68. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages
  • 69. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales
  • 70. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales
  • 71. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales
  • 72. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Master
  • 73. Tiered Biml Files: Behind the Scenes Connections Load Packages Master Package Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Master
  • 74. Tiered Biml Files: Behind the Scenes Connections Admin Source Destination Databases Schemas Tables Packages Load_Customer Load_Product Load_Sales Master
  • 75. How do you use Tiered Biml Files? 1. Create Biml files with specified tiers 2. Select all the tiered Biml files 3. Right-click and click Generate SSIS Packages 1 2 3
  • 77. Annotations and ObjectTags Annotations and ObjectTags are Key / Value pairs Use them to store and pass code between tiers Annotations: String / String ObjectTags: String / Object
  • 78. Annotations Create annotations: <OleDbConnection Name="Dest"> <Annotations> <Annotation Tag="Schema">aw</Annotation> </Annotations> </OleDbConnection> Use annotations: RootNode.OleDbConnections["Dest"].GetTag("Schema");
  • 79. ObjectTags Create ObjectTags: RootNode.ObjectTag["Filter"] = new List<string>{"Product","ProductCategory"}; Use ObjectTags: RootNode.ObjectTag["Filter"];
  • 81. Include Files Include common code in multiple files and projects Can include many file types: .biml .txt .sql .cs Use the include directive <#@ include file="CommonCode.biml" #> The directive will be replaced by the included file Works like an automated Copy & Paste
  • 86. CallBimlScript Works like a parameterized include (or stored procedure) File to be called (callee) specifies accepted parameters: <#@ property name="Parameter" type="String" #> File that calls (caller) passes parameters: <#=CallBimlScript("Common.biml", Parameter)#>
  • 92. How does all this work?
  • 93. Biml Basics Tools & Code SQL and SSIS from Metadata …the past 60 minutes…
  • 94. Where can I learn more? Free online training bimlscript.com
  • 95. Where can I learn more? cathrinew.net/BimlBook
  • 96. Get things done Start small Start simple Start with ugly code Keep going Expand Improve Deliver often
  • 97. Biml on Monday… …BimlBreak the rest of the week ☺