SlideShare a Scribd company logo
1 of 55
Download to read offline
Don't Repeat Yourself
An Introduction to Agile SSIS Development
Cathrine Wilhelmsen
Moderated by Regis Baccaro
2
Technical Assistance
2
Maximize your screen
with the zoom button
on the top of the
presentation window
Type your questions in
the question pane on
the right side
3
3
Thank You Presenting Sponsors
Supporting Sponsors
Gain insights through familiar
tools while balancing monitoring
and managing user created
content across structured and
unstructured sources.
Unifying computer, storage,
networking, and virtualization, Cisco
UCS is the optimal database and
business intelligence platform for SQL
Server.
Solutions from Dell help you
monitor, manage, protect and
improve your SQL Server
environment.
www.microsoft.com www.cisco.com www.software.dell.com
Planning on attending PASS Summit 2015? Start saving
today!
• The world’s largest gathering of SQL Server & BI professionals
• Take your SQL Server skills to the next level by learning from the world’s SQL
Server experts, in over 190 technical sessions
• Over 5000 attendees, representing 2000 companies, from 52 countries,
ready to network & learn
Contact your Local or Virtual Chapter for an
additional $150 discount.
$1795
until July 12th, 2015
Cathrine Wilhelmsen
5
Data Warehouse Architect and Developer
For the Norwegian financial company
Active in the Biml and SQL Server community
PASS Outstanding Volunteer, SQLSaturday Oslo Organizer and
SQL Server User Group Norway Board Member
cathrinewilhelmsen.net
@cathrinew
/in/cathrinewilhelmsen
Don't Repeat Yourself
An Introduction to Agile SSIS Development
Cathrine Wilhelmsen
7
Session Description
SSIS (SQL Server Integration Services) is a powerful tool for extracting, transforming
and loading data, but creating and maintaining a large number of SSIS packages can
be both tedious and time-consuming. Even if you use templates and follow best
practices you often have to repeat the same steps over and over again. Handling
metadata and schema changes is a manual process, and there are no easy ways to
implement new requirements in multiple packages at the same time.
It is time to bring the Don't Repeat Yourself (DRY) software engineering principle to
SSIS projects. First learn how to use Biml (Business Intelligence Markup Language)
and BimlScript to generate SSIS packages from database metadata and implement
changes in all those packages with just a few clicks. Then take the DRY principle one
step further and learn how to update all packages in multiple projects by separating and
reusing common code. See how Biml allows you to complete in a day what once took
more than a week!
7
8
Who are you?
SSIS and ETL developer?
Easily bored?
Tired of repetitive work?
8
9
Your work day?
Long development time?
Many SSIS packages?
Slow GUI editor?
9
10
Ever experienced this?
10
New standards!
Yay...Job done!
Ready for a change?
12
From Traditional SSIS to Agile SSIS
Traditional: Plumbing Agile: Business Logic
13
Traditional SSIS: Plumbing
Time wasted on dragging, dropping, connecting, aligning
Create the same package over and over and over again with just a few changes
Standards, patterns and templates must be defined up-front
Changes must be done in every single package
High risk of manual errors
More packages, more time
13
14
Agile SSIS: Business Logic
Spend time on what is unique in a package
Create a pattern once and reuse for all similar packages
Handle scope and requirement changes quickly and easily
Changes can be applied to all packages at once
Lower risk of manual errors
Longer time to start, but then reuse and scale
14
16
Will Biml solve all your problems?
Probably not...
Biml is a tool for generating SSIS packages
Biml is not a pre-defined data warehouse framework
Biml is not a tool for automated deployment
...but it will solve many problems!
16
17
How can Biml help you?
Timesaving: Many SSIS Packages from one Biml file
Reusable: Write once and run on any platform
Flexible: Start simple, expand as you learn
17
18
What is Business Intelligence Markup Language?
Easy to read and write XML dialect
Specifies business intelligence objects
Databases, schemas, tables, columns
SSIS packages
SSAS cubes, facts, dimensions (not in free version)
19
What do you need?
20
How does it work?
Generated packages are indistinguishable from manually created packages
21
Biml syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
22
Biml syntax: Root Element
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
23
Biml syntax: Collection of Elements
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
24
Biml syntax: Elements
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
25
Biml syntax: Attributes
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
26
Biml syntax: Full vs. Shorthand Syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
27
Biml files are placed in the Miscellaneous folder
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
28
Right-click on the Biml file to Generate SSIS Packages
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Package Name="EmptyPackage2"/>
</Packages>
</Biml>
29
From Biml to SSIS
30
From Biml to SSIS
31
.biml vs .dtsx: human-readable vs ALL THE CODE!
31
(150% zoom) (20% zoom)
33
The magic is in the
Extend Biml with C# or VB.NET code blocks
Import database structure and metadata
Loop over tables and columns
Add expressions to replace static values
Allows you to control and manipulate Biml code
34
BimlScript code blocks
<#@ … #> Directives (Instructions to the BimlCompiler)
<# … #> Control Blocks (Control logic)
<#= … #> Expression Control Blocks (Replace block with string value)
<#+ … #> Class Feature Control Blocks (Create helper methods)
35
BimlScript syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
36
BimlScript syntax: Control Blocks
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
37
BimlScript syntax: Expression Control Block
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
38
How does it work?
39
Yes, but how does it work?
39
40
Yes, but how does it actually work?
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load<#=table.Name#>"></Package>
<# } #>
</Packages>
</Biml>
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="LoadCustomer"></Package>
<Package Name="LoadProduct"></Package>
<Package Name="LoadSales"></Package>
</Packages>
</Biml>
41
Don't Repeat Yourself
Move common code to separate files
Centralize and reuse in many projects
Update code once for all projects
1. Split and combine Biml files
2. Include files
3. CallBimlScript with parameters
42
Don't Repeat Yourself: Split and combine Biml files
Solve logical dependencies and simulate manual workflows by using tiers
Tiers instruct the BimlCompiler to compile files from lowest to highest tier
<#@ template tier="1" #>
Higher tiers can use and might depend on objects from lower tiers
Tier 1 - Create database connections
Example: Tier 2 - Create loading packages
Tier 3 - Create master package to execute loading packages
43
Split and combine multiple Biml files
1. Create Biml files with specified tiers
2. Select all the tiered Biml files
3. Right-click and click Generate SSIS Packages
1
2
3
44
Don't Repeat Yourself: Include files
Include common code in multiple files and projects
Can include many file types: .biml .txt .sql .cs
Use the include directive
<#@ include file="CommonCode.biml" #>
The include directive will be replaced by the content of the included file
Include pulls code from the included file into the main file
Don't Repeat Yourself: Include files
45
46
Don't Repeat Yourself: CallBimlScript with parameters
Works like a parameterized include
File to be called (callee) specifies the input parameters it accepts
<#@ property name="Table" type="AstTableNode" #>
File that calls (caller) passes input parameters
<#=CallBimlScript("CommonCode.biml", Table)#>
CallBimlScript pushes parameters from the caller to the callee, and
the callee returns code
Don't Repeat Yourself: CallBimlScript with parameters
47
Don't Repeat Yourself: CallBimlScript with parameters
48
Demo
50
What do you do next?
1. Download BIDS Helper
2. Identify your SSIS patterns
3. Rewrite one SSIS package to Biml
(Not possible to reverse-engineer SSIS to Biml in BIDS Helper)
4. Expand with BimlScript
5. Separate and reuse common Biml code
6. ...never look back to the days of drag&drop 
7. Get involved in the SQL Server and Biml community!
51
Biml on Monday...
…BimlBreak the rest of the week
52
Thank you!
@cathrinew
cathrinewilhelmsen.net
no.linkedin.com/in/cathrinewilhelmsen
contact@cathrinewilhelmsen.net
cathrinewilhelmsen.net/biml
slideshare.net/cathrinewilhelmsen
Questions?
Thank You for Attending
Follow @pass24hop
Share your thoughts with hashtags
#pass24hop & #sqlpass
The Data Loading Performance
Presentation
Matan Yungman
Coming Up Next …

More Related Content

What's hot

Biml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into BimlBiml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into BimlCathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Cathrine Wilhelmsen
 
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)Cathrine Wilhelmsen
 
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Cathrine Wilhelmsen
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Cathrine Wilhelmsen
 
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...Cathrine Wilhelmsen
 
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)Cathrine Wilhelmsen
 

What's hot (20)

Biml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into BimlBiml Academy 2 - Lesson 5: Importing source metadata into Biml
Biml Academy 2 - Lesson 5: Importing source metadata into Biml
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vancouver)
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Iceland)
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
 
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Minnesota)
 
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
 
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
 
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
 
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
 
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
 
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Nashville)
 
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
 

Similar to Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours of PASS)

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)Cathrine Wilhelmsen
 
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on DatabricksCI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on DatabricksDatabricks
 
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)Cathrine Wilhelmsen
 
How to implement a truly modular ecommerce platform on the example of Spryker...
How to implement a truly modular ecommerce platform on the example of Spryker...How to implement a truly modular ecommerce platform on the example of Spryker...
How to implement a truly modular ecommerce platform on the example of Spryker...Fabian Wesner
 
Customer FX Technical Reference Sheet
Customer FX Technical Reference SheetCustomer FX Technical Reference Sheet
Customer FX Technical Reference SheetGoodCustomers
 
BI xPress - Pragmatic Works
BI xPress - Pragmatic WorksBI xPress - Pragmatic Works
BI xPress - Pragmatic WorksRobert Peters
 
Continuous Integration & Continuous Delivery
Continuous Integration & Continuous DeliveryContinuous Integration & Continuous Delivery
Continuous Integration & Continuous DeliveryDatabricks
 
Continuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday SloveniaContinuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday SloveniaDr. John Tunnicliffe
 
Continuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday SloveniaContinuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday SloveniaDr. John Tunnicliffe
 
generate IP CORES
generate IP CORESgenerate IP CORES
generate IP CORESguest296013
 

Similar to Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours of PASS) (15)

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: S...
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLDay 2018)
 
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on DatabricksCI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
 
BIML- What is it good for?
BIML- What is it good for?BIML- What is it good for?
BIML- What is it good for?
 
Readme
ReadmeReadme
Readme
 
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
Biml for Beginners: Script and Automate SSIS development (Hybrid VC)
 
How to implement a truly modular ecommerce platform on the example of Spryker...
How to implement a truly modular ecommerce platform on the example of Spryker...How to implement a truly modular ecommerce platform on the example of Spryker...
How to implement a truly modular ecommerce platform on the example of Spryker...
 
Ad505 dev blast
Ad505 dev blastAd505 dev blast
Ad505 dev blast
 
Isset Presentation @ EECI2009
Isset Presentation @ EECI2009Isset Presentation @ EECI2009
Isset Presentation @ EECI2009
 
Customer FX Technical Reference Sheet
Customer FX Technical Reference SheetCustomer FX Technical Reference Sheet
Customer FX Technical Reference Sheet
 
BI xPress - Pragmatic Works
BI xPress - Pragmatic WorksBI xPress - Pragmatic Works
BI xPress - Pragmatic Works
 
Continuous Integration & Continuous Delivery
Continuous Integration & Continuous DeliveryContinuous Integration & Continuous Delivery
Continuous Integration & Continuous Delivery
 
Continuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday SloveniaContinuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
 
Continuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday SloveniaContinuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
Continuous Integration and the Data Warehouse - PASS SQL Saturday Slovenia
 
generate IP CORES
generate IP CORESgenerate IP CORES
generate IP CORES
 

More from Cathrine Wilhelmsen

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Cathrine Wilhelmsen
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Cathrine Wilhelmsen
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Cathrine Wilhelmsen
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Cathrine Wilhelmsen
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Cathrine Wilhelmsen
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)Cathrine Wilhelmsen
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Cathrine Wilhelmsen
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Cathrine Wilhelmsen
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Cathrine Wilhelmsen
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...Cathrine Wilhelmsen
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Cathrine Wilhelmsen
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)Cathrine Wilhelmsen
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 

Recently uploaded

Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 

Recently uploaded (20)

Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 

Don't Repeat Yourself - An Introduction to Agile SSIS Development (24 Hours of PASS)

  • 1. Don't Repeat Yourself An Introduction to Agile SSIS Development Cathrine Wilhelmsen Moderated by Regis Baccaro
  • 2. 2 Technical Assistance 2 Maximize your screen with the zoom button on the top of the presentation window Type your questions in the question pane on the right side
  • 3. 3 3 Thank You Presenting Sponsors Supporting Sponsors Gain insights through familiar tools while balancing monitoring and managing user created content across structured and unstructured sources. Unifying computer, storage, networking, and virtualization, Cisco UCS is the optimal database and business intelligence platform for SQL Server. Solutions from Dell help you monitor, manage, protect and improve your SQL Server environment. www.microsoft.com www.cisco.com www.software.dell.com
  • 4. Planning on attending PASS Summit 2015? Start saving today! • The world’s largest gathering of SQL Server & BI professionals • Take your SQL Server skills to the next level by learning from the world’s SQL Server experts, in over 190 technical sessions • Over 5000 attendees, representing 2000 companies, from 52 countries, ready to network & learn Contact your Local or Virtual Chapter for an additional $150 discount. $1795 until July 12th, 2015
  • 5. Cathrine Wilhelmsen 5 Data Warehouse Architect and Developer For the Norwegian financial company Active in the Biml and SQL Server community PASS Outstanding Volunteer, SQLSaturday Oslo Organizer and SQL Server User Group Norway Board Member cathrinewilhelmsen.net @cathrinew /in/cathrinewilhelmsen
  • 6. Don't Repeat Yourself An Introduction to Agile SSIS Development Cathrine Wilhelmsen
  • 7. 7 Session Description SSIS (SQL Server Integration Services) is a powerful tool for extracting, transforming and loading data, but creating and maintaining a large number of SSIS packages can be both tedious and time-consuming. Even if you use templates and follow best practices you often have to repeat the same steps over and over again. Handling metadata and schema changes is a manual process, and there are no easy ways to implement new requirements in multiple packages at the same time. It is time to bring the Don't Repeat Yourself (DRY) software engineering principle to SSIS projects. First learn how to use Biml (Business Intelligence Markup Language) and BimlScript to generate SSIS packages from database metadata and implement changes in all those packages with just a few clicks. Then take the DRY principle one step further and learn how to update all packages in multiple projects by separating and reusing common code. See how Biml allows you to complete in a day what once took more than a week! 7
  • 8. 8 Who are you? SSIS and ETL developer? Easily bored? Tired of repetitive work? 8
  • 9. 9 Your work day? Long development time? Many SSIS packages? Slow GUI editor? 9
  • 10. 10 Ever experienced this? 10 New standards! Yay...Job done!
  • 11. Ready for a change?
  • 12. 12 From Traditional SSIS to Agile SSIS Traditional: Plumbing Agile: Business Logic
  • 13. 13 Traditional SSIS: Plumbing Time wasted on dragging, dropping, connecting, aligning Create the same package over and over and over again with just a few changes Standards, patterns and templates must be defined up-front Changes must be done in every single package High risk of manual errors More packages, more time 13
  • 14. 14 Agile SSIS: Business Logic Spend time on what is unique in a package Create a pattern once and reuse for all similar packages Handle scope and requirement changes quickly and easily Changes can be applied to all packages at once Lower risk of manual errors Longer time to start, but then reuse and scale 14
  • 15.
  • 16. 16 Will Biml solve all your problems? Probably not... Biml is a tool for generating SSIS packages Biml is not a pre-defined data warehouse framework Biml is not a tool for automated deployment ...but it will solve many problems! 16
  • 17. 17 How can Biml help you? Timesaving: Many SSIS Packages from one Biml file Reusable: Write once and run on any platform Flexible: Start simple, expand as you learn 17
  • 18. 18 What is Business Intelligence Markup Language? Easy to read and write XML dialect Specifies business intelligence objects Databases, schemas, tables, columns SSIS packages SSAS cubes, facts, dimensions (not in free version)
  • 19. 19 What do you need?
  • 20. 20 How does it work? Generated packages are indistinguishable from manually created packages
  • 21. 21 Biml syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 22. 22 Biml syntax: Root Element <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 23. 23 Biml syntax: Collection of Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 24. 24 Biml syntax: Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 25. 25 Biml syntax: Attributes <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 26. 26 Biml syntax: Full vs. Shorthand Syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 27. 27 Biml files are placed in the Miscellaneous folder <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 28. 28 Right-click on the Biml file to Generate SSIS Packages <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  • 31. 31 .biml vs .dtsx: human-readable vs ALL THE CODE! 31 (150% zoom) (20% zoom)
  • 32.
  • 33. 33 The magic is in the Extend Biml with C# or VB.NET code blocks Import database structure and metadata Loop over tables and columns Add expressions to replace static values Allows you to control and manipulate Biml code
  • 34. 34 BimlScript code blocks <#@ … #> Directives (Instructions to the BimlCompiler) <# … #> Control Blocks (Control logic) <#= … #> Expression Control Blocks (Replace block with string value) <#+ … #> Class Feature Control Blocks (Create helper methods)
  • 35. 35 BimlScript syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 36. 36 BimlScript syntax: Control Blocks <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 37. 37 BimlScript syntax: Expression Control Block <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  • 38. 38 How does it work?
  • 39. 39 Yes, but how does it work? 39
  • 40. 40 Yes, but how does it actually work? <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in RootNode.Tables) { #> <Package Name="Load<#=table.Name#>"></Package> <# } #> </Packages> </Biml> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="LoadCustomer"></Package> <Package Name="LoadProduct"></Package> <Package Name="LoadSales"></Package> </Packages> </Biml>
  • 41. 41 Don't Repeat Yourself Move common code to separate files Centralize and reuse in many projects Update code once for all projects 1. Split and combine Biml files 2. Include files 3. CallBimlScript with parameters
  • 42. 42 Don't Repeat Yourself: Split and combine Biml files Solve logical dependencies and simulate manual workflows by using tiers Tiers instruct the BimlCompiler to compile files from lowest to highest tier <#@ template tier="1" #> Higher tiers can use and might depend on objects from lower tiers Tier 1 - Create database connections Example: Tier 2 - Create loading packages Tier 3 - Create master package to execute loading packages
  • 43. 43 Split and combine multiple Biml files 1. Create Biml files with specified tiers 2. Select all the tiered Biml files 3. Right-click and click Generate SSIS Packages 1 2 3
  • 44. 44 Don't Repeat Yourself: Include files Include common code in multiple files and projects Can include many file types: .biml .txt .sql .cs Use the include directive <#@ include file="CommonCode.biml" #> The include directive will be replaced by the content of the included file Include pulls code from the included file into the main file
  • 45. Don't Repeat Yourself: Include files 45
  • 46. 46 Don't Repeat Yourself: CallBimlScript with parameters Works like a parameterized include File to be called (callee) specifies the input parameters it accepts <#@ property name="Table" type="AstTableNode" #> File that calls (caller) passes input parameters <#=CallBimlScript("CommonCode.biml", Table)#> CallBimlScript pushes parameters from the caller to the callee, and the callee returns code
  • 47. Don't Repeat Yourself: CallBimlScript with parameters 47
  • 48. Don't Repeat Yourself: CallBimlScript with parameters 48
  • 49. Demo
  • 50. 50 What do you do next? 1. Download BIDS Helper 2. Identify your SSIS patterns 3. Rewrite one SSIS package to Biml (Not possible to reverse-engineer SSIS to Biml in BIDS Helper) 4. Expand with BimlScript 5. Separate and reuse common Biml code 6. ...never look back to the days of drag&drop  7. Get involved in the SQL Server and Biml community!
  • 51. 51 Biml on Monday... …BimlBreak the rest of the week
  • 54. Thank You for Attending Follow @pass24hop Share your thoughts with hashtags #pass24hop & #sqlpass
  • 55. The Data Loading Performance Presentation Matan Yungman Coming Up Next …