SlideShare a Scribd company logo
Biml Tips and Tricks
Not just for SSIS Packages!
Cathrine Wilhelmsen
SQLGrillen · June 22nd 2018
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Session Description
"Wait, what? Biml is not just for generating SSIS packages?"
Absolutely not! Come and see how you can use Biml (Business Intelligence
Markup Language) to save time and speed up other Data Warehouse
development tasks. You can generate complex T-SQL statements with Biml
instead of using dynamic SQL, create test data and populate static dimensions,
and even compare tables and views across multiple servers and databases.
Don't Repeat Yourself, start automating those boring, manual tasks today!
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Cathrine Wilhelmsen
@cathrinew cathrinew.net
Data Warehouse & Business Intelligence Consultant
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
…the next 60 minutes…
T-SQL
Test Data Comparing
Dimensions
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Wait…
Can't I use
<something else>?
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Of course!
But Biml works well
with source metadata
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
And…
Biml is fun!
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
But… Biml? How?
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Traditional Biml
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Crazy Fun Biml
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
What do you need?
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Preview Pane
The power is in the…
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Traditional BimlScript
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Traditional BimlScript
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load_<#=table.Name#>" />
<# } #>
<Package Name="Load_Customer" />
<Package Name="Load_Product" />
<Package Name="Load_Sales" />
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
BimlScript to Biml
Preview Pane
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
BimlScript to... ???
Preview Pane
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Crazy Fun BimlScript
<# foreach (var table in RootNode.Tables) { #>
Yay, a package! <#=table.Name#> :)
<# } #>
Yay, a package! Load_Customer :)
Yay, a package! Load_Product :)
Yay, a package! Load_Sales :)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
T-SQL from Biml
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Built-in
T-SQL from Biml
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Built-in T-SQL from Biml
Functional T-SQL
GetSelectSql
GetDropAndCreateDdl
GetTableSql
SSMS Snippets
GetInsertSql
GetUpdateSql
GetDeleteSql
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Functional T-SQL: GetSelectSql
SELECT
[Column1]
,[Column2]
FROM [Schema].[Table]
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Functional T-SQL: GetDropAndCreateDdl
IF EXISTS (...)
DROP TABLE [Schema].[Table]
GO
CREATE TABLE [Schema].[Table] (
[Column1] int IDENTITY(1,1) NOT NULL
,[Column2] nvarchar(50) NOT NULL
-- Constraints
)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Functional T-SQL: GetTableSql
IF EXISTS (...)
DROP TABLE [Schema].[Table]
GO
CREATE TABLE [Schema].[Table] (
[Column1] int IDENTITY(1,1) NOT NULL
,[Column2] nvarchar(50) NOT NULL
-- Constraints
)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Functional T-SQL
Use directly in SSIS
Copy and run in SSMS
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
SSMS Snippets: GetDeleteSql
DELETE FROM [Schema].[Table]
WHERE <Search Conditions,,>
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
SSMS Snippets: GetInsertSql
INSERT INTO [Schema].[Table] (
[Column1]
,[Column1]
) VALUES (
<Column1,int,>,
<Column2,nvarchar(50),>
)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
SSMS Snippets: GetUpdateSql
UPDATE [Schema].[Table] SET
[Column1] = <Column1, int,>,
[Column2] = <Column2, nvarchar(50),>
WHERE <Search Conditions,,>
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Copy and run in SSMS
Replace Template Parameters with actual values
CTRL SHIFT M
SSMS Snippets
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Custom
T-SQL from Biml
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Custom T-SQL from Biml
Table Metadata
<#=table.Name#>
<#=table.SchemaQualifiedName#>
SELECT *
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Custom T-SQL from Biml
Column Methods
GetColumnList
GetColumnAssignmentList
GetColumnComparisonList
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Column Methods
Return code fragments
Use as building blocks in custom T-SQL
Filter columns by using lambda expressions
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Column Methods
Return code fragments
Use as building blocks in custom T-SQL
Filter columns by using lambda expressions
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
"A lambda expression is an anonymous
function that you can use to create
delegates or expression tree types"
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
"A lambda expression is an anonymous
function that you can use to create
delegates or expression tree types"
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
The arrow is the lambda operator
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
Input parameter is on the left side
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
Expression is on the right side
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Lambda Expressions
column => column.Name == "ColumnID"
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnList
Get columns for SELECT
[Column1], [Column2], [Column3]
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnList
GetColumnList()
GetColumnList("src")
GetColumnList(c => c.Name == "ColumnID")
GetColumnList(c => c.DataType != DbType.Guid, "src")
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnAssignmentList
Get columns for UPDATE … SET
[l].[Column1] = [r].[Column1]
,[l].[Column2] = [r].[Column2]
,[l].[Column3] = [r].[Column3]
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnAssignmentList
GetColumnAssignmentList()
GetColumnAssignmentList("dst", "src")
GetColumnAssignmentList(c => !c.IsUsedInPrimaryKey)
GetColumnAssignmentList(c => !c.IsUsedInPrimaryKey, "dst", "src")
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList
Get columns for JOIN … ON
[l].[Column1] = [r].[Column1]
AND [l].[Column2] = [r].[Column2]
AND [l].[Column3] = [r].[Column3]
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
GetColumnComparisonList
GetColumnComparisonList()
GetColumnComparisonList("!=")
GetColumnComparisonList("dst", "src")
GetColumnComparisonList(c => c.IsUsedInPrimaryKey)
GetColumnComparisonList(c => c.IsUsedInPrimaryKey, "dst", "src")
GetColumnComparisonList(c => c.IsUsedInPrimaryKey, "!=", "dst", "src")
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
T-SQL from Biml
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Test Data
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml + Random()
Create Random Test Data
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Random Class
Random()
Pseudo-random number generator
Random enough for practical purposes
Random(12345)
Always generates the same series of random numbers
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Random Methods
Next() Random number
Next(100) Random number smaller than specified value
Next(1000, 2000) Random number within specified range
NextDouble() Random decimal number between 0.0 and 1.0
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Helper Method
<#+
Random r = new Random();
public string GetTestData(AstTableColumnBaseNode column) {
switch (column.DataType) {
case DbType.Int32 :
return r.Next(Int32.MaxValue).ToString();
case DbType.String :
return "'" + GetRandomString(column.Length) + "'"
default :
return "NULL";
}
}
#>
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Helper Helper Method
<#+
private string GetRandomString(int length) {
var allowedChars = "ABC123";
var randomChars = new char[length];
for (int i = 0; i < length; i++) {
randomChars[i] = allowedChars[ r.Next(allowedChars.Length) ];
}
return new String(randomCharacters);
}
#>
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml + Random()
Very limited functionality
No dependencies
Works out-of-the box
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml + Bogus
Create Random and Specific Test Data
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Bogus Project
Simple and sane fake data generator for .NET
https://github.com/bchavez/Bogus
Advanced functionality for custom objects
…or simple functionality for Biml hacks :)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Address
Commerce
Company
Database
Date
Finance
Internet
Lorem
Name
Person
Phone
System
Bogus API
Supports all data types plus built-in test data, including:
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
The Bogus Methods
<#@ assembly name="Bogus.dll" #>
<#@ import namespace="Bogus" #>
<# var f = new Faker(); #>
<# Person p = new Person(); #>
INSERT INTO dbo.Employee(FirstName, LastName, Birthday) VALUES
(
'<#=p.FirstName#>',
'<#=p.LastName#>',
'<#=p.DateOfBirth.ToString("yyyyMMdd")#>'
)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml + Bogus()
Better functionality than Random()
Easier to understand, frequently updated
Requires additional files
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Installing Bogus
Install via Nuget:
Install-Package Bogus
Or download latest release .zip:
https://github.com/bchavez/Bogus/releases
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
Test Data with
Random() and Bogus
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Alternative (Better) Solutions
Mockaroo (free, but only 1000 rows)
https://mockaroo.com/
Redgate SQL Data Generator (licensed)
https://www.red-gate.com/products/sql-
development/sql-data-generator/index
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Static Dimensions
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Static Dimensions (or any kind of static table)
SCD Type 0: Not changing
Lookup Tables
1. Create table definition
2. Add static source rows
3. Generate INSERT statements
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Sources
Part of <Table> objects
Defines rows to be inserted when table is created
• Unknown dimension members
• Date dimension
• Code tables
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Source
<Table Name="Table" SchemaName="Database.Schema">
<Columns>
<Column Name="Column1" DataType="Int32" />
<Column Name="Column2" DataType="String" Length="10" />
</Columns>
<Sources>
<StaticSource Name="TableRows">
<Rows>
<Row>
<ColumnValues>
<ColumnValue ColumnName="Column1" Value="-1" />
<ColumnValue ColumnName="Column2" Value="'Unknown'" />
</ColumnValues>
</Row>
</Rows>
</StaticSource>
</Sources>
</Table>
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Biml Static Source
<# if (table.Sources.Any()) { #>
<#=TableToPackageLowerer.GetStaticSourceInsertStatements(
table.SchemaQualifiedName,
table.HasIdentity,
(AstTableStaticSourceNode)table.Sources.FirstOrDefault()
)#>
<# } #>
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Drop and Create
Facts and Dimensions
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Could not drop object 'Schema.Table'
because it is referenced by a
FOREIGN KEY constraint.
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Drop and Create
Create Order: Dim1 Dim2 Fact1 Fact2
Fact2 Fact1 Dim2 Dim1Drop Order:
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Drop and Create
Create Order: Dim1 Dim2 Fact1 Fact2
Fact2 Fact1 Dim2 Dim1Drop Order:
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Topological Sort
"A topological sort of a directed graph is a
linear ordering of its vertices such that for
every directed edge uv from vertex u to vertex
v, u comes before v in the ordering"
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Topological Sort
"A topological sort of a directed graph is a
linear ordering of its vertices such that for
every directed edge uv from vertex u to vertex
v, u comes before v in the ordering"
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Topological Sort
"Sorts objects by dependencies.
For example dimensions before fact tables."
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Topological Sort
"Sorts objects by dependencies.
For example dimensions before fact tables."
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Topological Sort
IEnumerable<T> DependencyAnalysis.TopologicalSort<T> (
IEnumerable<T> source,
Expression<Func<T, T>> relationProperty
)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Topological Sort
var dependentTables =
DependencyAnalysis.TopologicalSort<AstTableNode> (
RootNode.Tables,
t => t.Columns
.OfType<AstTableColumnTableReferenceNode>()
.Select(c => c.ForeignTable)
);
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
Static Dimensions
and Topological Sort
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Comparing
Databases
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Once upon a time…
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Panic!
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
How to Compare
Tables and Columns
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
LINQ
Sort
OrderBy, ThenBy
Filter
Where, OfType
Group
GroupBy
Aggregate
Count, Sum
Check Collections
All, Any, Contains
Set Operations
Except, Intersect
Project Collections
Select, SelectMany
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
LINQ
Sort
OrderBy, ThenBy
Filter
Where, OfType
Group
GroupBy
Aggregate
Count, Sum
Check Collections
All, Any, Contains
Set Operations
Except, Intersect
Project Collections
Select, SelectMany
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
LINQ: Filter Collections
Where()
Returns the filtered collection with all elements that meet the criteria
table.Where(t => t.Schema.Name == "dim")
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
LINQ: Set Operations
Except()
Returns elements in first collection that are not in second collection
table1.Except(table2)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
LINQ: Set Operations
Intersect()
Returns elements in first collection that are also in second collection
table1.Intersect(table2)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
LINQ: Project Collections
Select()
Creates a new collection from specified attributes
table.Select(t => t.Name)
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
DEMO
Comparing
Databases
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Alternative (Better) Solutions
Redgate SQL Compare (licensed)
https://www.red-gate.com/products/sql-
development/sql-compare/
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
…the past 60 minutes…
T-SQL
Test Data Comparing
Dimensions
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Is this useful?
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
What other ideas
do you have?
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
Have fun!
© 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
@cathrinew
cathrinew.net
hi@cathrinew.net
Biml resources and demo files:
cathrinew.net/biml

More Related Content

What's hot

Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Cathrine Wilhelmsen
 
Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)
Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)
Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)
Cathrine Wilhelmsen
 
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Cathrine Wilhelmsen
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Cathrine Wilhelmsen
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Cathrine Wilhelmsen
 
Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)
Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)
Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)
Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Cathrine Wilhelmsen
 
About Pragmatic Works
About Pragmatic WorksAbout Pragmatic Works
About Pragmatic Works
MILL5
 
O365Engage17 - Using Exchange Online to Classify and Secure Mail
O365Engage17 - Using Exchange Online to Classify and Secure MailO365Engage17 - Using Exchange Online to Classify and Secure Mail
O365Engage17 - Using Exchange Online to Classify and Secure Mail
NCCOMMS
 
Tools and Tips For Data Warehouse Developers (SQLGLA)
Tools and Tips For Data Warehouse Developers (SQLGLA)Tools and Tips For Data Warehouse Developers (SQLGLA)
Tools and Tips For Data Warehouse Developers (SQLGLA)
Cathrine Wilhelmsen
 
O365Engage17 - Options for staying compliant in exchange online
O365Engage17 - Options for staying compliant in exchange onlineO365Engage17 - Options for staying compliant in exchange online
O365Engage17 - Options for staying compliant in exchange online
NCCOMMS
 
Reach New Heights with Amazon Redshift
Reach New Heights with Amazon RedshiftReach New Heights with Amazon Redshift
Reach New Heights with Amazon Redshift
Matillion
 
Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013
Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013
Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013
Kai Wähner
 
Pick a Winner: How to Choose a Data Warehouse
Pick a Winner: How to Choose a Data WarehousePick a Winner: How to Choose a Data Warehouse
Pick a Winner: How to Choose a Data Warehouse
Matillion
 
O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...
O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...
O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...
NCCOMMS
 
O365Engage17 - Protecting O365 Data in a Modern World
O365Engage17 - Protecting O365 Data in a Modern WorldO365Engage17 - Protecting O365 Data in a Modern World
O365Engage17 - Protecting O365 Data in a Modern World
NCCOMMS
 
How to Choose a Data Warehouse
How to Choose a Data WarehouseHow to Choose a Data Warehouse
How to Choose a Data Warehouse
Matillion
 
Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...
Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...
Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...
Tyler Wishnoff
 

What's hot (20)

Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
Biml for Beginners: Script and Automate SSIS development (Malibu SQL Server U...
 
Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)
Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)
Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)
 
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
 
Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)
Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)
Building Dynamic Pipelines in Azure Data Factory (Data Saturday Holland)
 
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Chicago)
 
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
Level Up Your Biml: Best Practices and Coding Techniques (SQLBits 2018)
 
About Pragmatic Works
About Pragmatic WorksAbout Pragmatic Works
About Pragmatic Works
 
O365Engage17 - Using Exchange Online to Classify and Secure Mail
O365Engage17 - Using Exchange Online to Classify and Secure MailO365Engage17 - Using Exchange Online to Classify and Secure Mail
O365Engage17 - Using Exchange Online to Classify and Secure Mail
 
Tools and Tips For Data Warehouse Developers (SQLGLA)
Tools and Tips For Data Warehouse Developers (SQLGLA)Tools and Tips For Data Warehouse Developers (SQLGLA)
Tools and Tips For Data Warehouse Developers (SQLGLA)
 
O365Engage17 - Options for staying compliant in exchange online
O365Engage17 - Options for staying compliant in exchange onlineO365Engage17 - Options for staying compliant in exchange online
O365Engage17 - Options for staying compliant in exchange online
 
Reach New Heights with Amazon Redshift
Reach New Heights with Amazon RedshiftReach New Heights with Amazon Redshift
Reach New Heights with Amazon Redshift
 
Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013
Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013
Alternatives for Systems Integration in the NoSQL Era - NoSQL Roadshow 2013
 
Pick a Winner: How to Choose a Data Warehouse
Pick a Winner: How to Choose a Data WarehousePick a Winner: How to Choose a Data Warehouse
Pick a Winner: How to Choose a Data Warehouse
 
O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...
O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...
O365Engage17 - Smart Email Migration Knowing What’s Lurking in the ‘Dark Corn...
 
O365Engage17 - Protecting O365 Data in a Modern World
O365Engage17 - Protecting O365 Data in a Modern WorldO365Engage17 - Protecting O365 Data in a Modern World
O365Engage17 - Protecting O365 Data in a Modern World
 
How to Choose a Data Warehouse
How to Choose a Data WarehouseHow to Choose a Data Warehouse
How to Choose a Data Warehouse
 
Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...
Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...
Hassle-Free Data Lake Governance: Automating Your Analytics with a Semantic L...
 

Similar to Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)

Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Cathrine Wilhelmsen
 
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)Pipelines and Packages: Introduction to Azure Data Factory (24HOP)
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)
Cathrine Wilhelmsen
 
Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS
Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS
Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS
AWS Germany
 
OOP Best Practices in JavaScript
OOP Best Practices in JavaScriptOOP Best Practices in JavaScript
OOP Best Practices in JavaScript
Haim Michael
 
AppSync in real world - pitfalls, unexpected benefits & lessons learnt
AppSync in real world - pitfalls, unexpected benefits & lessons learntAppSync in real world - pitfalls, unexpected benefits & lessons learnt
AppSync in real world - pitfalls, unexpected benefits & lessons learnt
AWS User Group Bengaluru
 
Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018
Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018
Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018
Amazon Web Services
 
Creating a Machine Learning Factory
Creating a Machine Learning FactoryCreating a Machine Learning Factory
Creating a Machine Learning Factory
Amazon Web Services
 
Analyzing Streams
Analyzing StreamsAnalyzing Streams
Analyzing Streams
Amazon Web Services
 
AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...
AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...
AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...
Amazon Web Services
 
ServerlessConf 2018 Keynote - Debunking Serverless Myths
ServerlessConf 2018 Keynote - Debunking Serverless MythsServerlessConf 2018 Keynote - Debunking Serverless Myths
ServerlessConf 2018 Keynote - Debunking Serverless Myths
Tim Wagner
 
Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...
Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...
Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...
Amazon Web Services
 
The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...
The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...
The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...
Amazon Web Services
 
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter AnalysisIBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
Torsten Steinbach
 
Understanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web SummitUnderstanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web Summit
Amazon Web Services
 
Building Serverless ETL Pipelines
Building Serverless ETL PipelinesBuilding Serverless ETL Pipelines
Building Serverless ETL Pipelines
Amazon Web Services
 
Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...
Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...
Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...
Amazon Web Services
 
Tsar tech talk
Tsar tech talkTsar tech talk
Tsar tech talk
Anirudh Todi
 
TSAR (TimeSeries AggregatoR) Tech Talk
TSAR (TimeSeries AggregatoR) Tech TalkTSAR (TimeSeries AggregatoR) Tech Talk
TSAR (TimeSeries AggregatoR) Tech Talk
Anirudh Todi
 
Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018
Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018
Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018
Amazon Web Services
 
Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018
Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018
Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018
Amazon Web Services
 

Similar to Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018) (20)

Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
Level Up Your Biml: Best Practices and Coding Techniques (SQLSaturday Denver)
 
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)Pipelines and Packages: Introduction to Azure Data Factory (24HOP)
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)
 
Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS
Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS
Deep Dive into Concepts and Tools for Analyzing Streaming Data on AWS
 
OOP Best Practices in JavaScript
OOP Best Practices in JavaScriptOOP Best Practices in JavaScript
OOP Best Practices in JavaScript
 
AppSync in real world - pitfalls, unexpected benefits & lessons learnt
AppSync in real world - pitfalls, unexpected benefits & lessons learntAppSync in real world - pitfalls, unexpected benefits & lessons learnt
AppSync in real world - pitfalls, unexpected benefits & lessons learnt
 
Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018
Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018
Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018
 
Creating a Machine Learning Factory
Creating a Machine Learning FactoryCreating a Machine Learning Factory
Creating a Machine Learning Factory
 
Analyzing Streams
Analyzing StreamsAnalyzing Streams
Analyzing Streams
 
AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...
AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...
AWS IoT for Frictionless Consumer Experiences in Retail (RET201) - AWS re:Inv...
 
ServerlessConf 2018 Keynote - Debunking Serverless Myths
ServerlessConf 2018 Keynote - Debunking Serverless MythsServerlessConf 2018 Keynote - Debunking Serverless Myths
ServerlessConf 2018 Keynote - Debunking Serverless Myths
 
Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...
Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...
Architecting for Real-Time Insights with Amazon Kinesis (ANT310) - AWS re:Inv...
 
The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...
The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...
The Theory and Math Behind Data Privacy and Security Assurance (SEC301) - AWS...
 
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter AnalysisIBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
 
Understanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web SummitUnderstanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web Summit
 
Building Serverless ETL Pipelines
Building Serverless ETL PipelinesBuilding Serverless ETL Pipelines
Building Serverless ETL Pipelines
 
Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...
Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...
Ten Tips And Tricks for Improving Your GraphQL API with AWS AppSync (MOB401) ...
 
Tsar tech talk
Tsar tech talkTsar tech talk
Tsar tech talk
 
TSAR (TimeSeries AggregatoR) Tech Talk
TSAR (TimeSeries AggregatoR) Tech TalkTSAR (TimeSeries AggregatoR) Tech Talk
TSAR (TimeSeries AggregatoR) Tech Talk
 
Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018
Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018
Supercell – Scaling Mobile Games (GAM301) - AWS re:Invent 2018
 
Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018
Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018
Develop Integrations for Salesforce and AWS (API320) - AWS re:Invent 2018
 

More from Cathrine Wilhelmsen

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
Cathrine Wilhelmsen
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Cathrine Wilhelmsen
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Cathrine Wilhelmsen
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Cathrine Wilhelmsen
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Cathrine Wilhelmsen
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Cathrine Wilhelmsen
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
Cathrine Wilhelmsen
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Cathrine Wilhelmsen
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Cathrine Wilhelmsen
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Cathrine Wilhelmsen
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
Cathrine Wilhelmsen
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Cathrine Wilhelmsen
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
Cathrine Wilhelmsen
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Cathrine Wilhelmsen
 
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
 

Recently uploaded

06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus
Timothy Spann
 
8 things to know before you start to code in 2024
8 things to know before you start to code in 20248 things to know before you start to code in 2024
8 things to know before you start to code in 2024
ArianaRamos54
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
uevausa
 
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
blueshagoo1
 
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
nyvan3
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
aguty
 
Sample Devops SRE Product Companies .pdf
Sample Devops SRE  Product Companies .pdfSample Devops SRE  Product Companies .pdf
Sample Devops SRE Product Companies .pdf
Vineet
 
Data Scientist Machine Learning Profiles .pdf
Data Scientist Machine Learning  Profiles .pdfData Scientist Machine Learning  Profiles .pdf
Data Scientist Machine Learning Profiles .pdf
Vineet
 
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Marlon Dumas
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
osoyvvf
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
Bisnar Chase Personal Injury Attorneys
 
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
hyfjgavov
 
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
oaxefes
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
hqfek
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
vasanthatpuram
 
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdfreading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
perranet1
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
eudsoh
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
GeorgiiSteshenko
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
zsafxbf
 

Recently uploaded (20)

06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus
 
8 things to know before you start to code in 2024
8 things to know before you start to code in 20248 things to know before you start to code in 2024
8 things to know before you start to code in 2024
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
 
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
 
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
 
Sample Devops SRE Product Companies .pdf
Sample Devops SRE  Product Companies .pdfSample Devops SRE  Product Companies .pdf
Sample Devops SRE Product Companies .pdf
 
Data Scientist Machine Learning Profiles .pdf
Data Scientist Machine Learning  Profiles .pdfData Scientist Machine Learning  Profiles .pdf
Data Scientist Machine Learning Profiles .pdf
 
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
 
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
 
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
 
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdfreading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
 

Biml Tips and Tricks: Not Just for SSIS Packages! (SQLGrillen 2018)

  • 1. Biml Tips and Tricks Not just for SSIS Packages! Cathrine Wilhelmsen SQLGrillen · June 22nd 2018
  • 2. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
  • 3. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Session Description "Wait, what? Biml is not just for generating SSIS packages?" Absolutely not! Come and see how you can use Biml (Business Intelligence Markup Language) to save time and speed up other Data Warehouse development tasks. You can generate complex T-SQL statements with Biml instead of using dynamic SQL, create test data and populate static dimensions, and even compare tables and views across multiple servers and databases. Don't Repeat Yourself, start automating those boring, manual tasks today!
  • 4. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Cathrine Wilhelmsen @cathrinew cathrinew.net Data Warehouse & Business Intelligence Consultant
  • 5. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) …the next 60 minutes… T-SQL Test Data Comparing Dimensions
  • 6. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
  • 7. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Wait… Can't I use <something else>?
  • 8. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Of course! But Biml works well with source metadata
  • 9. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) And… Biml is fun!
  • 10. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) But… Biml? How?
  • 11. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Traditional Biml
  • 12. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Crazy Fun Biml
  • 13. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) What do you need?
  • 14. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Preview Pane The power is in the…
  • 15. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Traditional BimlScript
  • 16. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Traditional BimlScript <# foreach (var table in RootNode.Tables) { #> <Package Name="Load_<#=table.Name#>" /> <# } #> <Package Name="Load_Customer" /> <Package Name="Load_Product" /> <Package Name="Load_Sales" />
  • 17. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) BimlScript to Biml Preview Pane
  • 18. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) BimlScript to... ??? Preview Pane
  • 19. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Crazy Fun BimlScript <# foreach (var table in RootNode.Tables) { #> Yay, a package! <#=table.Name#> :) <# } #> Yay, a package! Load_Customer :) Yay, a package! Load_Product :) Yay, a package! Load_Sales :)
  • 20. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net)
  • 21. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) T-SQL from Biml
  • 22. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Built-in T-SQL from Biml
  • 23. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Built-in T-SQL from Biml Functional T-SQL GetSelectSql GetDropAndCreateDdl GetTableSql SSMS Snippets GetInsertSql GetUpdateSql GetDeleteSql
  • 24. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Functional T-SQL: GetSelectSql SELECT [Column1] ,[Column2] FROM [Schema].[Table]
  • 25. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Functional T-SQL: GetDropAndCreateDdl IF EXISTS (...) DROP TABLE [Schema].[Table] GO CREATE TABLE [Schema].[Table] ( [Column1] int IDENTITY(1,1) NOT NULL ,[Column2] nvarchar(50) NOT NULL -- Constraints )
  • 26. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Functional T-SQL: GetTableSql IF EXISTS (...) DROP TABLE [Schema].[Table] GO CREATE TABLE [Schema].[Table] ( [Column1] int IDENTITY(1,1) NOT NULL ,[Column2] nvarchar(50) NOT NULL -- Constraints )
  • 27. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Functional T-SQL Use directly in SSIS Copy and run in SSMS
  • 28. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) SSMS Snippets: GetDeleteSql DELETE FROM [Schema].[Table] WHERE <Search Conditions,,>
  • 29. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) SSMS Snippets: GetInsertSql INSERT INTO [Schema].[Table] ( [Column1] ,[Column1] ) VALUES ( <Column1,int,>, <Column2,nvarchar(50),> )
  • 30. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) SSMS Snippets: GetUpdateSql UPDATE [Schema].[Table] SET [Column1] = <Column1, int,>, [Column2] = <Column2, nvarchar(50),> WHERE <Search Conditions,,>
  • 31. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Copy and run in SSMS Replace Template Parameters with actual values CTRL SHIFT M SSMS Snippets
  • 32. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Custom T-SQL from Biml
  • 33. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Custom T-SQL from Biml Table Metadata <#=table.Name#> <#=table.SchemaQualifiedName#> SELECT *
  • 34. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Custom T-SQL from Biml Column Methods GetColumnList GetColumnAssignmentList GetColumnComparisonList
  • 35. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Column Methods Return code fragments Use as building blocks in custom T-SQL Filter columns by using lambda expressions
  • 36. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Column Methods Return code fragments Use as building blocks in custom T-SQL Filter columns by using lambda expressions
  • 37. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions "A lambda expression is an anonymous function that you can use to create delegates or expression tree types"
  • 38. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions "A lambda expression is an anonymous function that you can use to create delegates or expression tree types"
  • 39. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions column => column.Name == "ColumnID"
  • 40. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions column => column.Name == "ColumnID" The arrow is the lambda operator
  • 41. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions column => column.Name == "ColumnID" Input parameter is on the left side
  • 42. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions column => column.Name == "ColumnID" Expression is on the right side
  • 43. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Lambda Expressions column => column.Name == "ColumnID"
  • 44. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) GetColumnList Get columns for SELECT [Column1], [Column2], [Column3]
  • 45. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) GetColumnList GetColumnList() GetColumnList("src") GetColumnList(c => c.Name == "ColumnID") GetColumnList(c => c.DataType != DbType.Guid, "src")
  • 46. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) GetColumnAssignmentList Get columns for UPDATE … SET [l].[Column1] = [r].[Column1] ,[l].[Column2] = [r].[Column2] ,[l].[Column3] = [r].[Column3]
  • 47. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) GetColumnAssignmentList GetColumnAssignmentList() GetColumnAssignmentList("dst", "src") GetColumnAssignmentList(c => !c.IsUsedInPrimaryKey) GetColumnAssignmentList(c => !c.IsUsedInPrimaryKey, "dst", "src")
  • 48. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) GetColumnComparisonList Get columns for JOIN … ON [l].[Column1] = [r].[Column1] AND [l].[Column2] = [r].[Column2] AND [l].[Column3] = [r].[Column3]
  • 49. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) GetColumnComparisonList GetColumnComparisonList() GetColumnComparisonList("!=") GetColumnComparisonList("dst", "src") GetColumnComparisonList(c => c.IsUsedInPrimaryKey) GetColumnComparisonList(c => c.IsUsedInPrimaryKey, "dst", "src") GetColumnComparisonList(c => c.IsUsedInPrimaryKey, "!=", "dst", "src")
  • 50. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) DEMO T-SQL from Biml
  • 51. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Test Data
  • 52. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml + Random() Create Random Test Data
  • 53. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Random Class Random() Pseudo-random number generator Random enough for practical purposes Random(12345) Always generates the same series of random numbers
  • 54. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Random Methods Next() Random number Next(100) Random number smaller than specified value Next(1000, 2000) Random number within specified range NextDouble() Random decimal number between 0.0 and 1.0
  • 55. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml Helper Method <#+ Random r = new Random(); public string GetTestData(AstTableColumnBaseNode column) { switch (column.DataType) { case DbType.Int32 : return r.Next(Int32.MaxValue).ToString(); case DbType.String : return "'" + GetRandomString(column.Length) + "'" default : return "NULL"; } } #>
  • 56. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml Helper Helper Method <#+ private string GetRandomString(int length) { var allowedChars = "ABC123"; var randomChars = new char[length]; for (int i = 0; i < length; i++) { randomChars[i] = allowedChars[ r.Next(allowedChars.Length) ]; } return new String(randomCharacters); } #>
  • 57. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml + Random() Very limited functionality No dependencies Works out-of-the box
  • 58. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml + Bogus Create Random and Specific Test Data
  • 59. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Bogus Project Simple and sane fake data generator for .NET https://github.com/bchavez/Bogus Advanced functionality for custom objects …or simple functionality for Biml hacks :)
  • 60. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Address Commerce Company Database Date Finance Internet Lorem Name Person Phone System Bogus API Supports all data types plus built-in test data, including:
  • 61. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) The Bogus Methods <#@ assembly name="Bogus.dll" #> <#@ import namespace="Bogus" #> <# var f = new Faker(); #> <# Person p = new Person(); #> INSERT INTO dbo.Employee(FirstName, LastName, Birthday) VALUES ( '<#=p.FirstName#>', '<#=p.LastName#>', '<#=p.DateOfBirth.ToString("yyyyMMdd")#>' )
  • 62. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml + Bogus() Better functionality than Random() Easier to understand, frequently updated Requires additional files
  • 63. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Installing Bogus Install via Nuget: Install-Package Bogus Or download latest release .zip: https://github.com/bchavez/Bogus/releases
  • 64. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) DEMO Test Data with Random() and Bogus
  • 65. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Alternative (Better) Solutions Mockaroo (free, but only 1000 rows) https://mockaroo.com/ Redgate SQL Data Generator (licensed) https://www.red-gate.com/products/sql- development/sql-data-generator/index
  • 66. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Static Dimensions
  • 67. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Static Dimensions (or any kind of static table) SCD Type 0: Not changing Lookup Tables 1. Create table definition 2. Add static source rows 3. Generate INSERT statements
  • 68. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml Static Sources Part of <Table> objects Defines rows to be inserted when table is created • Unknown dimension members • Date dimension • Code tables
  • 69. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml Static Source <Table Name="Table" SchemaName="Database.Schema"> <Columns> <Column Name="Column1" DataType="Int32" /> <Column Name="Column2" DataType="String" Length="10" /> </Columns> <Sources> <StaticSource Name="TableRows"> <Rows> <Row> <ColumnValues> <ColumnValue ColumnName="Column1" Value="-1" /> <ColumnValue ColumnName="Column2" Value="'Unknown'" /> </ColumnValues> </Row> </Rows> </StaticSource> </Sources> </Table>
  • 70. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Biml Static Source <# if (table.Sources.Any()) { #> <#=TableToPackageLowerer.GetStaticSourceInsertStatements( table.SchemaQualifiedName, table.HasIdentity, (AstTableStaticSourceNode)table.Sources.FirstOrDefault() )#> <# } #>
  • 71. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Drop and Create Facts and Dimensions
  • 72. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Could not drop object 'Schema.Table' because it is referenced by a FOREIGN KEY constraint.
  • 73. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Drop and Create Create Order: Dim1 Dim2 Fact1 Fact2 Fact2 Fact1 Dim2 Dim1Drop Order:
  • 74. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Drop and Create Create Order: Dim1 Dim2 Fact1 Fact2 Fact2 Fact1 Dim2 Dim1Drop Order:
  • 75. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Topological Sort "A topological sort of a directed graph is a linear ordering of its vertices such that for every directed edge uv from vertex u to vertex v, u comes before v in the ordering"
  • 76. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Topological Sort "A topological sort of a directed graph is a linear ordering of its vertices such that for every directed edge uv from vertex u to vertex v, u comes before v in the ordering"
  • 77. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Topological Sort "Sorts objects by dependencies. For example dimensions before fact tables."
  • 78. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Topological Sort "Sorts objects by dependencies. For example dimensions before fact tables."
  • 79. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Topological Sort IEnumerable<T> DependencyAnalysis.TopologicalSort<T> ( IEnumerable<T> source, Expression<Func<T, T>> relationProperty )
  • 80. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Topological Sort var dependentTables = DependencyAnalysis.TopologicalSort<AstTableNode> ( RootNode.Tables, t => t.Columns .OfType<AstTableColumnTableReferenceNode>() .Select(c => c.ForeignTable) );
  • 81. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) DEMO Static Dimensions and Topological Sort
  • 82. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Comparing Databases
  • 83. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Once upon a time…
  • 84. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Panic!
  • 85. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) How to Compare Tables and Columns
  • 86. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) LINQ Sort OrderBy, ThenBy Filter Where, OfType Group GroupBy Aggregate Count, Sum Check Collections All, Any, Contains Set Operations Except, Intersect Project Collections Select, SelectMany
  • 87. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) LINQ Sort OrderBy, ThenBy Filter Where, OfType Group GroupBy Aggregate Count, Sum Check Collections All, Any, Contains Set Operations Except, Intersect Project Collections Select, SelectMany
  • 88. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) LINQ: Filter Collections Where() Returns the filtered collection with all elements that meet the criteria table.Where(t => t.Schema.Name == "dim")
  • 89. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) LINQ: Set Operations Except() Returns elements in first collection that are not in second collection table1.Except(table2)
  • 90. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) LINQ: Set Operations Intersect() Returns elements in first collection that are also in second collection table1.Intersect(table2)
  • 91. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) LINQ: Project Collections Select() Creates a new collection from specified attributes table.Select(t => t.Name)
  • 92. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) DEMO Comparing Databases
  • 93. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Alternative (Better) Solutions Redgate SQL Compare (licensed) https://www.red-gate.com/products/sql- development/sql-compare/
  • 94. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) …the past 60 minutes… T-SQL Test Data Comparing Dimensions
  • 95. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Is this useful?
  • 96. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) What other ideas do you have?
  • 97. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) Have fun!
  • 98. © 2018 Cathrine Wilhelmsen (contact@cathrinewilhelmsen.net) @cathrinew cathrinew.net hi@cathrinew.net Biml resources and demo files: cathrinew.net/biml