SlideShare a Scribd company logo
1 of 21
Download to read offline
Biml Academy 2
Importing, Staging, and Loading Data with Biml
Andy Leonard · Scott Currie · Cathrine Wilhelmsen
Biml Academy 2
Lesson 5: Importing source metadata into Biml (Cathrine)
Lesson 6: Staging data with Biml (Andy)
Lesson 7: Metadata-driven staging pattern with Biml (Andy)
Lesson 8: BimlFlex for Data Vault data warehouses (Scott)
Lesson 9: Biml Q&A with several BimlHeroes!
Cathrine Wilhelmsen
@cathrinew
cathrinew.net
Data Warehouse Architect
Business Intelligence Developer
Lesson 5:
Importing source metadata into Biml
Methods for importing metadata
ImportTableNodes
ImportDB
GetDatabaseSchema
Methods for importing metadata
ImportTableNodes
Import tables and views in one schema
Filter tables using LIKE % wildcard syntax
Methods for importing metadata
ImportDB
Import all schemas, tables and views
Filter using LIKE % wildcard syntax
Methods for importing metadata
GetDatabaseSchema
Import all schemas, tables and views
Filter using collections
Why ImportDB and GetDatabaseSchema?
ImportDB / ImportTableNodes
• SELECT * on each imported table
• SELECT to get defaults, indexes etc.
• Using SchemaOnly mode
• Doesn't actually run the query, but
returns schema for the result set
GetDatabaseSchema
• Figures out target database type
(SQL Server, Oracle, Teradata, DB2,
PostgreSQL, Ingres, OpenEdge etc.)
• Queries the relevant information
schema for that database type
A few queries for the entire importMultiple queries per table
But why keep both if one is faster?
Support for legacy scripts :)
They can return different results – where both are valid/correct
Different data type mapping defaults in different systems
(Bulk Copy, SSIS and Excel are all different)
Biml supports the two most commonly used approaches
Methods and parameters
ImportTableNodes(schema, tableFilter)
ImportTableNodes(schema, tableFilter, importOptions)
ImportDB()
ImportDB(schemaFilter, tableFilter)
ImportDB(schemaFilter, tableFilter, importOptions)
GetDatabaseSchema()
GetDatabaseSchema(importOptions)
GetDatabaseSchema(includedSchemas, includedTables, importOptions)
Methods and parameters: string value
ImportTableNodes(schema, tableFilter)
ImportTableNodes(schema, tableFilter, importOptions)
ImportDB()
ImportDB(schemaFilter, tableFilter)
ImportDB(schemaFilter, tableFilter, importOptions)
GetDatabaseSchema()
GetDatabaseSchema(importOptions)
GetDatabaseSchema(includedSchemas, includedTables, importOptions)
Methods and parameters: string filter
ImportTableNodes(schema, tableFilter)
ImportTableNodes(schema, tableFilter, importOptions)
ImportDB()
ImportDB(schemaFilter, tableFilter)
ImportDB(schemaFilter, tableFilter, importOptions)
GetDatabaseSchema()
GetDatabaseSchema(importOptions)
GetDatabaseSchema(includedSchemas, includedTables, importOptions)
Methods and parameters: collections
ImportTableNodes(schema, tableFilter)
ImportTableNodes(schema, tableFilter, importOptions)
ImportDB()
ImportDB(schemaFilter, tableFilter)
ImportDB(schemaFilter, tableFilter, importOptions)
GetDatabaseSchema()
GetDatabaseSchema(importOptions)
GetDatabaseSchema(includedSchemas, includedTables, importOptions)
Methods and parameters: ImportOptions
ImportTableNodes(schema, tableFilter)
ImportTableNodes(schema, tableFilter, importOptions)
ImportDB()
ImportDB(schemaFilter, tableFilter)
ImportDB(schemaFilter, tableFilter, importOptions)
GetDatabaseSchema()
GetDatabaseSchema(importOptions)
GetDatabaseSchema(includedSchemas, includedTables, importOptions)
ImportOptions specify objects to exclude
Don't exclude anything
Views
Identity Specifications
Primary Keys
Foreign Keys
Unique Keys
Indexes
Column Defaults
Check Constraints
ImportOptions.None
ImportOptions.ExcludeViews
ImportOptions.ExcludeIdentity
ImportOptions.ExcludePrimaryKey
ImportOptions.ExcludeForeignKey
ImportOptions.ExcludeUniqueKey
ImportOptions.ExcludeIndex
ImportOptions.ExcludeColumnDefault
ImportOptions.ExcludeCheckConstraint
Demo time!
Additional methods for importing metadata
GenerateSchemaNode
Returns AstSchemaNode
GenerateSchemaNodes
Returns IEnumerable<AstSchemaNode>
Additional methods for importing metadata
GenerateTableNode
Returns AstTableNode
GenerateTableNodes
Returns IEnumerable<AstTableNode>
Demo time!
Biml Academy 2
Lesson 5: Importing source metadata into Biml (Cathrine)
Lesson 6: Staging data with Biml (Andy)
Lesson 7: Metadata-driven staging pattern with Biml (Andy)
Lesson 8: BimlFlex for delivering Data Vault data warehouses (Scott)
Lesson 9: Biml Q&A with several BimlHeroes!

More Related Content

What's hot

Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...Cathrine Wilhelmsen
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...Cathrine Wilhelmsen
 
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...Cathrine Wilhelmsen
 
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...Cathrine Wilhelmsen
 
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)Cathrine Wilhelmsen
 
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Cathrine Wilhelmsen
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Cathrine Wilhelmsen
 
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )Cathrine Wilhelmsen
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)Cathrine Wilhelmsen
 
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Cathrine Wilhelmsen
 
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Cathrine Wilhelmsen
 
BIML: BI to the next level
BIML: BI to the next levelBIML: BI to the next level
BIML: BI to the next levelDavide Mauri
 
MWLUG 2016 : AD117 : Xpages & jQuery DataTables
MWLUG 2016 : AD117 : Xpages & jQuery DataTablesMWLUG 2016 : AD117 : Xpages & jQuery DataTables
MWLUG 2016 : AD117 : Xpages & jQuery DataTablesMichael Smith
 
Lessons learned from the worlds largest XPage project
Lessons learned from the worlds largest XPage projectLessons learned from the worlds largest XPage project
Lessons learned from the worlds largest XPage projectMark Roden
 
Real World Caching with Ruby on Rails
Real World Caching with Ruby on RailsReal World Caching with Ruby on Rails
Real World Caching with Ruby on RailsDavid Roberts
 

What's hot (20)

Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
Biml for Beginners: Speed up your SSIS development (Malta Microsoft Data Plat...
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Ex...
 
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
Generate SSIS packages automatically with Biml and BimlScript (SQLKonferenz 2...
 
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
S.M.A.R.T. Biml - Standardize, Model, Automate, Reuse and Transform (SQLSatur...
 
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
Level Up Your Biml: Best Practices and Coding Techniques (TUGA IT 2016)
 
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
Don't Repeat Yourself - Agile SSIS Development with Biml and BimlScript (SQL ...
 
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
Biml for Beginners - Generating SSIS Packages with BimlScript (SQLSaturday Go...
 
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)Biml for Beginners: Speed up your SSIS development (SQLBits XV)
Biml for Beginners: Speed up your SSIS development (SQLBits XV)
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Chicago)
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Vienna)
 
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
Upgrading from SSIS Package Deployment to Project Deployment (SQLSaturday Den...
 
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
Biml for Beginners: Speed up your SSIS development (SQL PASS Edmonton )
 
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
Biml for Beginners: Speed up your SSIS development (SQLSaturday Tallinn)
 
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
Tools and Tips: From Accidental to Efficient Data Warehouse Developer (SQLSat...
 
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
Biml for Beginners: Script and Automate SSIS development (Capital Area SQL Se...
 
BIML: BI to the next level
BIML: BI to the next levelBIML: BI to the next level
BIML: BI to the next level
 
MWLUG 2016 : AD117 : Xpages & jQuery DataTables
MWLUG 2016 : AD117 : Xpages & jQuery DataTablesMWLUG 2016 : AD117 : Xpages & jQuery DataTables
MWLUG 2016 : AD117 : Xpages & jQuery DataTables
 
Lessons learned from the worlds largest XPage project
Lessons learned from the worlds largest XPage projectLessons learned from the worlds largest XPage project
Lessons learned from the worlds largest XPage project
 
Real World Caching with Ruby on Rails
Real World Caching with Ruby on RailsReal World Caching with Ruby on Rails
Real World Caching with Ruby on Rails
 
Azure and Umbraco CMS
Azure and Umbraco CMSAzure and Umbraco CMS
Azure and Umbraco CMS
 

Similar to Biml Academy 2 - Lesson 5: Importing source metadata into Biml

QTP Automation Testing Tutorial 7
QTP Automation Testing Tutorial 7QTP Automation Testing Tutorial 7
QTP Automation Testing Tutorial 7Akash Tyagi
 
Intake 38 data access 5
Intake 38 data access 5Intake 38 data access 5
Intake 38 data access 5Mahmoud Ouf
 
Acutate - Using Stored Procedure
Acutate - Using Stored ProcedureAcutate - Using Stored Procedure
Acutate - Using Stored ProcedureAishwarya Savant
 
Building Modern Websites with ASP.NET by Rachel Appel
Building Modern Websites with ASP.NET by Rachel AppelBuilding Modern Websites with ASP.NET by Rachel Appel
Building Modern Websites with ASP.NET by Rachel Appel.NET Conf UY
 
Advanced integration services on microsoft ssis 1
Advanced integration services on microsoft ssis 1Advanced integration services on microsoft ssis 1
Advanced integration services on microsoft ssis 1Skillwise Group
 
OWB11gR2 - Extending ETL
OWB11gR2 - Extending ETL OWB11gR2 - Extending ETL
OWB11gR2 - Extending ETL Suraj Bang
 
Basics of Model/View Qt programming
Basics of Model/View Qt programmingBasics of Model/View Qt programming
Basics of Model/View Qt programmingICS
 
Part2 Best Practices for Managing Optimizer Statistics
Part2 Best Practices for Managing Optimizer StatisticsPart2 Best Practices for Managing Optimizer Statistics
Part2 Best Practices for Managing Optimizer StatisticsMaria Colgan
 
Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Stamatis Zampetakis
 
Custom Star Creation for Ellucain's Enterprise Data Warehouse
Custom Star Creation for Ellucain's Enterprise Data WarehouseCustom Star Creation for Ellucain's Enterprise Data Warehouse
Custom Star Creation for Ellucain's Enterprise Data WarehouseBryan L. Mack
 
Database Management System - SQL beginner Training
Database Management System - SQL beginner Training Database Management System - SQL beginner Training
Database Management System - SQL beginner Training Moutasm Tamimi
 
Real-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSASReal-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSASLynn Langit
 
JAM819 - Native API Deep Dive: Data Storage and Retrieval
JAM819 - Native API Deep Dive: Data Storage and RetrievalJAM819 - Native API Deep Dive: Data Storage and Retrieval
JAM819 - Native API Deep Dive: Data Storage and RetrievalDr. Ranbijay Kumar
 
Go faster with_native_compilation Part-2
Go faster with_native_compilation Part-2Go faster with_native_compilation Part-2
Go faster with_native_compilation Part-2Rajeev Rastogi (KRR)
 
Go Faster With Native Compilation
Go Faster With Native CompilationGo Faster With Native Compilation
Go Faster With Native CompilationPGConf APAC
 

Similar to Biml Academy 2 - Lesson 5: Importing source metadata into Biml (20)

QTP Automation Testing Tutorial 7
QTP Automation Testing Tutorial 7QTP Automation Testing Tutorial 7
QTP Automation Testing Tutorial 7
 
Sql server-function
Sql server-functionSql server-function
Sql server-function
 
Intake 38 data access 5
Intake 38 data access 5Intake 38 data access 5
Intake 38 data access 5
 
Acutate - Using Stored Procedure
Acutate - Using Stored ProcedureAcutate - Using Stored Procedure
Acutate - Using Stored Procedure
 
Intake 37 ef2
Intake 37 ef2Intake 37 ef2
Intake 37 ef2
 
Quick dive to pandas
Quick dive to pandasQuick dive to pandas
Quick dive to pandas
 
Building Modern Websites with ASP.NET by Rachel Appel
Building Modern Websites with ASP.NET by Rachel AppelBuilding Modern Websites with ASP.NET by Rachel Appel
Building Modern Websites with ASP.NET by Rachel Appel
 
Advanced integration services on microsoft ssis 1
Advanced integration services on microsoft ssis 1Advanced integration services on microsoft ssis 1
Advanced integration services on microsoft ssis 1
 
OWB11gR2 - Extending ETL
OWB11gR2 - Extending ETL OWB11gR2 - Extending ETL
OWB11gR2 - Extending ETL
 
Basics of Model/View Qt programming
Basics of Model/View Qt programmingBasics of Model/View Qt programming
Basics of Model/View Qt programming
 
Part2 Best Practices for Managing Optimizer Statistics
Part2 Best Practices for Managing Optimizer StatisticsPart2 Best Practices for Managing Optimizer Statistics
Part2 Best Practices for Managing Optimizer Statistics
 
Go Faster With Native Compilation
Go Faster With Native CompilationGo Faster With Native Compilation
Go Faster With Native Compilation
 
Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21Apache Calcite Tutorial - BOSS 21
Apache Calcite Tutorial - BOSS 21
 
Custom Star Creation for Ellucain's Enterprise Data Warehouse
Custom Star Creation for Ellucain's Enterprise Data WarehouseCustom Star Creation for Ellucain's Enterprise Data Warehouse
Custom Star Creation for Ellucain's Enterprise Data Warehouse
 
Developing a new Epsilon EMC driver
Developing a new Epsilon EMC driverDeveloping a new Epsilon EMC driver
Developing a new Epsilon EMC driver
 
Database Management System - SQL beginner Training
Database Management System - SQL beginner Training Database Management System - SQL beginner Training
Database Management System - SQL beginner Training
 
Real-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSASReal-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSAS
 
JAM819 - Native API Deep Dive: Data Storage and Retrieval
JAM819 - Native API Deep Dive: Data Storage and RetrievalJAM819 - Native API Deep Dive: Data Storage and Retrieval
JAM819 - Native API Deep Dive: Data Storage and Retrieval
 
Go faster with_native_compilation Part-2
Go faster with_native_compilation Part-2Go faster with_native_compilation Part-2
Go faster with_native_compilation Part-2
 
Go Faster With Native Compilation
Go Faster With Native CompilationGo Faster With Native Compilation
Go Faster With Native Compilation
 

More from Cathrine Wilhelmsen

Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Cathrine Wilhelmsen
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Cathrine Wilhelmsen
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Cathrine Wilhelmsen
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Cathrine Wilhelmsen
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)Cathrine Wilhelmsen
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Cathrine Wilhelmsen
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Cathrine Wilhelmsen
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Cathrine Wilhelmsen
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...Cathrine Wilhelmsen
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Cathrine Wilhelmsen
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)Cathrine Wilhelmsen
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
 

Recently uploaded

Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBoston Institute of Analytics
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Boston Institute of Analytics
 
DATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etcDATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etclalithasri22
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaManalVerma4
 
Role of Consumer Insights in business transformation
Role of Consumer Insights in business transformationRole of Consumer Insights in business transformation
Role of Consumer Insights in business transformationAnnie Melnic
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...Jack Cole
 
Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...
Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...
Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...ThinkInnovation
 
Statistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfStatistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfnikeshsingh56
 
Presentation of project of business person who are success
Presentation of project of business person who are successPresentation of project of business person who are success
Presentation of project of business person who are successPratikSingh115843
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...Dr Arash Najmaei ( Phd., MBA, BSc)
 
Digital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfDigital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfNicoChristianSunaryo
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelBoston Institute of Analytics
 
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...ThinkInnovation
 

Recently uploaded (16)

Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
 
DATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etcDATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etc
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in India
 
Role of Consumer Insights in business transformation
Role of Consumer Insights in business transformationRole of Consumer Insights in business transformation
Role of Consumer Insights in business transformation
 
Data Analysis Project: Stroke Prediction
Data Analysis Project: Stroke PredictionData Analysis Project: Stroke Prediction
Data Analysis Project: Stroke Prediction
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
 
Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...
Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...
Decision Making Under Uncertainty - Is It Better Off Joining a Partnership or...
 
Statistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfStatistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdf
 
Presentation of project of business person who are success
Presentation of project of business person who are successPresentation of project of business person who are success
Presentation of project of business person who are success
 
2023 Survey Shows Dip in High School E-Cigarette Use
2023 Survey Shows Dip in High School E-Cigarette Use2023 Survey Shows Dip in High School E-Cigarette Use
2023 Survey Shows Dip in High School E-Cigarette Use
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
 
Digital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfDigital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdf
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
 
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
Predictive Analysis - Using Insight-informed Data to Plan Inventory in Next 6...
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 

Biml Academy 2 - Lesson 5: Importing source metadata into Biml

  • 1. Biml Academy 2 Importing, Staging, and Loading Data with Biml Andy Leonard · Scott Currie · Cathrine Wilhelmsen
  • 2. Biml Academy 2 Lesson 5: Importing source metadata into Biml (Cathrine) Lesson 6: Staging data with Biml (Andy) Lesson 7: Metadata-driven staging pattern with Biml (Andy) Lesson 8: BimlFlex for Data Vault data warehouses (Scott) Lesson 9: Biml Q&A with several BimlHeroes!
  • 3. Cathrine Wilhelmsen @cathrinew cathrinew.net Data Warehouse Architect Business Intelligence Developer
  • 4. Lesson 5: Importing source metadata into Biml
  • 5. Methods for importing metadata ImportTableNodes ImportDB GetDatabaseSchema
  • 6. Methods for importing metadata ImportTableNodes Import tables and views in one schema Filter tables using LIKE % wildcard syntax
  • 7. Methods for importing metadata ImportDB Import all schemas, tables and views Filter using LIKE % wildcard syntax
  • 8. Methods for importing metadata GetDatabaseSchema Import all schemas, tables and views Filter using collections
  • 9. Why ImportDB and GetDatabaseSchema? ImportDB / ImportTableNodes • SELECT * on each imported table • SELECT to get defaults, indexes etc. • Using SchemaOnly mode • Doesn't actually run the query, but returns schema for the result set GetDatabaseSchema • Figures out target database type (SQL Server, Oracle, Teradata, DB2, PostgreSQL, Ingres, OpenEdge etc.) • Queries the relevant information schema for that database type A few queries for the entire importMultiple queries per table
  • 10. But why keep both if one is faster? Support for legacy scripts :) They can return different results – where both are valid/correct Different data type mapping defaults in different systems (Bulk Copy, SSIS and Excel are all different) Biml supports the two most commonly used approaches
  • 11. Methods and parameters ImportTableNodes(schema, tableFilter) ImportTableNodes(schema, tableFilter, importOptions) ImportDB() ImportDB(schemaFilter, tableFilter) ImportDB(schemaFilter, tableFilter, importOptions) GetDatabaseSchema() GetDatabaseSchema(importOptions) GetDatabaseSchema(includedSchemas, includedTables, importOptions)
  • 12. Methods and parameters: string value ImportTableNodes(schema, tableFilter) ImportTableNodes(schema, tableFilter, importOptions) ImportDB() ImportDB(schemaFilter, tableFilter) ImportDB(schemaFilter, tableFilter, importOptions) GetDatabaseSchema() GetDatabaseSchema(importOptions) GetDatabaseSchema(includedSchemas, includedTables, importOptions)
  • 13. Methods and parameters: string filter ImportTableNodes(schema, tableFilter) ImportTableNodes(schema, tableFilter, importOptions) ImportDB() ImportDB(schemaFilter, tableFilter) ImportDB(schemaFilter, tableFilter, importOptions) GetDatabaseSchema() GetDatabaseSchema(importOptions) GetDatabaseSchema(includedSchemas, includedTables, importOptions)
  • 14. Methods and parameters: collections ImportTableNodes(schema, tableFilter) ImportTableNodes(schema, tableFilter, importOptions) ImportDB() ImportDB(schemaFilter, tableFilter) ImportDB(schemaFilter, tableFilter, importOptions) GetDatabaseSchema() GetDatabaseSchema(importOptions) GetDatabaseSchema(includedSchemas, includedTables, importOptions)
  • 15. Methods and parameters: ImportOptions ImportTableNodes(schema, tableFilter) ImportTableNodes(schema, tableFilter, importOptions) ImportDB() ImportDB(schemaFilter, tableFilter) ImportDB(schemaFilter, tableFilter, importOptions) GetDatabaseSchema() GetDatabaseSchema(importOptions) GetDatabaseSchema(includedSchemas, includedTables, importOptions)
  • 16. ImportOptions specify objects to exclude Don't exclude anything Views Identity Specifications Primary Keys Foreign Keys Unique Keys Indexes Column Defaults Check Constraints ImportOptions.None ImportOptions.ExcludeViews ImportOptions.ExcludeIdentity ImportOptions.ExcludePrimaryKey ImportOptions.ExcludeForeignKey ImportOptions.ExcludeUniqueKey ImportOptions.ExcludeIndex ImportOptions.ExcludeColumnDefault ImportOptions.ExcludeCheckConstraint
  • 18. Additional methods for importing metadata GenerateSchemaNode Returns AstSchemaNode GenerateSchemaNodes Returns IEnumerable<AstSchemaNode>
  • 19. Additional methods for importing metadata GenerateTableNode Returns AstTableNode GenerateTableNodes Returns IEnumerable<AstTableNode>
  • 21. Biml Academy 2 Lesson 5: Importing source metadata into Biml (Cathrine) Lesson 6: Staging data with Biml (Andy) Lesson 7: Metadata-driven staging pattern with Biml (Andy) Lesson 8: BimlFlex for delivering Data Vault data warehouses (Scott) Lesson 9: Biml Q&A with several BimlHeroes!