SlideShare a Scribd company logo
1 of 46
Download to read offline
Cathrine Wilhelmsen, Senior BI Consultant, Inmeta
Moderated By: Giuliana Grecco
Pipelines and Packages:
Introduction to
Azure Data Factory
If you require assistance
during the session, type
your inquiry into the
question pane on the right
side.
Maximize your screen with
the zoom button on the
top of the presentation
window.
Please fill in the short
evaluation following the
session. It will appear in
your web browser.
Technical Assistance
Thank you to our Presenting Sponsors
PASS’ flagship event
November 5-8
Seattle, Washington
Free 1-day local
training events
Local user groups
around the world
Online special interest
user groups
Business analytics
training
Get involved
Explore
everything
PASS has
to offer
Free Online Resources
Newsletters
PASS.org
Cathrine Wilhelmsen
Senior BI Consultant, Inmeta
Work Things
Senior BI Consultant and Microsoft Data
Platform Tech Lead in Inmeta, Norway.
Specialties: ADF, SSIS, Biml and T-SQL
Geeky Things
Fun Things
Active in the PASS community as a speaker,
blogger and chronic volunteer.
Likes: coffee, chocolate and cat gifs :)
/cathrinewilhelmsen
@cathrinew
cathrinew.net
hi@cathrinew.net
Cathrine Wilhelmsen, Senior BI Consultant, Inmeta
Moderated By: Giuliana Grecco
Pipelines and Packages:
Introduction to
Azure Data Factory
Past Learnings
and Future Visions
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
10 years ago…
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
SSIS
SQL Server Integration Services
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Then…
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
ADF v1
Azure Data Factory Version 1
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Today…
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
ADF v2
Azure Data Factory Version 2
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Why?
How?
What?
Azure
Data Factory
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Azure Data Factory
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Movement and Integration
Azure Data Factory
Pipelines
Activities
Datasets
Data Flows Triggers
Integration RuntimesLinked Services
Templates
DEMO
Azure
Data Factory
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Wait…
I already have
thousands of SSIS
packages!
SSIS
Lift and Shift
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Why Lift and Shift SSIS?
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
1.
2.
3.
4.
How to Lift and Shift SSIS
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
SSIS Integration Runtime
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
https://sqlbits.com/Sessions/Event18/Deeper_Integration
_and_New_Transformation_for_SSIS_in_ADF
SSIS Integration Runtime Deep Dive
DEMO
SSIS
Lift and Shift
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
SSIS Lift and Shift: Lessons Learned
like all VMs
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
≈
≈
≈
≈
≈
≈
ADF vs SSIS
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline
Linked Service
Source
Sink
Activity
Data Flow
≈
≈
≈
≈
≈
≈
Package
Connection Manager
Source
Destination
Control Flow Task
Data Flow
ADF vs SSIS
Mapping
Data Flows
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Mapping Data Flows
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Mapping Data Flows Example
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Kamil Nowinski's Cheat Sheet
github.com/SQLPlayer/CheatSheets/blob/master/ADFDF-Cheat-Sheet-sqlplayer.pdf
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Schema Drift
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Schema Drift in SSIS
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Schema Drift in ADF
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Oh no!
DEMO
Mapping
Data Flows
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Lessons Learned
© 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Good luck!
Thank you :)
Questions?
Thank you for attending!
@sqlpass
#sqlpass
@PASScommunity
Learn more from Cathrine Wilhelmsen
@cathrinew hi@cathrinew.net
Pipelines and Packages: Introduction to Azure Data Factory (24HOP)

More Related Content

What's hot

What's hot (20)

Azure Data Factory Data Flows Training (Sept 2020 Update)
Azure Data Factory Data Flows Training (Sept 2020 Update)Azure Data Factory Data Flows Training (Sept 2020 Update)
Azure Data Factory Data Flows Training (Sept 2020 Update)
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Mapping Data Flows Training deck Q1 CY22
Mapping Data Flows Training deck Q1 CY22Mapping Data Flows Training deck Q1 CY22
Mapping Data Flows Training deck Q1 CY22
 
When NOT to use Apache Kafka?
When NOT to use Apache Kafka?When NOT to use Apache Kafka?
When NOT to use Apache Kafka?
 
Migrating SSIS to the cloud
Migrating SSIS to the cloudMigrating SSIS to the cloud
Migrating SSIS to the cloud
 
Introduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse ArchitectureIntroduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse Architecture
 
AWS Simple Storage Service (s3)
AWS Simple Storage Service (s3) AWS Simple Storage Service (s3)
AWS Simple Storage Service (s3)
 
Delta from a Data Engineer's Perspective
Delta from a Data Engineer's PerspectiveDelta from a Data Engineer's Perspective
Delta from a Data Engineer's Perspective
 
Messaging in the AWS Cloud
Messaging in the AWS CloudMessaging in the AWS Cloud
Messaging in the AWS Cloud
 
Migration Planning with AWS Application Discovery Service - ENT308 - Chicago ...
Migration Planning with AWS Application Discovery Service - ENT308 - Chicago ...Migration Planning with AWS Application Discovery Service - ENT308 - Chicago ...
Migration Planning with AWS Application Discovery Service - ENT308 - Chicago ...
 
AWS 101
AWS 101AWS 101
AWS 101
 
AWS network services
AWS network servicesAWS network services
AWS network services
 
Azure Data Factory v2
Azure Data Factory v2Azure Data Factory v2
Azure Data Factory v2
 
Azure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudAzure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the Cloud
 
1- Introduction of Azure data factory.pptx
1- Introduction of Azure data factory.pptx1- Introduction of Azure data factory.pptx
1- Introduction of Azure data factory.pptx
 
AWS Security Best Practices (March 2017)
AWS Security Best Practices (March 2017)AWS Security Best Practices (March 2017)
AWS Security Best Practices (March 2017)
 
Fundamentals of Stream Processing with Apache Beam, Tyler Akidau, Frances Perry
Fundamentals of Stream Processing with Apache Beam, Tyler Akidau, Frances Perry Fundamentals of Stream Processing with Apache Beam, Tyler Akidau, Frances Perry
Fundamentals of Stream Processing with Apache Beam, Tyler Akidau, Frances Perry
 
Migration Planning
Migration PlanningMigration Planning
Migration Planning
 
Serverless Kafka and Spark in a Multi-Cloud Lakehouse Architecture
Serverless Kafka and Spark in a Multi-Cloud Lakehouse ArchitectureServerless Kafka and Spark in a Multi-Cloud Lakehouse Architecture
Serverless Kafka and Spark in a Multi-Cloud Lakehouse Architecture
 
Introduction to DevOps on AWS
Introduction to DevOps on AWSIntroduction to DevOps on AWS
Introduction to DevOps on AWS
 

Similar to Pipelines and Packages: Introduction to Azure Data Factory (24HOP)

Analytics in a Day Virtual Workshop
Analytics in a Day Virtual WorkshopAnalytics in a Day Virtual Workshop
Analytics in a Day Virtual Workshop
CCG
 
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Denodo
 

Similar to Pipelines and Packages: Introduction to Azure Data Factory (24HOP) (20)

Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
 
Visualize Data and See Your Business Clearly with Sequel
Visualize Data and See Your Business Clearly with SequelVisualize Data and See Your Business Clearly with Sequel
Visualize Data and See Your Business Clearly with Sequel
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
A Connected Data Landscape: Virtualization and the Internet of Things
A Connected Data Landscape: Virtualization and the Internet of ThingsA Connected Data Landscape: Virtualization and the Internet of Things
A Connected Data Landscape: Virtualization and the Internet of Things
 
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
Creating Visual Transformations in Azure Data Factory (dataMinds Connect)
 
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
Biml for Beginners: Script and Automate SSIS development (SQLSaturday Finland)
 
Analytics in a Day Virtual Workshop
Analytics in a Day Virtual WorkshopAnalytics in a Day Virtual Workshop
Analytics in a Day Virtual Workshop
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Power BI dataflows と Power Platform Data Integration の使いどころ
Power BI dataflows と Power Platform Data Integration の使いどころPower BI dataflows と Power Platform Data Integration の使いどころ
Power BI dataflows と Power Platform Data Integration の使いどころ
 
Analytics in a Day Virtual Workshop
Analytics in a Day Virtual WorkshopAnalytics in a Day Virtual Workshop
Analytics in a Day Virtual Workshop
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Frank Gens - Clouds and Beyond: Positioning for the Next 20 Years in Enterpri...
Frank Gens - Clouds and Beyond: Positioning for the Next 20 Years in Enterpri...Frank Gens - Clouds and Beyond: Positioning for the Next 20 Years in Enterpri...
Frank Gens - Clouds and Beyond: Positioning for the Next 20 Years in Enterpri...
 
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
 
IBM Cloud pak for data brochure
IBM Cloud pak for data   brochureIBM Cloud pak for data   brochure
IBM Cloud pak for data brochure
 
Smarter Analytics: Supporting the Enterprise with Automation
Smarter Analytics: Supporting the Enterprise with AutomationSmarter Analytics: Supporting the Enterprise with Automation
Smarter Analytics: Supporting the Enterprise with Automation
 
Ingesting Click Data for Analytics
Ingesting Click Data for AnalyticsIngesting Click Data for Analytics
Ingesting Click Data for Analytics
 
Ingesting click events for analytics
Ingesting click events for analyticsIngesting click events for analytics
Ingesting click events for analytics
 

More from Cathrine Wilhelmsen

Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
 
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
 

Recently uploaded

Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Bertram Ludäscher
 
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
HyderabadDolls
 
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
HyderabadDolls
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
HyderabadDolls
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
gajnagarg
 
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
nirzagarg
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
nirzagarg
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
nirzagarg
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
Health
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
ahmedjiabur940
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
nirzagarg
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
wsppdmt
 

Recently uploaded (20)

Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
 
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
 
Kings of Saudi Arabia, information about them
Kings of Saudi Arabia, information about themKings of Saudi Arabia, information about them
Kings of Saudi Arabia, information about them
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
Jodhpur Park | Call Girls in Kolkata Phone No 8005736733 Elite Escort Service...
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
 
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
 
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi ArabiaIn Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
In Riyadh ((+919101817206)) Cytotec kit @ Abortion Pills Saudi Arabia
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for Research
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
 

Pipelines and Packages: Introduction to Azure Data Factory (24HOP)

  • 1. Cathrine Wilhelmsen, Senior BI Consultant, Inmeta Moderated By: Giuliana Grecco Pipelines and Packages: Introduction to Azure Data Factory
  • 2. If you require assistance during the session, type your inquiry into the question pane on the right side. Maximize your screen with the zoom button on the top of the presentation window. Please fill in the short evaluation following the session. It will appear in your web browser. Technical Assistance
  • 3. Thank you to our Presenting Sponsors
  • 4. PASS’ flagship event November 5-8 Seattle, Washington Free 1-day local training events Local user groups around the world Online special interest user groups Business analytics training Get involved Explore everything PASS has to offer Free Online Resources Newsletters PASS.org
  • 5. Cathrine Wilhelmsen Senior BI Consultant, Inmeta Work Things Senior BI Consultant and Microsoft Data Platform Tech Lead in Inmeta, Norway. Specialties: ADF, SSIS, Biml and T-SQL Geeky Things Fun Things Active in the PASS community as a speaker, blogger and chronic volunteer. Likes: coffee, chocolate and cat gifs :) /cathrinewilhelmsen @cathrinew cathrinew.net hi@cathrinew.net
  • 6. Cathrine Wilhelmsen, Senior BI Consultant, Inmeta Moderated By: Giuliana Grecco Pipelines and Packages: Introduction to Azure Data Factory
  • 8. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) 10 years ago…
  • 9. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) SSIS SQL Server Integration Services
  • 10. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
  • 11. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Then…
  • 12. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) ADF v1 Azure Data Factory Version 1
  • 13. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
  • 14. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
  • 15. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Today…
  • 16. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) ADF v2 Azure Data Factory Version 2
  • 17. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
  • 18. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Why? How? What?
  • 20. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Azure Data Factory
  • 21. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Data Movement and Integration Azure Data Factory Pipelines Activities Datasets Data Flows Triggers Integration RuntimesLinked Services Templates
  • 23. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Wait… I already have thousands of SSIS packages!
  • 25. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Why Lift and Shift SSIS?
  • 26. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) 1. 2. 3. 4. How to Lift and Shift SSIS
  • 27. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) SSIS Integration Runtime
  • 28. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) https://sqlbits.com/Sessions/Event18/Deeper_Integration _and_New_Transformation_for_SSIS_in_ADF SSIS Integration Runtime Deep Dive
  • 30. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) SSIS Lift and Shift: Lessons Learned like all VMs
  • 31. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) ≈ ≈ ≈ ≈ ≈ ≈ ADF vs SSIS
  • 32. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Linked Service Source Sink Activity Data Flow ≈ ≈ ≈ ≈ ≈ ≈ Package Connection Manager Source Destination Control Flow Task Data Flow ADF vs SSIS
  • 34. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Mapping Data Flows
  • 35. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Mapping Data Flows Example
  • 36. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Kamil Nowinski's Cheat Sheet github.com/SQLPlayer/CheatSheets/blob/master/ADFDF-Cheat-Sheet-sqlplayer.pdf
  • 37. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Schema Drift
  • 38. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Schema Drift in SSIS
  • 39. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Schema Drift in ADF
  • 40. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Oh no!
  • 42. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Lessons Learned
  • 43. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Good luck! Thank you :)
  • 45. Thank you for attending! @sqlpass #sqlpass @PASScommunity Learn more from Cathrine Wilhelmsen @cathrinew hi@cathrinew.net