SlideShare a Scribd company logo
1 of 53
Download to read offline
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Lessons Learned:
Understanding Pipeline Pricing
in Azure Data Factory and Azure Synapse Analytics
Cathrine Wilhelmsen
DataMinutes #2 • January 21st, 2022
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Session Abstract
Azure Data Factory and Synapse Pipelines pricing is easy, right? No upfront
costs. Pay only for what you use. It's a wonderful world for data integration
developers. Just a few clicks and your solution is ready!
But what do you present to management when they ask for cost estimates? "I
guess we just have to wait for the next invoice" is rarely an acceptable answer.
It is definitely not an acceptable answer if that invoice turns out to be
unexpectedly high.
In this session, we will look at the bits and pieces that make up the Azure Data
Factory and Synapse Pipelines pricing models, how to monitor consumption
and manage costs, and cover a few important gotchas to be aware of…
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
@cathrinew
cathrinew.net
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipelines are
awesome, right?
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
No upfront costs!
Just start developing!
Pay only for usage!
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
“But how much will
it cost per month?”
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
“I guess we just have to
wait for the invoice!"
Pipeline
Pricing
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
Orchestration and Execution
Data Flows
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
(Pretty cheap, but everything has a cost)
Orchestration and Execution
Data Flows
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations: Azure Data Factory
Read / Write: $0.50 per 50 000 entities modified
Operations: Create, Read, Update, Delete
Monitoring: $0.25 per 50 000 run records retrieved
Operations: Get, List
* Prices in USD from January 2022
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations: Azure Synapse
First 1 Million Operations: Free!
Operations: Create, Read, Update, Delete, Get, List
Additional Operations: $0.25 per 50 000 operations
Operations: Create, Read, Update, Delete, Get, List
* Prices in USD from January 2022
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
Orchestration and Execution
(Be aware of the biggest gotchas)
Data Flows
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Integration Runtimes
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Orchestration
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Orchestration
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per run:
Activity runs, debug runs, trigger executions
$0.001 $0.001 $0.0015
Data Movement per DIU-hour:
Copy Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
Auto = 4+
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
Minimum = 2
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Pipeline Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Pipeline Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
200x 500x
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: External Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: External Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
4000x 10000x
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
0.0167 hours
= 1 minute
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
0.0333 hours
= 2 minutes
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
Orchestration and Execution
Data Flows
(Can be expensive, but worth it for the right use cases)
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Rounding Up
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour = $2.288 per hour
General Purpose: 4 (+4 Driver Cores)
Standard: $0.346 per vCore-hour = $2.786 per hour
Memory Optimized: 4 (+4 Driver Cores)
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour = $77.792 per hour
General Purpose: 256 (+16 Driver Cores)
Standard: $0.346 per vCore-hour = $94.112 per hour
Memory Optimized: 256 (+16 Driver Cores)
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
Lessons
Learned
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Just because you can…
doesn’t mean you should…
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
= $0.0000835 = $0.0167 = $0.000008325
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$1.59
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$159
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$3816
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$45792
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Again
Just because you can…
doesn’t mean you should…
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Everything has a price
We need to understand the pricing
model to create solutions optimized
for performance and cost
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
“I guess we just have to
wait for the invoice!"
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Here’s our estimate!
azure.microsoft.com/pricing/details/data-factory
azure.microsoft.com/pricing/details/synapse-analytics
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Thank you!
hi@cathrinew.net
@cathrinew
cathrinew.net

More Related Content

What's hot

What's hot (20)

Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
 
Azure data platform overview
Azure data platform overviewAzure data platform overview
Azure data platform overview
 
Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?
 
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data ArchitectureModernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
 
Azure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoAzure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene Polonichko
 
Databricks Fundamentals
Databricks FundamentalsDatabricks Fundamentals
Databricks Fundamentals
 
Azure Databricks - An Introduction (by Kris Bock)
Azure Databricks - An Introduction (by Kris Bock)Azure Databricks - An Introduction (by Kris Bock)
Azure Databricks - An Introduction (by Kris Bock)
 
Data Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and GovernanceData Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and Governance
 
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
 
Azure Data Factory v2
Azure Data Factory v2Azure Data Factory v2
Azure Data Factory v2
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Azure data factory
Azure data factoryAzure data factory
Azure data factory
 
Data modelling 101
Data modelling 101Data modelling 101
Data modelling 101
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
Azure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekAzure Data Factory for Azure Data Week
Azure Data Factory for Azure Data Week
 
Introduction to Data Engineering
Introduction to Data EngineeringIntroduction to Data Engineering
Introduction to Data Engineering
 
Modern Data architecture Design
Modern Data architecture DesignModern Data architecture Design
Modern Data architecture Design
 
Azure Database Services for MySQL PostgreSQL and MariaDB
Azure Database Services for MySQL PostgreSQL and MariaDBAzure Database Services for MySQL PostgreSQL and MariaDB
Azure Database Services for MySQL PostgreSQL and MariaDB
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
 
Snowflake Overview
Snowflake OverviewSnowflake Overview
Snowflake Overview
 

Similar to Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics (DataMinutes #2)

Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA
 
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
Amazon Web Services
 

Similar to Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics (DataMinutes #2) (20)

Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
 
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
 
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
 
Nonprofit IT Budgeting
Nonprofit IT BudgetingNonprofit IT Budgeting
Nonprofit IT Budgeting
 
What Does It Cost for Microsoft Dynamics in the Cloud?
What Does It Cost for Microsoft Dynamics in the Cloud?What Does It Cost for Microsoft Dynamics in the Cloud?
What Does It Cost for Microsoft Dynamics in the Cloud?
 
Deploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data SolutionsDeploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data Solutions
 
Visualizing S3 Web Page Log Data
Visualizing S3 Web Page Log DataVisualizing S3 Web Page Log Data
Visualizing S3 Web Page Log Data
 
Future Grid Overview 2018
Future Grid Overview 2018Future Grid Overview 2018
Future Grid Overview 2018
 
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
 
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
 
Gitex2010 ICT strategies moving to the cloud v11
Gitex2010 ICT strategies moving to the cloud v11Gitex2010 ICT strategies moving to the cloud v11
Gitex2010 ICT strategies moving to the cloud v11
 
The Last Frontier- Virtualization, Hybrid Management and the Cloud
The Last Frontier-  Virtualization, Hybrid Management and the CloudThe Last Frontier-  Virtualization, Hybrid Management and the Cloud
The Last Frontier- Virtualization, Hybrid Management and the Cloud
 
Radical Cloud Consolidation on The Ball
Radical Cloud Consolidation on The BallRadical Cloud Consolidation on The Ball
Radical Cloud Consolidation on The Ball
 
Serverless - Lunch&Learn CleverToday - Mars 2017
Serverless - Lunch&Learn CleverToday - Mars 2017Serverless - Lunch&Learn CleverToday - Mars 2017
Serverless - Lunch&Learn CleverToday - Mars 2017
 
Evolution of Microservices - Craft Conference
Evolution of Microservices - Craft ConferenceEvolution of Microservices - Craft Conference
Evolution of Microservices - Craft Conference
 
Iot meets Serverless
Iot meets ServerlessIot meets Serverless
Iot meets Serverless
 
Stor simple presentation customers
 Stor simple presentation customers Stor simple presentation customers
Stor simple presentation customers
 
Intuit Analytics Cloud 101
Intuit Analytics Cloud 101Intuit Analytics Cloud 101
Intuit Analytics Cloud 101
 
Jumpstart Azure
Jumpstart AzureJumpstart Azure
Jumpstart Azure
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 

More from Cathrine Wilhelmsen

Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
 
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
 
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
Building Dynamic Data Pipelines in Azure Data Factory (Microsoft Ignite 2019)
 

Recently uploaded

obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
yulianti213969
 
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
a8om7o51
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
pwgnohujw
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
dq9vz1isj
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Stephen266013
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
yulianti213969
 
原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证
pwgnohujw
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Valters Lauzums
 
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
zifhagzkk
 

Recently uploaded (20)

Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeCredit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
 
Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"Aggregations - The Elasticsearch "GROUP BY"
Aggregations - The Elasticsearch "GROUP BY"
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchers
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
 
What is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationWhat is Insertion Sort. Its basic information
What is Insertion Sort. Its basic information
 
社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
 
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
Data Analysis Project Presentation : NYC Shooting Cluster Analysis
Data Analysis Project Presentation : NYC Shooting Cluster AnalysisData Analysis Project Presentation : NYC Shooting Cluster Analysis
Data Analysis Project Presentation : NYC Shooting Cluster Analysis
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证
 
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
Statistics Informed Decisions Using Data 5th edition by Michael Sullivan solu...
 
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
Data Analytics for Digital Marketing Lecture for Advanced Digital & Social Me...
 
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
 

Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics (DataMinutes #2)

  • 1. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics Cathrine Wilhelmsen DataMinutes #2 • January 21st, 2022
  • 2. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Session Abstract Azure Data Factory and Synapse Pipelines pricing is easy, right? No upfront costs. Pay only for what you use. It's a wonderful world for data integration developers. Just a few clicks and your solution is ready! But what do you present to management when they ask for cost estimates? "I guess we just have to wait for the next invoice" is rarely an acceptable answer. It is definitely not an acceptable answer if that invoice turns out to be unexpectedly high. In this session, we will look at the bits and pieces that make up the Azure Data Factory and Synapse Pipelines pricing models, how to monitor consumption and manage costs, and cover a few important gotchas to be aware of…
  • 3. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) @cathrinew cathrinew.net
  • 4. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipelines are awesome, right?
  • 5. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) No upfront costs! Just start developing! Pay only for usage!
  • 6. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) “But how much will it cost per month?”
  • 7. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) “I guess we just have to wait for the invoice!"
  • 9. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations Orchestration and Execution Data Flows
  • 10. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations (Pretty cheap, but everything has a cost) Orchestration and Execution Data Flows
  • 11. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations: Azure Data Factory Read / Write: $0.50 per 50 000 entities modified Operations: Create, Read, Update, Delete Monitoring: $0.25 per 50 000 run records retrieved Operations: Get, List * Prices in USD from January 2022
  • 12. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations: Azure Synapse First 1 Million Operations: Free! Operations: Create, Read, Update, Delete, Get, List Additional Operations: $0.25 per 50 000 operations Operations: Create, Read, Update, Delete, Get, List * Prices in USD from January 2022
  • 13. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations Orchestration and Execution (Be aware of the biggest gotchas) Data Flows
  • 14. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Integration Runtimes Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 15. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Orchestration Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 16. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Orchestration Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per run: Activity runs, debug runs, trigger executions $0.001 $0.001 $0.0015 Data Movement per DIU-hour: Copy Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 17. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 18. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 19. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 20. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. Auto = 4+
  • 21. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 Minimum = 2 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 22. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Pipeline Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 23. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Pipeline Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. 200x 500x
  • 24. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: External Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 25. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: External Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. 4000x 10000x
  • 26. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 27. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 28. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 29. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 30. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 0.0167 hours = 1 minute * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 31. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 0.0333 hours = 2 minutes * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 32. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations Orchestration and Execution Data Flows (Can be expensive, but worth it for the right use cases)
  • 33. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 34. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Rounding Up Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 35. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 36. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 37. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 38. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour = $2.288 per hour General Purpose: 4 (+4 Driver Cores) Standard: $0.346 per vCore-hour = $2.786 per hour Memory Optimized: 4 (+4 Driver Cores) * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 39. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour = $77.792 per hour General Purpose: 256 (+16 Driver Cores) Standard: $0.346 per vCore-hour = $94.112 per hour Memory Optimized: 256 (+16 Driver Cores) * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 41. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Just because you can… doesn’t mean you should…
  • 42. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example
  • 43. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example = $0.0000835 = $0.0167 = $0.000008325
  • 44. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053
  • 45. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $1.59
  • 46. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $159
  • 47. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $3816
  • 48. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $45792
  • 49. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Again Just because you can… doesn’t mean you should…
  • 50. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Everything has a price We need to understand the pricing model to create solutions optimized for performance and cost
  • 51. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) “I guess we just have to wait for the invoice!"
  • 52. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Here’s our estimate! azure.microsoft.com/pricing/details/data-factory azure.microsoft.com/pricing/details/synapse-analytics
  • 53. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Thank you! hi@cathrinew.net @cathrinew cathrinew.net