SlideShare a Scribd company logo
1 of 53
Download to read offline
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Lessons Learned:
Understanding Pipeline Pricing
in Azure Data Factory and Azure Synapse Analytics
Cathrine Wilhelmsen
DataMinutes #2 • January 21st, 2022
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Session Abstract
Azure Data Factory and Synapse Pipelines pricing is easy, right? No upfront
costs. Pay only for what you use. It's a wonderful world for data integration
developers. Just a few clicks and your solution is ready!
But what do you present to management when they ask for cost estimates? "I
guess we just have to wait for the next invoice" is rarely an acceptable answer.
It is definitely not an acceptable answer if that invoice turns out to be
unexpectedly high.
In this session, we will look at the bits and pieces that make up the Azure Data
Factory and Synapse Pipelines pricing models, how to monitor consumption
and manage costs, and cover a few important gotchas to be aware of…
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
@cathrinew
cathrinew.net
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipelines are
awesome, right?
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
No upfront costs!
Just start developing!
Pay only for usage!
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
“But how much will
it cost per month?”
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
“I guess we just have to
wait for the invoice!"
Pipeline
Pricing
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
Orchestration and Execution
Data Flows
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
(Pretty cheap, but everything has a cost)
Orchestration and Execution
Data Flows
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations: Azure Data Factory
Read / Write: $0.50 per 50 000 entities modified
Operations: Create, Read, Update, Delete
Monitoring: $0.25 per 50 000 run records retrieved
Operations: Get, List
* Prices in USD from January 2022
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations: Azure Synapse
First 1 Million Operations: Free!
Operations: Create, Read, Update, Delete, Get, List
Additional Operations: $0.25 per 50 000 operations
Operations: Create, Read, Update, Delete, Get, List
* Prices in USD from January 2022
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
Orchestration and Execution
(Be aware of the biggest gotchas)
Data Flows
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Integration Runtimes
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Orchestration
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Orchestration
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per run:
Activity runs, debug runs, trigger executions
$0.001 $0.001 $0.0015
Data Movement per DIU-hour:
Copy Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
Auto = 4+
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Data Movement
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
Minimum = 2
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Pipeline Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Pipeline Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebook, Databricks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
200x 500x
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: External Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: External Activity
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
4000x 10000x
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
0.0167 hours
= 1 minute
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline Pricing: Rounding Up
Azure IR Azure Managed VNet IR Self-Hosted IR
Orchestration per 1000 runs:
Activity runs, debug runs, trigger executions
$1 $1 $1.50
Data Movement per DIU-hour:
Copy Data Activity
$0.25 $0.25 $0.10
Pipeline Activity per hour:
Lookup, Get Metadata, Delete etc.
$0.005 $1 $0.002
External Activity per hour:
Stored Procedure, Notebooks etc.
$0.00025 $1 $0.0001
0.0333 hours
= 2 minutes
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Operations
Orchestration and Execution
Data Flows
(Can be expensive, but worth it for the right use cases)
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Rounding Up
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour
General Purpose
Standard: $0.346 per vCore-hour
Memory Optimized
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour = $2.288 per hour
General Purpose: 4 (+4 Driver Cores)
Standard: $0.346 per vCore-hour = $2.786 per hour
Memory Optimized: 4 (+4 Driver Cores)
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Flows: Cluster Size
Basic: $0.286 per vCore-hour = $77.792 per hour
General Purpose: 256 (+16 Driver Cores)
Standard: $0.346 per vCore-hour = $94.112 per hour
Memory Optimized: 256 (+16 Driver Cores)
* Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
Lessons
Learned
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Just because you can…
doesn’t mean you should…
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
= $0.0000835 = $0.0167 = $0.000008325
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$1.59
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$159
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$3816
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Example
$0.053
$45792
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Again
Just because you can…
doesn’t mean you should…
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Everything has a price
We need to understand the pricing
model to create solutions optimized
for performance and cost
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
“I guess we just have to
wait for the invoice!"
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Here’s our estimate!
azure.microsoft.com/pricing/details/data-factory
azure.microsoft.com/pricing/details/synapse-analytics
© 2022 Cathrine Wilhelmsen (hi@cathrinew.net)
Thank you!
hi@cathrinew.net
@cathrinew
cathrinew.net

More Related Content

What's hot

Azure data factory
Azure data factoryAzure data factory
Azure data factoryDavid Giard
 
Microsoft Azure Data Factory Hands-On Lab Overview Slides
Microsoft Azure Data Factory Hands-On Lab Overview SlidesMicrosoft Azure Data Factory Hands-On Lab Overview Slides
Microsoft Azure Data Factory Hands-On Lab Overview SlidesMark Kromer
 
Snowflake essentials
Snowflake essentialsSnowflake essentials
Snowflake essentialsqureshihamid
 
Building Lakehouses on Delta Lake with SQL Analytics Primer
Building Lakehouses on Delta Lake with SQL Analytics PrimerBuilding Lakehouses on Delta Lake with SQL Analytics Primer
Building Lakehouses on Delta Lake with SQL Analytics PrimerDatabricks
 
Microsoft Data Integration Pipelines: Azure Data Factory and SSIS
Microsoft Data Integration Pipelines: Azure Data Factory and SSISMicrosoft Data Integration Pipelines: Azure Data Factory and SSIS
Microsoft Data Integration Pipelines: Azure Data Factory and SSISMark Kromer
 
Azure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudAzure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudMark Kromer
 
Introduction to Azure Data Factory
Introduction to Azure Data FactoryIntroduction to Azure Data Factory
Introduction to Azure Data FactorySlava Kokaev
 
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)Cathrine Wilhelmsen
 
Azure Data Factory Data Flow
Azure Data Factory Data FlowAzure Data Factory Data Flow
Azure Data Factory Data FlowMark Kromer
 
Snowflake for Data Engineering
Snowflake for Data EngineeringSnowflake for Data Engineering
Snowflake for Data EngineeringHarald Erb
 
Intro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeIntro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeKent Graziano
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
 
Introducing Azure SQL Data Warehouse
Introducing Azure SQL Data WarehouseIntroducing Azure SQL Data Warehouse
Introducing Azure SQL Data WarehouseJames Serra
 
Data cleansing and prep with synapse data flows
Data cleansing and prep with synapse data flowsData cleansing and prep with synapse data flows
Data cleansing and prep with synapse data flowsMark Kromer
 
ADF Mapping Data Flows Training Slides V1
ADF Mapping Data Flows Training Slides V1ADF Mapping Data Flows Training Slides V1
ADF Mapping Data Flows Training Slides V1Mark Kromer
 
Modern Data Warehouse with Azure Synapse.pdf
Modern Data Warehouse with Azure Synapse.pdfModern Data Warehouse with Azure Synapse.pdf
Modern Data Warehouse with Azure Synapse.pdfKeyla Dolores Méndez
 
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Edureka!
 
Introducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data WarehouseIntroducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data WarehouseSnowflake Computing
 

What's hot (20)

Azure Synapse Analytics
Azure Synapse AnalyticsAzure Synapse Analytics
Azure Synapse Analytics
 
Azure data factory
Azure data factoryAzure data factory
Azure data factory
 
Microsoft Azure Data Factory Hands-On Lab Overview Slides
Microsoft Azure Data Factory Hands-On Lab Overview SlidesMicrosoft Azure Data Factory Hands-On Lab Overview Slides
Microsoft Azure Data Factory Hands-On Lab Overview Slides
 
Snowflake essentials
Snowflake essentialsSnowflake essentials
Snowflake essentials
 
Modern data warehouse
Modern data warehouseModern data warehouse
Modern data warehouse
 
Building Lakehouses on Delta Lake with SQL Analytics Primer
Building Lakehouses on Delta Lake with SQL Analytics PrimerBuilding Lakehouses on Delta Lake with SQL Analytics Primer
Building Lakehouses on Delta Lake with SQL Analytics Primer
 
Microsoft Data Integration Pipelines: Azure Data Factory and SSIS
Microsoft Data Integration Pipelines: Azure Data Factory and SSISMicrosoft Data Integration Pipelines: Azure Data Factory and SSIS
Microsoft Data Integration Pipelines: Azure Data Factory and SSIS
 
Azure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudAzure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the Cloud
 
Introduction to Azure Data Factory
Introduction to Azure Data FactoryIntroduction to Azure Data Factory
Introduction to Azure Data Factory
 
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
Pipelines and Packages: Introduction to Azure Data Factory (DATA:Scotland 2019)
 
Azure Data Factory Data Flow
Azure Data Factory Data FlowAzure Data Factory Data Flow
Azure Data Factory Data Flow
 
Snowflake for Data Engineering
Snowflake for Data EngineeringSnowflake for Data Engineering
Snowflake for Data Engineering
 
Intro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeIntro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on Snowflake
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Introducing Azure SQL Data Warehouse
Introducing Azure SQL Data WarehouseIntroducing Azure SQL Data Warehouse
Introducing Azure SQL Data Warehouse
 
Data cleansing and prep with synapse data flows
Data cleansing and prep with synapse data flowsData cleansing and prep with synapse data flows
Data cleansing and prep with synapse data flows
 
ADF Mapping Data Flows Training Slides V1
ADF Mapping Data Flows Training Slides V1ADF Mapping Data Flows Training Slides V1
ADF Mapping Data Flows Training Slides V1
 
Modern Data Warehouse with Azure Synapse.pdf
Modern Data Warehouse with Azure Synapse.pdfModern Data Warehouse with Azure Synapse.pdf
Modern Data Warehouse with Azure Synapse.pdf
 
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
 
Introducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data WarehouseIntroducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data Warehouse
 

Similar to Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics (DataMinutes #2)

Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...Cathrine Wilhelmsen
 
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...Data Con LA
 
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"Cisco Canada
 
What Does It Cost for Microsoft Dynamics in the Cloud?
What Does It Cost for Microsoft Dynamics in the Cloud?What Does It Cost for Microsoft Dynamics in the Cloud?
What Does It Cost for Microsoft Dynamics in the Cloud?Concerto Cloud Services
 
Deploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data SolutionsDeploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data SolutionsMarco Parenzan
 
Visualizing S3 Web Page Log Data
Visualizing S3 Web Page Log DataVisualizing S3 Web Page Log Data
Visualizing S3 Web Page Log DataStacey Graham
 
Future Grid Overview 2018
Future Grid Overview 2018Future Grid Overview 2018
Future Grid Overview 2018Chris J Law
 
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)svub
 
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...Amazon Web Services
 
Gitex2010 ICT strategies moving to the cloud v11
Gitex2010 ICT strategies moving to the cloud v11Gitex2010 ICT strategies moving to the cloud v11
Gitex2010 ICT strategies moving to the cloud v11Jorge Sebastiao
 
The Last Frontier- Virtualization, Hybrid Management and the Cloud
The Last Frontier-  Virtualization, Hybrid Management and the CloudThe Last Frontier-  Virtualization, Hybrid Management and the Cloud
The Last Frontier- Virtualization, Hybrid Management and the CloudKellyn Pot'Vin-Gorman
 
Radical Cloud Consolidation on The Ball
Radical Cloud Consolidation on The BallRadical Cloud Consolidation on The Ball
Radical Cloud Consolidation on The BallKallex
 
Serverless - Lunch&Learn CleverToday - Mars 2017
Serverless - Lunch&Learn CleverToday - Mars 2017Serverless - Lunch&Learn CleverToday - Mars 2017
Serverless - Lunch&Learn CleverToday - Mars 2017Brice Argenson
 
Evolution of Microservices - Craft Conference
Evolution of Microservices - Craft ConferenceEvolution of Microservices - Craft Conference
Evolution of Microservices - Craft ConferenceAdrian Cockcroft
 
Iot meets Serverless
Iot meets ServerlessIot meets Serverless
Iot meets ServerlessNarendran R
 
Stor simple presentation customers
 Stor simple presentation customers Stor simple presentation customers
Stor simple presentation customersJarek Sokolnicki
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Cathrine Wilhelmsen
 

Similar to Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics (DataMinutes #2) (20)

Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2...
 
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
Data Con LA 2022 - Supercharge your Snowflake Data Cloud from a Snowflake Dat...
 
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
Enterprise Data Center and Cloud: "Efficiency, Speed, Disruption"
 
Nonprofit IT Budgeting
Nonprofit IT BudgetingNonprofit IT Budgeting
Nonprofit IT Budgeting
 
What Does It Cost for Microsoft Dynamics in the Cloud?
What Does It Cost for Microsoft Dynamics in the Cloud?What Does It Cost for Microsoft Dynamics in the Cloud?
What Does It Cost for Microsoft Dynamics in the Cloud?
 
Deploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data SolutionsDeploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data Solutions
 
Visualizing S3 Web Page Log Data
Visualizing S3 Web Page Log DataVisualizing S3 Web Page Log Data
Visualizing S3 Web Page Log Data
 
Future Grid Overview 2018
Future Grid Overview 2018Future Grid Overview 2018
Future Grid Overview 2018
 
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
Collections vs. Meteods - Meteor Meetup Athens (#1 2014-11-25)
 
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
AWS December 2015 Webinar Series - Strategies to Quantify TCO & Optimize Cost...
 
Gitex2010 ICT strategies moving to the cloud v11
Gitex2010 ICT strategies moving to the cloud v11Gitex2010 ICT strategies moving to the cloud v11
Gitex2010 ICT strategies moving to the cloud v11
 
The Last Frontier- Virtualization, Hybrid Management and the Cloud
The Last Frontier-  Virtualization, Hybrid Management and the CloudThe Last Frontier-  Virtualization, Hybrid Management and the Cloud
The Last Frontier- Virtualization, Hybrid Management and the Cloud
 
Radical Cloud Consolidation on The Ball
Radical Cloud Consolidation on The BallRadical Cloud Consolidation on The Ball
Radical Cloud Consolidation on The Ball
 
Serverless - Lunch&Learn CleverToday - Mars 2017
Serverless - Lunch&Learn CleverToday - Mars 2017Serverless - Lunch&Learn CleverToday - Mars 2017
Serverless - Lunch&Learn CleverToday - Mars 2017
 
Evolution of Microservices - Craft Conference
Evolution of Microservices - Craft ConferenceEvolution of Microservices - Craft Conference
Evolution of Microservices - Craft Conference
 
Iot meets Serverless
Iot meets ServerlessIot meets Serverless
Iot meets Serverless
 
Stor simple presentation customers
 Stor simple presentation customers Stor simple presentation customers
Stor simple presentation customers
 
Intuit Analytics Cloud 101
Intuit Analytics Cloud 101Intuit Analytics Cloud 101
Intuit Analytics Cloud 101
 
Jumpstart Azure
Jumpstart AzureJumpstart Azure
Jumpstart Azure
 
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
Visually Transform Data in Azure Data Factory or Azure Synapse Analytics (PAS...
 

More from Cathrine Wilhelmsen

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Cathrine Wilhelmsen
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Cathrine Wilhelmsen
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Cathrine Wilhelmsen
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Cathrine Wilhelmsen
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Cathrine Wilhelmsen
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)Cathrine Wilhelmsen
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Cathrine Wilhelmsen
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Cathrine Wilhelmsen
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Cathrine Wilhelmsen
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Cathrine Wilhelmsen
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...Cathrine Wilhelmsen
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Cathrine Wilhelmsen
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)Cathrine Wilhelmsen
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Cathrine Wilhelmsen
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Cathrine Wilhelmsen
 
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Cathrine Wilhelmsen
 

More from Cathrine Wilhelmsen (20)

Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
Website Analytics in My Pocket using Microsoft Fabric (SQLBits 2024)
 
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
Data Integration using Data Factory in Microsoft Fabric (ESPC Microsoft Fabri...
 
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
Choosing between Fabric, Synapse and Databricks (Data Left Unattended 2023)
 
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
Data Integration with Data Factory (Microsoft Fabric Day Oslo 2023)
 
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
The Battle of the Data Transformation Tools (PASS Data Community Summit 2023)
 
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
Building an End-to-End Solution in Microsoft Fabric: From Dataverse to Power ...
 
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
Website Analytics in my Pocket using Microsoft Fabric (AdaCon 2023)
 
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
Choosing Between Microsoft Fabric, Azure Synapse Analytics and Azure Data Fac...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (D...
 
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
Stressed, Depressed, or Burned Out? The Warning Signs You Shouldn't Ignore (S...
 
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ..."I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
"I can't keep up!" - Turning Discomfort into Personal Growth in a Fast-Paced ...
 
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
Lessons Learned: Implementing Azure Synapse Analytics in a Rapidly-Changing S...
 
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
6 Tips for Building Confidence as a Public Speaker (SQLBits 2022)
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
Understanding Azure Data Factory: The What, When, and Why (NIC 2020)
 
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)Azure Data Factory for the SSIS Developer (SentryOne Webinar)
Azure Data Factory for the SSIS Developer (SentryOne Webinar)
 
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
Azure Synapse Analytics Teaser (Microsoft TechX Oslo 2019)
 

Recently uploaded

The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxTasha Penwell
 
Principles and Practices of Data Visualization
Principles and Practices of Data VisualizationPrinciples and Practices of Data Visualization
Principles and Practices of Data VisualizationKianJazayeri1
 
Rithik Kumar Singh codealpha pythohn.pdf
Rithik Kumar Singh codealpha pythohn.pdfRithik Kumar Singh codealpha pythohn.pdf
Rithik Kumar Singh codealpha pythohn.pdfrahulyadav957181
 
Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxHimangsuNath
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfblazblazml
 
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfWorld Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfsimulationsindia
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...Dr Arash Najmaei ( Phd., MBA, BSc)
 
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Boston Institute of Analytics
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Thomas Poetter
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024Susanna-Assunta Sansone
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaManalVerma4
 
Cyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataCyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataTecnoIncentive
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxHaritikaChhatwal1
 
What To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxWhat To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxSimranPal17
 
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...Amil Baba Dawood bangali
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI  Streaming - AI Max PrincetonReal-Time AI  Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max PrincetonTimothy Spann
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Boston Institute of Analytics
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBoston Institute of Analytics
 

Recently uploaded (20)

The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
 
Principles and Practices of Data Visualization
Principles and Practices of Data VisualizationPrinciples and Practices of Data Visualization
Principles and Practices of Data Visualization
 
Rithik Kumar Singh codealpha pythohn.pdf
Rithik Kumar Singh codealpha pythohn.pdfRithik Kumar Singh codealpha pythohn.pdf
Rithik Kumar Singh codealpha pythohn.pdf
 
Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptx
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
 
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfWorld Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
 
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in India
 
Cyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataCyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded data
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptx
 
What To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxWhat To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptx
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI  Streaming - AI Max PrincetonReal-Time AI  Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max Princeton
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
 

Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics (DataMinutes #2)

  • 1. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azure Synapse Analytics Cathrine Wilhelmsen DataMinutes #2 • January 21st, 2022
  • 2. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Session Abstract Azure Data Factory and Synapse Pipelines pricing is easy, right? No upfront costs. Pay only for what you use. It's a wonderful world for data integration developers. Just a few clicks and your solution is ready! But what do you present to management when they ask for cost estimates? "I guess we just have to wait for the next invoice" is rarely an acceptable answer. It is definitely not an acceptable answer if that invoice turns out to be unexpectedly high. In this session, we will look at the bits and pieces that make up the Azure Data Factory and Synapse Pipelines pricing models, how to monitor consumption and manage costs, and cover a few important gotchas to be aware of…
  • 3. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) @cathrinew cathrinew.net
  • 4. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipelines are awesome, right?
  • 5. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) No upfront costs! Just start developing! Pay only for usage!
  • 6. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) “But how much will it cost per month?”
  • 7. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) “I guess we just have to wait for the invoice!"
  • 9. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations Orchestration and Execution Data Flows
  • 10. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations (Pretty cheap, but everything has a cost) Orchestration and Execution Data Flows
  • 11. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations: Azure Data Factory Read / Write: $0.50 per 50 000 entities modified Operations: Create, Read, Update, Delete Monitoring: $0.25 per 50 000 run records retrieved Operations: Get, List * Prices in USD from January 2022
  • 12. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations: Azure Synapse First 1 Million Operations: Free! Operations: Create, Read, Update, Delete, Get, List Additional Operations: $0.25 per 50 000 operations Operations: Create, Read, Update, Delete, Get, List * Prices in USD from January 2022
  • 13. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations Orchestration and Execution (Be aware of the biggest gotchas) Data Flows
  • 14. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Integration Runtimes Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 15. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Orchestration Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 16. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Orchestration Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per run: Activity runs, debug runs, trigger executions $0.001 $0.001 $0.0015 Data Movement per DIU-hour: Copy Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 17. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 18. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 19. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 20. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. Auto = 4+
  • 21. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Data Movement Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 Minimum = 2 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 22. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Pipeline Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 23. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Pipeline Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebook, Databricks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. 200x 500x
  • 24. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: External Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 25. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: External Activity Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. 4000x 10000x
  • 26. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 27. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 28. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 29. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 30. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 0.0167 hours = 1 minute * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 31. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Pipeline Pricing: Rounding Up Azure IR Azure Managed VNet IR Self-Hosted IR Orchestration per 1000 runs: Activity runs, debug runs, trigger executions $1 $1 $1.50 Data Movement per DIU-hour: Copy Data Activity $0.25 $0.25 $0.10 Pipeline Activity per hour: Lookup, Get Metadata, Delete etc. $0.005 $1 $0.002 External Activity per hour: Stored Procedure, Notebooks etc. $0.00025 $1 $0.0001 0.0333 hours = 2 minutes * Prices in USD from January 2022. All activities are prorated by the minute and rounded up.
  • 32. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Operations Orchestration and Execution Data Flows (Can be expensive, but worth it for the right use cases)
  • 33. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 34. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Rounding Up Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 35. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 36. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 37. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour General Purpose Standard: $0.346 per vCore-hour Memory Optimized * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 38. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour = $2.288 per hour General Purpose: 4 (+4 Driver Cores) Standard: $0.346 per vCore-hour = $2.786 per hour Memory Optimized: 4 (+4 Driver Cores) * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 39. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Data Flows: Cluster Size Basic: $0.286 per vCore-hour = $77.792 per hour General Purpose: 256 (+16 Driver Cores) Standard: $0.346 per vCore-hour = $94.112 per hour Memory Optimized: 256 (+16 Driver Cores) * Prices in USD from January 2022. All activities are prorated by the minute and rounded up. The minimum cluster size is 8 vCores.
  • 41. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Just because you can… doesn’t mean you should…
  • 42. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example
  • 43. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example = $0.0000835 = $0.0167 = $0.000008325
  • 44. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053
  • 45. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $1.59
  • 46. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $159
  • 47. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $3816
  • 48. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Example $0.053 $45792
  • 49. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Again Just because you can… doesn’t mean you should…
  • 50. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Everything has a price We need to understand the pricing model to create solutions optimized for performance and cost
  • 51. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) “I guess we just have to wait for the invoice!"
  • 52. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Here’s our estimate! azure.microsoft.com/pricing/details/data-factory azure.microsoft.com/pricing/details/synapse-analytics
  • 53. © 2022 Cathrine Wilhelmsen (hi@cathrinew.net) Thank you! hi@cathrinew.net @cathrinew cathrinew.net