Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2019)

157 views

Published on

Lessons Learned: Understanding Azure Data Factory Pricing (Presented at Microsoft Ignite on November 6th, 2019)

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Lessons Learned: Understanding Azure Data Factory Pricing (Microsoft Ignite 2019)

  1. 1. @cathrinew | cathrinew.net
  2. 2. Session Abstract
  3. 3. @cathrinew cathrinew.net
  4. 4. Azure Data Factory pricing is easy!
  5. 5. No upfront costs! Pay only for usage!
  6. 6. …but how much will that actually cost?
  7. 7. "I guess we just have to wait for the invoice?"
  8. 8. Azure Data Factory Pricing
  9. 9. Azure Data Factory Operations Data Pipeline Orchestration and Execution Data Flow Debugging and Execution SQL Server Integration Services
  10. 10. Azure Data Factory Operations Read / Write: $0.50 per 50 000 modified entities Operations: Create, Read, Update, Delete Entities: Datasets, Linked Services, Pipelines, Integration Runtimes, Triggers Monitoring: $0.25 per 50 000 run records retrieved Operations: Get, List Run Records: Pipeline Runs, Activity Runs, Trigger Runs, Debug Runs
  11. 11. Azure Data Factory Operations Data Pipeline Orchestration and Execution Data Flow Debugging and Execution SQL Server Integration Services
  12. 12. Data Pipelines: Self-Hosted $1.50 per 1000 runs $0.10 per DIU-hour $0.002 per hour $0.0001 per hour
  13. 13. Data Pipelines: Azure $1 per 1000 runs $0.25 per DIU-hour $0.005 per hour $0.00025 per hour
  14. 14. Data Pipelines: Azure Orchestration: $1 per 1000 runs Activity, trigger and debug runs Execution: Data Movement Activities: $0.25 per DIU-hour Pipeline Activities: $0.005 per hour External Activities: $0.00025 per hour All activities are prorated by the minute and rounded up
  15. 15. Auto = 4
  16. 16. Min = 2
  17. 17. Data Pipelines: Azure Orchestration: $1 per 1000 runs Activity, trigger and debug runs Execution: Data Movement Activities: $0.25 per DIU-hour Pipeline Activities: $0.005 per hour External Activities: $0.00025 per hour All activities are prorated by the minute and rounded up
  18. 18. Actual time: 1 sec = Billed for: 1 min
  19. 19. Actual time: 1 min 1 sec = Billed for: 2 min
  20. 20. Actual time: 1 min 30 sec = Billed for: 2 min
  21. 21. Actual time: 9 min 1 sec = Billed for: 10 min*DIUs
  22. 22. Data Pipelines: Additional Costs $0.80 per month $0.05 - $0.087 per GB
  23. 23. Azure Data Factory Operations Data Pipeline Orchestration and Execution Data Flow Debugging and Execution SQL Server Integration Services
  24. 24. Data Flow Debugging and Execution $0.199 per vCore-hour $0.268 per vCore-hour $0.345 per vCore-hour
  25. 25. Data Flow Debugging and Execution Compute Optimized: $0.199 per vCore-hour General Purpose: $0.268 per vCore-hour Memory Optimized: $0.345 per vCore-hour Charges are prorated by the minute and rounded up The minimum cluster size to run a data flow is 8 vCores
  26. 26. Data Flow Debugging and Execution Compute Optimized: $0.199 per vCore-hour General Purpose: $0.268 per vCore-hour Memory Optimized: $0.345 per vCore-hour Charges are prorated by the minute and rounded up The minimum cluster size to run a data flow is 8 vCores
  27. 27. Data Flow Debugging and Execution
  28. 28. Data Flow Debugging and Execution $1.592 per hour $2.144 per hour $2.760 per hour
  29. 29. Data Flow Debugging and Execution $54.128 per hour $72.896 per hour $93.84 per hour
  30. 30. Azure Data Factory Operations Data Pipeline Orchestration and Execution Data Flow Debugging and Execution SQL Server Integration Services
  31. 31. SQL Server Integration Services $0.592 per node per hour $18.212 per node per hour
  32. 32. SQL Server Integration Services $1.665 per node per hour $35.372 per node per hour
  33. 33. SQL Server Integration Services Enterprise D1 V2: $1.665 per node per hour 1 core, 3.5 GB RAM, 50 GB Temporary Storage Enterprise E64 V3: $35.372 per node per hour 64 cores, 432 GB RAM, 1600 GB Temporary Storage
  34. 34. SQL Server Integration Services
  35. 35. SQL Server Integration Services
  36. 36. Lessons Learned
  37. 37. Lessons Learned
  38. 38. Lessons Learned
  39. 39. Total cost per pipeline run: $0.17
  40. 40. Total cost per pipeline run: $0.17 Run once per night for 30 days: $5.1
  41. 41. Total cost per pipeline run: $0.17 $510
  42. 42. Total cost per pipeline run: $0.17 $12240
  43. 43. Lessons Learned
  44. 44. Lessons Learned
  45. 45. Pricing Calculator azure.microsoft.com/pricing/ calculator/?service=data-factory
  46. 46. "I guess we just have to wait for the invoice?"
  47. 47. "No, I got this!"
  48. 48. Good luck!
  49. 49. @cathrinew cathrinew.net hi@cathrinew.net thank you!

×