SlideShare a Scribd company logo
Deep Dive into
Durable Functions
CloudBrew
Joonas Westlin
Speaker Intro
• Joonas Westlin
• Developer / Architect @ Zure
@JoonasWestlin
joonasw.net
Key takeaways from this presentation
• Orchestrator and activity communication
• State storage using the Azure Storage provider
• Differences in other durability providers
• Goal: improve your ability to debug Durable Functions
Introduction/refresher
• Durable Functions = extension for Azure Functions that allows
execution of Durable Task orchestrations
• Durable Task framework allows developers to build long-running
workflows with persistent state, using well known async-await
constructs in .NET
• Orchestrator functions define the workflow, activity functions do
non-deterministic work
Durable Task workers
• Workers execute orchestrators and activities by requesting work
items from the durability provider
• Multiple workers run in parallel = increased throughput
• Activity work items can run on any worker, but orchestrator work
items cannot
• Danger of state corruption if more than 1 worker takes a work item for
same orchestration simultaneously
Durability providers
Storage
Service
Bus
Netherite SQL
Redis
Service
Fabric
Emulator
Storage durability provider
• Convenient choice for Azure Functions as it has a Storage account
already
• Cheap option, consumption-based pricing in Storage
• Tip: using a v1 Storage account is cheaper (not in new regions)
• Can run emulator locally for Storage
Storage task hub
Orchestrator and activity communication
• Orchestrator gets executed several times; each time a set of
outbound messages is generated for activities to be triggered
• These are sent through the durability provider and workers
receive them
• Once the worker is done with the activity, it sends a message back
to the orchestrator
Control queue 0
Control queue 1
Control queue 2
Control queue 3
Work item queue
Demo: Durable Function
execution
Durable Entities
• We looked at orchestrators and activities but where are Durable
Entities?
• Entities are just orchestrators
• At high level, you can think of entities as an orchestrator that:
1. Gets state from input
2. Waits for an external event (operation)
3. Computes new state based on event
4. Restarts itself with new state as input
Other things provided by Durable Functions
extension
• Durable HTTP
• [Deterministic]
• Replay-safe logger
What about those other
durability providers?
Netherite (public preview)
• Higher performance + higher cost
• In high throughput scenarios, cheaper than high scale of other providers
• Supported by Durable Functions
• Not Consumption plan though
• Uses Event Hubs for orchestrator/activity messaging
• Even though Event Hubs do persist data, it is replicated to Azure Storage
after receive
• Max 32 partitions + 20 throughput units = 20 MB/s
• Uses Azure Storage blobs for storing partition state
• Harder to inspect current state
Netherite task hub
SQL (public preview)
• Uses SQL tables for everything
• Supported by Durable Functions
• Not Consumption plan though
• Stored procedures contain most logic
• Portable, no Azure connection required
• Can do data encryption, backups etc.
• Multi-tenant scenarios with schema per tenant
SQL task hub
SQL table contents
Service Bus (+Storage)
• Mature and transactionally consistent
• Not supported by Durable Functions
• Uses 3 Service Bus queues
• Orchestrator = messages for orchestrators
• Worker = messages for activities
• Tracking = orchestration state tracking
• Requires an instance store + blob store, comes with Storage
implementations for them
• State and history tracking
• Oversized messages and sessions
Redis
• Uses a Redis database
• Not supported by Durable Functions
• Workers notified of new messages through Redis channels
• Actual messages sent through Redis lists
• Portable since you can run Redis pretty much anywhere
Service Fabric
• Supported only within a Service Fabric cluster
• Uses SF reliable collections for state storage
• Some features limited, e.g. cannot query instance state if it has
completed over an hour ago
Emulator
• Used for testing DTfx
• Fully in-memory
• Can use for integration testing your own orchestrations if you use
raw DTfx
• Not designed for production use
Demo: Durability providers
with raw Durable Task
Links
• https://westl.in/slides
• https://learn.microsoft.com/en-us/azure/azure-functions/durable/
• https://github.com/Azure/azure-functions-durable-extension
• https://github.com/Azure/durabletask
RACE with us!
Deep Dive into Durable Functions

More Related Content

What's hot

Monitoring Kubernetes with Prometheus
Monitoring Kubernetes with PrometheusMonitoring Kubernetes with Prometheus
Monitoring Kubernetes with Prometheus
Grafana Labs
 
Introduction to Prometheus
Introduction to PrometheusIntroduction to Prometheus
Introduction to Prometheus
Julien Pivotto
 
Observability For Modern Applications
Observability For Modern ApplicationsObservability For Modern Applications
Observability For Modern Applications
Amazon Web Services
 
Why Kubernetes on Azure
Why Kubernetes on AzureWhy Kubernetes on Azure
Why Kubernetes on Azure
Microsoft Tech Community
 
Common issues with Apache Kafka® Producer
Common issues with Apache Kafka® ProducerCommon issues with Apache Kafka® Producer
Common issues with Apache Kafka® Producer
confluent
 
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
Amazon Web Services Korea
 
Prometheus
PrometheusPrometheus
Prometheus
wyukawa
 
Apache Airflow Architecture
Apache Airflow ArchitectureApache Airflow Architecture
Apache Airflow Architecture
Gerard Toonstra
 
Elastic Observability keynote
Elastic Observability keynoteElastic Observability keynote
Elastic Observability keynote
Elasticsearch
 
Orchestrating workflows Apache Airflow on GCP & AWS
Orchestrating workflows Apache Airflow on GCP & AWSOrchestrating workflows Apache Airflow on GCP & AWS
Orchestrating workflows Apache Airflow on GCP & AWS
Derrick Qin
 
Getting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
Syah Dwi Prihatmoko
 
Microservices
Microservices Microservices
Microservices
Trieu Nguyen
 
Monitoring With Prometheus
Monitoring With PrometheusMonitoring With Prometheus
Monitoring With Prometheus
Knoldus Inc.
 
Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...
Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...
Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...
Amazon Web Services
 
MeetUp Monitoring with Prometheus and Grafana (September 2018)
MeetUp Monitoring with Prometheus and Grafana (September 2018)MeetUp Monitoring with Prometheus and Grafana (September 2018)
MeetUp Monitoring with Prometheus and Grafana (September 2018)
Lucas Jellema
 
KubeCon EU 2022: From Kubernetes to PaaS to Err What's Next
KubeCon EU 2022: From Kubernetes to PaaS to Err What's NextKubeCon EU 2022: From Kubernetes to PaaS to Err What's Next
KubeCon EU 2022: From Kubernetes to PaaS to Err What's Next
Daniel Bryant
 
Event Driven Software Architecture Pattern
Event Driven Software Architecture PatternEvent Driven Software Architecture Pattern
Event Driven Software Architecture Pattern
jeetendra mandal
 
Grafana introduction
Grafana introductionGrafana introduction
Grafana introduction
Rico Chen
 
DevOps and Continuous Delivery Reference Architectures (including Nexus and o...
DevOps and Continuous Delivery Reference Architectures (including Nexus and o...DevOps and Continuous Delivery Reference Architectures (including Nexus and o...
DevOps and Continuous Delivery Reference Architectures (including Nexus and o...
Sonatype
 
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
SlideTeam
 

What's hot (20)

Monitoring Kubernetes with Prometheus
Monitoring Kubernetes with PrometheusMonitoring Kubernetes with Prometheus
Monitoring Kubernetes with Prometheus
 
Introduction to Prometheus
Introduction to PrometheusIntroduction to Prometheus
Introduction to Prometheus
 
Observability For Modern Applications
Observability For Modern ApplicationsObservability For Modern Applications
Observability For Modern Applications
 
Why Kubernetes on Azure
Why Kubernetes on AzureWhy Kubernetes on Azure
Why Kubernetes on Azure
 
Common issues with Apache Kafka® Producer
Common issues with Apache Kafka® ProducerCommon issues with Apache Kafka® Producer
Common issues with Apache Kafka® Producer
 
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
 
Prometheus
PrometheusPrometheus
Prometheus
 
Apache Airflow Architecture
Apache Airflow ArchitectureApache Airflow Architecture
Apache Airflow Architecture
 
Elastic Observability keynote
Elastic Observability keynoteElastic Observability keynote
Elastic Observability keynote
 
Orchestrating workflows Apache Airflow on GCP & AWS
Orchestrating workflows Apache Airflow on GCP & AWSOrchestrating workflows Apache Airflow on GCP & AWS
Orchestrating workflows Apache Airflow on GCP & AWS
 
Getting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
 
Microservices
Microservices Microservices
Microservices
 
Monitoring With Prometheus
Monitoring With PrometheusMonitoring With Prometheus
Monitoring With Prometheus
 
Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...
Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...
Instrumenting Kubernetes for Observability Using AWS X-Ray and Amazon CloudWa...
 
MeetUp Monitoring with Prometheus and Grafana (September 2018)
MeetUp Monitoring with Prometheus and Grafana (September 2018)MeetUp Monitoring with Prometheus and Grafana (September 2018)
MeetUp Monitoring with Prometheus and Grafana (September 2018)
 
KubeCon EU 2022: From Kubernetes to PaaS to Err What's Next
KubeCon EU 2022: From Kubernetes to PaaS to Err What's NextKubeCon EU 2022: From Kubernetes to PaaS to Err What's Next
KubeCon EU 2022: From Kubernetes to PaaS to Err What's Next
 
Event Driven Software Architecture Pattern
Event Driven Software Architecture PatternEvent Driven Software Architecture Pattern
Event Driven Software Architecture Pattern
 
Grafana introduction
Grafana introductionGrafana introduction
Grafana introduction
 
DevOps and Continuous Delivery Reference Architectures (including Nexus and o...
DevOps and Continuous Delivery Reference Architectures (including Nexus and o...DevOps and Continuous Delivery Reference Architectures (including Nexus and o...
DevOps and Continuous Delivery Reference Architectures (including Nexus and o...
 
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
 

Similar to Deep Dive into Durable Functions

Building a document e-signing workflow with Azure Durable Functions
Building a document e-signing workflow with Azure Durable FunctionsBuilding a document e-signing workflow with Azure Durable Functions
Building a document e-signing workflow with Azure Durable Functions
Joonas Westlin
 
Azure Messaging with Azure Functions
Azure Messaging with Azure FunctionsAzure Messaging with Azure Functions
Azure Messaging with Azure Functions
Josh Carlisle
 
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018 Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Antonios Chatzipavlis
 
Debugging Microservices - key challenges and techniques - Microservices Odesa...
Debugging Microservices - key challenges and techniques - Microservices Odesa...Debugging Microservices - key challenges and techniques - Microservices Odesa...
Debugging Microservices - key challenges and techniques - Microservices Odesa...
Lohika_Odessa_TechTalks
 
Tech talk microservices debugging
Tech talk microservices debuggingTech talk microservices debugging
Tech talk microservices debugging
Andrey Kolodnitsky
 
Running & Monitoring Docker at Scale
Running & Monitoring Docker at ScaleRunning & Monitoring Docker at Scale
Running & Monitoring Docker at Scale
Datadog
 
(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment
BIOVIA
 
Building large scale, job processing systems with Scala Akka Actor framework
Building large scale, job processing systems with Scala Akka Actor frameworkBuilding large scale, job processing systems with Scala Akka Actor framework
Building large scale, job processing systems with Scala Akka Actor framework
Vignesh Sukumar
 
Deep Dive into Azure Data Factory v2
Deep Dive into Azure Data Factory v2Deep Dive into Azure Data Factory v2
Deep Dive into Azure Data Factory v2
Eric Bragas
 
Insider operating system
Insider   operating systemInsider   operating system
Insider operating system
Aditi Saxena
 
Database as a Service (DBaaS) on Kubernetes
Database as a Service (DBaaS) on KubernetesDatabase as a Service (DBaaS) on Kubernetes
Database as a Service (DBaaS) on Kubernetes
ObjectRocket
 
Tupperware: Containerized Deployment at FB
Tupperware: Containerized Deployment at FBTupperware: Containerized Deployment at FB
Tupperware: Containerized Deployment at FB
Docker, Inc.
 
Real Time Operating Systems, Dynamic Precision: Exploring the Realm of Real-...
Real Time Operating Systems,  Dynamic Precision: Exploring the Realm of Real-...Real Time Operating Systems,  Dynamic Precision: Exploring the Realm of Real-...
Real Time Operating Systems, Dynamic Precision: Exploring the Realm of Real-...
Adobe2801
 
Scott Schnoll - Exchange server 2013 high availability and site resilience
Scott Schnoll - Exchange server 2013 high availability and site resilienceScott Schnoll - Exchange server 2013 high availability and site resilience
Scott Schnoll - Exchange server 2013 high availability and site resilience
Nordic Infrastructure Conference
 
Stay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolithStay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolith
Markus Eisele
 
Scheduling Thread
Scheduling  ThreadScheduling  Thread
Scheduling Thread
MuhammadBilal187526
 
Azure servicefabric
Azure servicefabricAzure servicefabric
Azure servicefabric
Abhishek Sur
 
What's new in JBoss ON 3.2
What's new in JBoss ON 3.2What's new in JBoss ON 3.2
What's new in JBoss ON 3.2
Thomas Segismont
 
Alluxio 2.0 & Near Real-time Big Data Platform w/ Spark & Alluxio
Alluxio 2.0 & Near Real-time Big Data Platform w/ Spark & AlluxioAlluxio 2.0 & Near Real-time Big Data Platform w/ Spark & Alluxio
Alluxio 2.0 & Near Real-time Big Data Platform w/ Spark & Alluxio
Alluxio, Inc.
 
Azure Messaging Crossroads
Azure Messaging CrossroadsAzure Messaging Crossroads
Azure Messaging Crossroads
Sean Feldman
 

Similar to Deep Dive into Durable Functions (20)

Building a document e-signing workflow with Azure Durable Functions
Building a document e-signing workflow with Azure Durable FunctionsBuilding a document e-signing workflow with Azure Durable Functions
Building a document e-signing workflow with Azure Durable Functions
 
Azure Messaging with Azure Functions
Azure Messaging with Azure FunctionsAzure Messaging with Azure Functions
Azure Messaging with Azure Functions
 
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018 Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
 
Debugging Microservices - key challenges and techniques - Microservices Odesa...
Debugging Microservices - key challenges and techniques - Microservices Odesa...Debugging Microservices - key challenges and techniques - Microservices Odesa...
Debugging Microservices - key challenges and techniques - Microservices Odesa...
 
Tech talk microservices debugging
Tech talk microservices debuggingTech talk microservices debugging
Tech talk microservices debugging
 
Running & Monitoring Docker at Scale
Running & Monitoring Docker at ScaleRunning & Monitoring Docker at Scale
Running & Monitoring Docker at Scale
 
(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment(ATS6-PLAT07) Managing AEP in an enterprise environment
(ATS6-PLAT07) Managing AEP in an enterprise environment
 
Building large scale, job processing systems with Scala Akka Actor framework
Building large scale, job processing systems with Scala Akka Actor frameworkBuilding large scale, job processing systems with Scala Akka Actor framework
Building large scale, job processing systems with Scala Akka Actor framework
 
Deep Dive into Azure Data Factory v2
Deep Dive into Azure Data Factory v2Deep Dive into Azure Data Factory v2
Deep Dive into Azure Data Factory v2
 
Insider operating system
Insider   operating systemInsider   operating system
Insider operating system
 
Database as a Service (DBaaS) on Kubernetes
Database as a Service (DBaaS) on KubernetesDatabase as a Service (DBaaS) on Kubernetes
Database as a Service (DBaaS) on Kubernetes
 
Tupperware: Containerized Deployment at FB
Tupperware: Containerized Deployment at FBTupperware: Containerized Deployment at FB
Tupperware: Containerized Deployment at FB
 
Real Time Operating Systems, Dynamic Precision: Exploring the Realm of Real-...
Real Time Operating Systems,  Dynamic Precision: Exploring the Realm of Real-...Real Time Operating Systems,  Dynamic Precision: Exploring the Realm of Real-...
Real Time Operating Systems, Dynamic Precision: Exploring the Realm of Real-...
 
Scott Schnoll - Exchange server 2013 high availability and site resilience
Scott Schnoll - Exchange server 2013 high availability and site resilienceScott Schnoll - Exchange server 2013 high availability and site resilience
Scott Schnoll - Exchange server 2013 high availability and site resilience
 
Stay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolithStay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolith
 
Scheduling Thread
Scheduling  ThreadScheduling  Thread
Scheduling Thread
 
Azure servicefabric
Azure servicefabricAzure servicefabric
Azure servicefabric
 
What's new in JBoss ON 3.2
What's new in JBoss ON 3.2What's new in JBoss ON 3.2
What's new in JBoss ON 3.2
 
Alluxio 2.0 & Near Real-time Big Data Platform w/ Spark & Alluxio
Alluxio 2.0 & Near Real-time Big Data Platform w/ Spark & AlluxioAlluxio 2.0 & Near Real-time Big Data Platform w/ Spark & Alluxio
Alluxio 2.0 & Near Real-time Big Data Platform w/ Spark & Alluxio
 
Azure Messaging Crossroads
Azure Messaging CrossroadsAzure Messaging Crossroads
Azure Messaging Crossroads
 

More from Joonas Westlin

Using feature flags in an ASP.NET Core app on Azure
Using feature flags in an ASP.NET Core app on AzureUsing feature flags in an ASP.NET Core app on Azure
Using feature flags in an ASP.NET Core app on Azure
Joonas Westlin
 
Deep Dive into Durable Functions
Deep Dive into Durable FunctionsDeep Dive into Durable Functions
Deep Dive into Durable Functions
Joonas Westlin
 
Zero Credential Development with Managed Identities
Zero Credential Development with Managed IdentitiesZero Credential Development with Managed Identities
Zero Credential Development with Managed Identities
Joonas Westlin
 
Zero Credential Development with Managed Identities
Zero Credential Development with Managed IdentitiesZero Credential Development with Managed Identities
Zero Credential Development with Managed Identities
Joonas Westlin
 
Zero credential development with managed identities
Zero credential development with managed identitiesZero credential development with managed identities
Zero credential development with managed identities
Joonas Westlin
 
Zero credential development with managed identities
Zero credential development with managed identitiesZero credential development with managed identities
Zero credential development with managed identities
Joonas Westlin
 
Zero Credential Development with Managed Identities for Azure resources
Zero Credential Development with Managed Identities for Azure resourcesZero Credential Development with Managed Identities for Azure resources
Zero Credential Development with Managed Identities for Azure resources
Joonas Westlin
 
Zero Credential Development with Managed Identities
Zero Credential Development with Managed IdentitiesZero Credential Development with Managed Identities
Zero Credential Development with Managed Identities
Joonas Westlin
 
7 Deadly Sins in Azure AD App Development
7 Deadly Sins in Azure AD App Development7 Deadly Sins in Azure AD App Development
7 Deadly Sins in Azure AD App Development
Joonas Westlin
 
Introduction to Azure AD and Azure AD B2C
Introduction to Azure AD and Azure AD B2CIntroduction to Azure AD and Azure AD B2C
Introduction to Azure AD and Azure AD B2C
Joonas Westlin
 
Get rid of credentials from your code: Using Managed identities for Azure res...
Get rid of credentials from your code: Using Managed identities for Azure res...Get rid of credentials from your code: Using Managed identities for Azure res...
Get rid of credentials from your code: Using Managed identities for Azure res...
Joonas Westlin
 
Programming with Azure Active Directory
Programming with Azure Active DirectoryProgramming with Azure Active Directory
Programming with Azure Active Directory
Joonas Westlin
 

More from Joonas Westlin (12)

Using feature flags in an ASP.NET Core app on Azure
Using feature flags in an ASP.NET Core app on AzureUsing feature flags in an ASP.NET Core app on Azure
Using feature flags in an ASP.NET Core app on Azure
 
Deep Dive into Durable Functions
Deep Dive into Durable FunctionsDeep Dive into Durable Functions
Deep Dive into Durable Functions
 
Zero Credential Development with Managed Identities
Zero Credential Development with Managed IdentitiesZero Credential Development with Managed Identities
Zero Credential Development with Managed Identities
 
Zero Credential Development with Managed Identities
Zero Credential Development with Managed IdentitiesZero Credential Development with Managed Identities
Zero Credential Development with Managed Identities
 
Zero credential development with managed identities
Zero credential development with managed identitiesZero credential development with managed identities
Zero credential development with managed identities
 
Zero credential development with managed identities
Zero credential development with managed identitiesZero credential development with managed identities
Zero credential development with managed identities
 
Zero Credential Development with Managed Identities for Azure resources
Zero Credential Development with Managed Identities for Azure resourcesZero Credential Development with Managed Identities for Azure resources
Zero Credential Development with Managed Identities for Azure resources
 
Zero Credential Development with Managed Identities
Zero Credential Development with Managed IdentitiesZero Credential Development with Managed Identities
Zero Credential Development with Managed Identities
 
7 Deadly Sins in Azure AD App Development
7 Deadly Sins in Azure AD App Development7 Deadly Sins in Azure AD App Development
7 Deadly Sins in Azure AD App Development
 
Introduction to Azure AD and Azure AD B2C
Introduction to Azure AD and Azure AD B2CIntroduction to Azure AD and Azure AD B2C
Introduction to Azure AD and Azure AD B2C
 
Get rid of credentials from your code: Using Managed identities for Azure res...
Get rid of credentials from your code: Using Managed identities for Azure res...Get rid of credentials from your code: Using Managed identities for Azure res...
Get rid of credentials from your code: Using Managed identities for Azure res...
 
Programming with Azure Active Directory
Programming with Azure Active DirectoryProgramming with Azure Active Directory
Programming with Azure Active Directory
 

Recently uploaded

WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
dakas1
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
Peter Muessig
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
VALiNTRY360
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
ShulagnaSarkar2
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
Rakesh Kumar R
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
Quickdice ERP
 
GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Green Software Development
 
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
Bert Jan Schrijver
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Remote DBA Services
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 
zOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL DifferenceszOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL Differences
YousufSait3
 

Recently uploaded (20)

WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
 
GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
 
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 
zOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL DifferenceszOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL Differences
 

Deep Dive into Durable Functions

  • 1. Deep Dive into Durable Functions CloudBrew Joonas Westlin
  • 2. Speaker Intro • Joonas Westlin • Developer / Architect @ Zure @JoonasWestlin joonasw.net
  • 3. Key takeaways from this presentation • Orchestrator and activity communication • State storage using the Azure Storage provider • Differences in other durability providers • Goal: improve your ability to debug Durable Functions
  • 4. Introduction/refresher • Durable Functions = extension for Azure Functions that allows execution of Durable Task orchestrations • Durable Task framework allows developers to build long-running workflows with persistent state, using well known async-await constructs in .NET • Orchestrator functions define the workflow, activity functions do non-deterministic work
  • 5. Durable Task workers • Workers execute orchestrators and activities by requesting work items from the durability provider • Multiple workers run in parallel = increased throughput • Activity work items can run on any worker, but orchestrator work items cannot • Danger of state corruption if more than 1 worker takes a work item for same orchestration simultaneously
  • 7. Storage durability provider • Convenient choice for Azure Functions as it has a Storage account already • Cheap option, consumption-based pricing in Storage • Tip: using a v1 Storage account is cheaper (not in new regions) • Can run emulator locally for Storage
  • 9. Orchestrator and activity communication • Orchestrator gets executed several times; each time a set of outbound messages is generated for activities to be triggered • These are sent through the durability provider and workers receive them • Once the worker is done with the activity, it sends a message back to the orchestrator
  • 10. Control queue 0 Control queue 1 Control queue 2 Control queue 3 Work item queue
  • 12. Durable Entities • We looked at orchestrators and activities but where are Durable Entities? • Entities are just orchestrators • At high level, you can think of entities as an orchestrator that: 1. Gets state from input 2. Waits for an external event (operation) 3. Computes new state based on event 4. Restarts itself with new state as input
  • 13. Other things provided by Durable Functions extension • Durable HTTP • [Deterministic] • Replay-safe logger
  • 14. What about those other durability providers?
  • 15. Netherite (public preview) • Higher performance + higher cost • In high throughput scenarios, cheaper than high scale of other providers • Supported by Durable Functions • Not Consumption plan though • Uses Event Hubs for orchestrator/activity messaging • Even though Event Hubs do persist data, it is replicated to Azure Storage after receive • Max 32 partitions + 20 throughput units = 20 MB/s • Uses Azure Storage blobs for storing partition state • Harder to inspect current state
  • 17. SQL (public preview) • Uses SQL tables for everything • Supported by Durable Functions • Not Consumption plan though • Stored procedures contain most logic • Portable, no Azure connection required • Can do data encryption, backups etc. • Multi-tenant scenarios with schema per tenant
  • 20. Service Bus (+Storage) • Mature and transactionally consistent • Not supported by Durable Functions • Uses 3 Service Bus queues • Orchestrator = messages for orchestrators • Worker = messages for activities • Tracking = orchestration state tracking • Requires an instance store + blob store, comes with Storage implementations for them • State and history tracking • Oversized messages and sessions
  • 21. Redis • Uses a Redis database • Not supported by Durable Functions • Workers notified of new messages through Redis channels • Actual messages sent through Redis lists • Portable since you can run Redis pretty much anywhere
  • 22. Service Fabric • Supported only within a Service Fabric cluster • Uses SF reliable collections for state storage • Some features limited, e.g. cannot query instance state if it has completed over an hour ago
  • 23. Emulator • Used for testing DTfx • Fully in-memory • Can use for integration testing your own orchestrations if you use raw DTfx • Not designed for production use
  • 25. Links • https://westl.in/slides • https://learn.microsoft.com/en-us/azure/azure-functions/durable/ • https://github.com/Azure/azure-functions-durable-extension • https://github.com/Azure/durabletask

Editor's Notes

  1. Regions that came online after Oct 1, 2020 have similar price for v1 and v2 storage
  2. Control queues trigger orchestrators # of partitions = # of control queues Max 1 worker listens to each queue, controlled by blob leases Work item queue triggers activities Only one queue, many listeners Two tables used to store state, one for current orchestration statuses and one for orchestration history