The Durable Functions extension to Azure Functions allows developers to build workflows using higher level abstractions. But what is really going on under the hood? It can be useful to know how all the pieces connect in case you are every trying to solve a trickier issue. In this presentation, we will take a deep dive into the internals of the Durable Functions extension and the Durable Task framework. In addition to the default Azure Storage durability provider, we will look at the other officially supported providers and how they differ in their implementations.

1. Deep Dive into
Durable Functions
CloudBurst
Joonas Westlin

2. Speaker Intro
• Joonas Westlin
• Developer / Architect @ Zure
@JoonasWestlin
joonasw.net

3. Key takeaways from this presentation
• Orchestrator and activity communication
• State storage using the Azure Storage provider
• Differences in other durability providers
• Goal: improve your ability to debug Durable Functions

4. Introduction/refresher
• Durable Functions = extension for Azure Functions that allows
execution of Durable Task orchestrations
• Durable Task framework allows developers to build long-running
workflows with persistent state, using well known async-await
constructs in .NET
• Orchestrator functions define the workflow, activity functions do
non-deterministic work

5. Durable Task workers
• Workers execute orchestrators and activities by requesting work
items from the durability provider
• Multiple workers run in parallel = increased throughput
• Activity work items can run on any worker, but orchestrator work
items cannot
• Danger of state corruption if more than 1 worker takes a work item for
same orchestration simultaneously

6. Durability providers
Storage
Service
Bus
Netherite SQL
Redis
Service
Fabric
Emulator

7. Storage durability provider
• Convenient choice for Azure Functions as it has a Storage account
already
• Cheap option, consumption-based pricing in Storage
• Tip: using a v1 Storage account is cheaper (not in new regions)
• Can run emulator locally for Storage

8. Storage task hub

9. Orchestrator and activity communication
• Orchestrator gets executed several times; each time a set of
outbound messages is generated for activities to be triggered
• These are sent through the durability provider and workers
receive them
• Once the worker is done with the activity, it sends a message back
to the orchestrator

10. Control queue 0
Control queue 1
Control queue 2
Control queue 3
Work item queue

11. Demo: Durable Function
execution

12. Durable Entities
• We looked at orchestrators and activities but where are Durable
Entities?
• Entities are just orchestrators
• At high level, you can think of entities as an orchestrator that:
1. Gets state from input
2. Waits for an external event (operation)
3. Computes new state based on event
4. Restarts itself with new state as input

13. Other things provided by Durable Functions
extension
• Durable HTTP
• [Deterministic]
• Replay-safe logger

14. What about those other
durability providers?

15. Netherite (public preview)
• Higher performance + higher cost
• In high throughput scenarios, cheaper than high scale of other providers
• Supported by Durable Functions
• Not Consumption plan though
• Uses Event Hubs for orchestrator/activity messaging
• Even though Event Hubs do persist data, it is replicated to Azure Storage
after receive
• Max 32 partitions + 20 throughput units = 20 MB/s
• Uses Azure Storage blobs for storing partition state
• Harder to inspect current state

16. Netherite task hub

17. SQL (public preview)
• Uses SQL tables for everything
• Supported by Durable Functions
• Not Consumption plan though
• Stored procedures contain most logic
• Portable, no Azure connection required
• Can do data encryption, backups etc.
• Multi-tenant scenarios with schema per tenant

18. SQL task hub

19. SQL table contents

20. Service Bus (+Storage)
• Mature and transactionally consistent
• Not supported by Durable Functions
• Uses 3 Service Bus queues
• Orchestrator = messages for orchestrators
• Worker = messages for activities
• Tracking = orchestration state tracking
• Requires an instance store + blob store, comes with Storage
implementations for them
• State and history tracking
• Oversized messages and sessions

21. Redis
• Uses a Redis database
• Not supported by Durable Functions
• Workers notified of new messages through Redis channels
• Actual messages sent through Redis lists
• Portable since you can run Redis pretty much anywhere

22. Service Fabric
• Supported only within a Service Fabric cluster
• Uses SF reliable collections for state storage
• Some features limited, e.g. cannot query instance state if it has
completed over an hour ago

23. Emulator
• Used for testing DTfx
• Fully in-memory
• Can use for integration testing your own orchestrations if you use
raw DTfx
• Not designed for production use

24. Demo: Durability providers
with raw Durable Task

25. Links
• https://westl.in/slides
• https://learn.microsoft.com/en-us/azure/azure-functions/durable/
• https://github.com/Azure/azure-functions-durable-extension
• https://github.com/Azure/durabletask