Skip to main content
Paco de la Cruz
Azure Durable Functions
Serverless & Stateful Orchestrations
@pacodelacruz
Global Integration Bootcamp 2019, Melbourne
@pacodelacruz
linkedin.com/in/pacodelacruz
pacodelacruzag.wordpress.com
slideshare.net/pacodelac/presentations
Agenda
Evolution of Application Platforms
On-premises IaaS PaaS Serverless
“Serverless” and its benefits
Event-driven scaling
not resource-driven
Pay only for what
you use
Server abstraction
Focus on value
Azure Functions in a nutshell
Event Triggers Code Outputs
React and get inputs from
a growing list of services
(triggers and input
bindings)
C#, F#,
Node.js, Java,
Phyton (Preview)
Send results to a
growing list of services
(output bindings)
Some challenges of Azure Functions
Manageable Sequencing
+ Error Handling / Compensation
Fanning-out & Fanning-in Http-based
Async Long-running APIs
Human InteractionFlexible Automated Long-running
Process Monitoring
External Events Correlation
Start
Get Status
Durable Function Patterns
1: Function Chaining 2: Fanning-out & Fanning-in 3: Async HTTP APIs
5: Human Interaction4: Monitoring ?: External Events Correlation
Start
Get Status
Durable Functions in a nutshell
Based on
Durable Task Framework
Using Azure Storage
(Fully Managed and Abstracted)
To Implement stateful
workflows-as-code
(C#, F# and Node.js)
Azure Functions
Extension
Durable Functions Components
Activity Function Activity Function Activity Function
Orchestrator Function
Orchestration Client
Start
Get Status
Send Event
Wait for Completion
Terminate
Stateless
Single Step
Inputs and Outputs
Stateful
Process Manager
Call Activity Functions
Advanced Retries
Error Handling
Fall Back / Compensation
Checkpointing
Dehydrates during activities
Rehydrates at responses / events
Orchestration Function Limitations
Orchestration code must
• Be deterministic
(e.g. no NewGuid(), Random, DateTime.Now(), Http calls, etc.)
• Be non-blocking: (no I/O, Thread.Sleep(), etc.)
• Never initiate any async operation: without using its
context
• Avoid infinite loops
There are workarounds
Orchestration Considerations
Every activity function invocation and response are sent via queues
Activity Function outputs are checkpointed and persisted
Every async call persists the orchestration state into storage table
Every awaited activity function response, timer or external event,
will recover the persisted state into memory
The orchestration is replayed up-to the last persisted point
Activity Functions are not replayed
Event-Sourcing pattern is used
Function Chaining Pattern
public static async Task<object> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallActivityAsync<object>("F1");
var y = await ctx.CallActivityAsync<object>("F2", x);
return await ctx.CallActivityAsync<object>("F3", y);
}
catch (Exception ex)
{
// error handling / fall back / compensation
}
}
Function Chaining Pattern
public static async Task<object> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallActivityAsync<object>("F1");
var y = await ctx.CallActivityAsync<object>("F2", x);
return await ctx.CallActivityAsync<object>("F3", y);
}
catch (Exception ex)
{
// error handling / fall back / compensation
}
}
Function Chaining Pattern
public static async Task<object> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
try
{
var x = await ctx.CallActivityAsync<object>("F1");
var y = await ctx.CallActivityAsync<object>("F2", x);
return await ctx.CallActivityAsync<object>("F3", y);
}
catch (Exception ex)
{
// error handling / fall back / compensation
}
}
Fan-out & Fan-In Pattern
public static async Task<int> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
object[] workBatch = await ctx.CallActivityAsync<object[]>("F1");
var tasks = new Task<long>[workBatch.Length];
for (int i = 0; i < workBatch.Length; i++)
{
tasks[i] = ctx.CallActivityAsync<int>("F2", workBatch[i]);
}
await Task.WhenAll(tasks);
long sum = tasks.Sum(t => t.Result);
return sum;
}
Fan-out & Fan-In Pattern
public static async Task<int> Run(
[OrchestrationTrigger] DurableOrchestrationContext ctx)
{
object[] workBatch = await ctx.CallActivityAsync<object[]>("F1");
var tasks = new Task<long>[workBatch.Length];
for (int i = 0; i < workBatch.Length; i++)
{
tasks[i] = ctx.CallActivityAsync<int>("F2", workBatch[i]);
}
await Task.WhenAll(tasks);
long sum = tasks.Sum(t => t.Result);
return sum;
}
CallActivityWithRetryAsync
Orchestration Client
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync("myOrchestrator", eventData);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var res = starter.CreateCheckStatusResponse(req, instanceId);
return res;
}
Orchestration Client
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync("myOrchestrator", eventData);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var res = starter.CreateCheckStatusResponse(req, instanceId);
return res;
}
Orchestration Client
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClientBase starter, ILogger log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync("myOrchestrator", eventData);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var res = starter.CreateCheckStatusResponse(req, instanceId);
return res;
}
Durable Functions vs Logic Apps?
vs
Durable Functions Logic Apps
Both allow implementing advanced workflow patterns
C#, F# and JavaScript Visual designer and WDL
Bindings (~ 20 supported) 250+ connectors
Portable Runtime Runs only on Azure
Monitoring based on App Insights & APIs Rich monitoring & management tools
Serverless + dedicated & isolated Serverless + dedicated & isolated (ISE)
platform.deloitte.com.au/articles/azure-durable-functions-vs-logic-apps
Additional Resources
Twitter @azurefunctions
Documentation aka.ms/durablefunctions
Live Web Cast aka.ms/azurefunctionslive
Repos github.com/Azure/azure-functions-durable-extension
github.com/Azure/azure-functions-durable-js
Samples github.com/Azure/azure-functions-durable-
extension/tree/master/samples
Pluralsight course app.pluralsight.com/library/courses/
azure-durable-functions-fundamentals
Any questions so far?
Let’s have
some fun!
Demo 1
Development and Debugging
Furry Models
Cat Application Approval via email
Orchestration
Function
Start
Send Approval
Request via Email
Create Timer
External
Event
Timer
Expires
WhenAny
Move Blob to
Corresponding Container
End
(Requests
blob container)
(Approved or Rejected
blob container)
Orchestration
Function
Start
Send Approval
Request via Email
Create Timer
External
Event
Timer
Expires
WhenAny
Move Blob to
Corresponding Container
End
BlobTrigger
Function
HttpTrigger
Process Approval
(Requests
blob container)
(Approved or Rejected
blob container)
Orchestration Client Activity Functions
ActivityTrigger
Send Approval
Request via Email
ActivityTrigger
Move Blob to
Corresponding Container
HttpTrigger
Check Status
Demo 2
Execution & Monitoring on Azure
Furry Models
Cat Application Approval via Slack
Orchestration
Function
Start
Send Approval
Request via Slack
Create Timer
External
Event
Timer
Expires
WhenAny
Move Blob to
Corresponding Container
End
BlobTrigger
Function
HttpTrigger
Process Approval
(Requests
blob container)
(Approved or Rejected
blob container)
Orchestration Client Activity Functions
ActivityTrigger
Send Approval
Request via Slack
ActivityTrigger
Move Blob to
Corresponding Container
HttpTrigger
Check Status
Let your Cat apply!
Great opportunity to get famous!
Details about the demos:
platform.deloitte.com.au/articles/azure-durable-functions-approval-workflow-with-sendgrid
platform.deloitte.com.au/articles/azure-durable-functions-approval-workflow-with-slack
github.com/pacodelacruz/durable-functions-furry-models
Q & A
@pacodelacruz
linkedin.com/in/pacodelacruz
pacodelacruzag.wordpress.com
slideshare.net/pacodelac/presentations
Thanks!