Working with data
using Azure
Functions
Liam Moat
Steph Locke

What we’re here to talk
about
• What are Azure Functions?
• How to position serverless for your workloads
• Using Azure Functions for data processing

Steph & Liam
Steph Locke
Tw: @theStephLocke
Li: stephanielocke
GH: stephlocke
Liam Moat
Li: liammoatcom
GH: liammoat

Azure Functions

Level
of
abstraction
Focus on business logic
Physical server
Virtual machine
PaaS
Containerization
Serverless

No infrastructure
management
Developers can just focus on
their code—without needing
to worry about provisioning
and managing infrastructure
Instant, event-
driven scalability
Application components
react to events and triggers
in near real-time with
virtually unlimited scalability
Pay-per-use
Only pay for what you use:
billing is typically calculated
on the number of function
calls, code execution time, and
memory used*
*Supporting services, like storage and networking, may be charged separately.

Functions-as-a-Service programming model use functions to achieve true serverless compute
Single
responsibility
Functions are single-
purposed, reusable pieces of
code that process an input
and return a result
Short-lived
Functions don’t stick around
when finished executing,
freeing up resources
for further executions
Stateless
Functions don’t hold any
persistent state and
don’t rely on the state of any
other processes
Event-driven
and scalable
Functions respond to
predefined events, and are
instantly replicated
as many times as needed

An event-based, serverless compute experience that accelerates app development
Integrated
programming
model
Use built-in triggers and
bindings to define when a
function is invoked and to what
data it connects
End-to-end
development
experience
Take advantage of a complete,
end-to-end development
experience with Functions—from
building and debugging
locally on major platforms like
Windows, macOS, and Linux
to deploying and monitoring in
the cloud
Hosting
options
flexibility
Choose the deployment model
that better fits your business
needs without compromising
development experience
Fully
managed and
cost-effective
Automated and flexible scaling
based on your workload
volume, keeping the focus on
adding value instead of
managing infrastructure

Integrated programming model
Azure Functions features input/output bindings
which provide a means of pulling data or
pushing data to other services. These bindings
work for both Microsoft and third-party services
without the need to hard-code integrations.
Trigger
Input binding
Output binding
Trigger object
Your code
Input object
Output object

The “Old” Way - Pseudocode
func Run()
{
var connectionString = CloudConfigurationManager.GetSetting("storage:connection")
var storageAccount = CloudStorageAccount.Parse(connectionString)
var eventHubClient = EventHubClient.CreateFromConnectionString(eventHubConnection)
func Poll()
{
var queueClient = storageAccount.CreateCloudQueueClient()
var queue = queueClient.GetQueueReference("myqueue-items")
queue.CreateIfNotExists()
var msg = queue.PeekMessage().AsString
var tableClient = storageAccount.CreateCloudTableClient()
var table = tableClient.GetTableReference("people")
table.CreateIfNotExists()
var customer = table.Execute(TableOperation.Retrieve<Customer>(“Customer”, msg))
// do something with customer - business logic goes here.
eventHubClient.Send(new EventData(...))
queue.DeleteMessage(msg);
Sleep(10 seconds)
Poll()
}
Poll()
}

Triggers - Pseudocode
func Run([QueueTrigger("myqueue-items")] string msg)
{
var connectionString = CloudConfigurationManager.GetSetting("storage:connection")
var storageAccount = CloudStorageAccount.Parse(connectionString)
var eventHubClient = EventHubClient.CreateFromConnectionString(eventHubConnection)
var tableClient = storageAccount.CreateCloudTableClient()
var table = tableClient.GetTableReference("people")
table.CreateIfNotExists()
var customer = table.Execute(TableOperation.Retrieve<Customer>("Customer", msg))
// do something with customer - business logic goes here.
eventHubClient.Send(new EventData(...))
}

Inputs - Pseudocode
func Run([QueueTrigger("myqueue-items")] string myQueueItem
[Table("people", "my-partition", "{queueTrigger}")] Customer customer)
{
var eventHubClient = EventHubClient.CreateFromConnectionString(eventHubConnection)
// do something with customer - business logic goes here.
eventHubClient.Send(new EventData(...))
}

Outputs - Pseudocode
[return: EventHub("event-hub", Connection = "EventHubConnection")]
Func EventData Run([QueueTrigger("myqueue-items")] string myQueueItem
[Table("MyTable", "MyPartition", "{queueTrigger}")] Customer customer)
{
// do something with customer - business logic goes here.
return new EventData(...)
}

Streamlining
connections and
improving security
Use Managed Identities for downstream
connections and for clients that invoke
Functions
Leverage Azure Key Vault for services
without support for MI
Managed Identities and Key Vault secrets
allows others to manage access and
minimises risk of leaked credentials
Use Azure Functions v4 for MI support

Managed Identities + Infrastructure as Code FTW
Step 1:
Assign the Managed Identity
access to resources
Step 2:
Add simplified values to app
settings
Step 3:
Use simple connection
references in Functions

DEMO: Creating an Azure Function project
IaC (bicep), VS Code (Azure Functions and bicep extensions), Azure
Function Core Tools, Github Copilot

Working with data

Automation of scheduled tasks
S C E N A R I O E X A M P L E
Financial services
A customer database is analyzed
for duplicate entries every
15 minutes, to avoid multiple
communications being sent out
to same customers
A function cleans a database
every 15 minutes…
…deduplicating entries
based on business logic

Handling data with a schedule
Timer.cs
[FunctionName("TimerTriggerCSharp")]
public static void Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, ILogger log)
{
// Business logic goes here…
}
Use NCRONTAB specifications for schedules
Schedules can be managed in appsettings to put all schedules in a single location
Use inputs and outputs to handle whatever you need to do

Real-time stream processing
S C E N A R I O E X A M P L E
ISV
Huge amounts of telemetry
data is collected from a massive
cloud app. That data is
processed in near real-time and
stored in a DB for use
in an analytics dashboard
App or device
producing data
Event Hubs ingests
telemetry data A function processes
the data…
…and sends it to
Cosmos DB
Data used for
dashboard
visualizations

Handling events
Queue.cs
public static class QueueFunctions
{
[FunctionName("QueueTrigger")]
public static void QueueTrigger(
[QueueTrigger("items")] string myQueueItem,
ILogger log)
{
// Business logic goes here…
}
}
Use a trigger that listens to an event publisher
Process messages with a schema to improve quality
Choose to process single messages or micro-batches

Handling data on change
S C E N A R I O E X A M P L E
Financial Services
Colleagues use mobile banking
to reimburse each other for
lunch: the person who paid for
lunch requests payment through
his mobile app, triggering a
notification on his colleagues’
phones.

Handling data when there’s changes
CosmosDB.cs
[FunctionName("CosmosTrigger")]
public static void Run([CosmosDBTrigger(
databaseName: "CorpDB",
containerName: "CorpDB",
Connection = "CorpDB",
LeaseContainerName = "leases")]IReadOnlyList<Person> documents,
ILogger log)
{
// Business logic goes here…
}
Use a trigger for a source that has CDC – primarily Cosmos DB and Azure SQL
Process messages with a schema to improve quality
Use inputs to get additional record sets to support processing

Handling data on request
S C E N A R I O E X A M P L E
Professional Services
A SaaS solution provides
extensibility through webhooks,
which can be implemented
through Functions, to automate
certain workflows.

Handling data when requested
HTTP.cs
[FunctionName("HttpTriggerCSharp")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
HttpRequest req, ILogger log)
{
// Business logic goes here…
}
Use HTTP triggers to create APIs or webhook driven activities
Use OpenAPI decorators to add documentation to your functions
Use Managed Identity on resources that connect or do pass through auth where
possible

?
Workflows and orchestration
with Durable Functions
P A T T E R N S / U S E C A S E S
Durable Functions is an
extension of Azure Functions
that lets you write stateful
functions in a serverless
compute environment
Manageable sequencing +
error handling/compensation
Fanning out and fanning in External events correlation
Flexible automated long-running
process monitoring
Start
Get status
Http-based async long-
running APIs Human interaction

Handling data with multiple steps
Durable.cs
[FunctionName("Chaining")]
public static async Task<object> Run(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
var x = await context.CallActivityAsync<object>("F1", null);
await context.CallActivityAsync<object>("F2", x);
}
Orchestrate complex or stateful data flows using Durable Functions
Use for different cases like aggregating, fanning out, human in the loop, or
chaining functions

DEMO: Creating different Azure Functions
VS Code (Azure Functions & Azurite extensions), Azure Functions Core
Tools, Github Copilot, Event Hub, Azure SQL, CosmosDB

Closing

What we talked about
• What are Azure Functions?
• How to position serverless for your workloads
• Using Azure Functions for data processing

Try it yourself
Learn with our Cloud Skills
Challenge
aka.ms/sqlbits-dwf
Check out our repo to see
things in detail
aka.ms/sqlbits-dwf-demo
Give it ago with free Azure
resources
Free Services