#AIMSTalks
Serverless Messaging with
Microsoft Azure
Steef-Jan Wiggers
Azure & IoT Domain Lead
steefjan.wiggers@codit.eu
+31 653 12 29 57
@SteefJan
nl.linkedin.com/in/steefjan
What can you expected in this session
• Serverless messaging
• Messaging In Azure
• Messaging scenarios
• Demo’s
Serverless
5
Evolution to Serverless
• You can choose:
• VM →
• Containers →
• Orchestrators →
• PaaS →
• Serverless
On-premise IaaS PaaS Serverless
“Serverless computing is a cloud-computing execution model in
which the cloud provider dynamically manages the allocation of
machine resources. Pricing is based on the actual amount of
resources consumed by an application, rather than on pre-
purchased units of capacity. It is a form of utility computing.”
Source (Wikipedia Serverless Computing):
https://en.wikipedia.org/wiki/Serverless_computing
Defining Serverless
Thoughts on Serverless
- Infinite scale requires atomic workloads and
infinite resources.
- Reality is resources are always finite and many
workloads not atomic.
- Messaging is for decoupling and transferring
state or data secure and reliably.
Serverless in Azure
- Serverless Compute → Azure Functions
- Serverless Database → Azure Cosmos DB
- Serverless Events → Azure Event Grid
- Serverless Realtime → Azure Signal R Service
- Serverless Workflow → Logic Apps &
Durable Functions
- Serverless IoT → IoT Hubs
- Serverless Analytics → Application Insights
Serverless Messaging
Message: >Down
Message: >Stay
Message: >Come
Is this serverless messaging?
What I mean is: You have options!
We have various messaging capabilities in Azure!
Event Hubs
Telemetry
stream
ingestion
Service Bus &
Azure Queues
Cloud
messaging
Event Grid
Event
distribution
Logic Apps
Workflow
and LOB
Integration
IoT Hub
IoT messaging
and manage-
ment
Notification
Hubs
Mobile push
notifications
Relay
Discovery,
Firewall/NAT
Traversal
Azure Messaging Services
What do people do with messaging?
- Order processing
- Logging / telemetry
- Connected devices
- Notification / event driven systems
Messaging Services in Azure
•Azure Service Bus
•Event Hubs
•Event Grid
•Storage Queue
Service Bus Event Hubs Storage Queues
Big data streamingEnterprise messaging Simple task queues
Event Grid
Cross cloud reactive
eventing
Building Blocks
Enterprise Messaging
Business state transition
• Transfer of money and material
• Central arbiter of state / truth of transition or ownership
• Rich control of communication features
• You know a lot ahead of time
• You know the nouns:
• Who
• What
• Where
• Competitive offering from Amazon: SQS, Amazon MQ, Google
Pub/Sub
Reacting dynamically to the world around you
• Cross App / Service / Cloud / Organization
• Light weight
• Low cost
• Few features, but important ones
• You probably don’t know the nouns and may not
care
Eventing
Service Bus
Enterprise Messaging
Azure Event Grid
Reactive Programming
• Queue Semantics
• Rich filters
• Instantaneous consistency
• Strict ordering
• JMS
• Geo-replication & Availability
• Rich broker features
• Event raised by sources
• Independent individual
messages
• Push mechanism
• Filtering and routing
• Pay as you go
• Fan out
Push-Pull - Push-Push
Cloud Pub/Sub Models
Task Queue
Coordinate simple tasks across compute (workers)
• Low cost
• Pay as you Go
• Few features
• Easy to use
• Have as many queues as you like
Cloud Queues
Service Bus Queues - Storage Queues
Service Bus
Queues
Storage
Queues
• Queue Semantics
• Rich filters
• Instantaneous consistency
• Strict ordering
• JMS
• Geo-replication & Availability
• Rich broker features
• Part of Storage infrastructure
• Simple REST-based interface
• Pull mechanism
• Reliable
• Persistence messaging
• Only queues
Big Data Streams
Flow data and telemetry in real-time
• Durable re-playable buffer
• Designed as a stream
• Events to an event hub via AMQP or HTTP
• Real-time and batch processing
• Feeding Big Data and Analytics
• Pre-purchase capacity in terms of throughput units
• Offers an Kafka Endpoint
• Competitive offering from Amazon Kinesis, Apache Kafka, and
Google Pub/Sub
Brought to you by Wagner Silveira - https://goo.gl/LcjMQB
Kafka Endpoint - Demo
Consumer
Groups
Producer
Consumer
Event Hubs
Produce - Consume
.NET Code
Messaging considerations
Protocol
Format
Size
Security
Frequency
Reliability
Monitoring
Networking
VMSS
Cloud Services
VMs
Logic Apps
SQL DB
Express Route
ASA
SQL DW
Power BI
Data Lake
Cosmos DB
Functions
AKS
ACS
Messaging scenario’s
Tollbooth License Plate Recognition
License plate OCR
• Use the Recognition Services Computer Vision API and its built-
in OCR capabilities
• The image processing function can make a REST call to send the
photo and read the JSON data in return
Smart solution
Storage blob Azure
Event Grid
Function Cognitive
Services
Vision API
Smart Services - Demo
Change feed
Serverless Home Automation - Demo
data
data
data
data
start/stop
data
start/stop
start/stop
Cosmos DB
Event Hubs
IoT Hub
Functions
data
data
notify
The processing required by an application can easily be broken
down into a set of independent steps.
Pipes and filters
Microservices containers
Request DataProvisioning API
Gateway
Provision Request
Message
Storage
SQL Instance
in ElasticPool
Key Vault
Provisioning Queue
Provisioning
WebJob
Trigger
Provision
Store
Connection
strings
Provision
Container
Azure AppService
Provision Request
Message
Microservice processing - Demo
Request
message
Container
ServiceBus
Listener
Service
Get Blob
Edi-file Blob
Storage
Json-file Blob
Storage
Storage
Explorer
message
ServiceBus
Listener
Service
Next step!
K8S – Microservice Log
Use Azure Messaging Services together
Data and event pipeline - Demo
Summary
Choose the right messaging service(s)
Workloads
Cross-platform
Costs
DevOps and Management
Security and compliance
Happy Messaging!
Thank you!
Keep in touch.
Call or mail us. Ask us. Happy to help.

Serverless Messaging with Microsoft Azure by Steef-Jan Wiggers

  • 1.
  • 2.
    Serverless Messaging with MicrosoftAzure Steef-Jan Wiggers Azure & IoT Domain Lead
  • 3.
    steefjan.wiggers@codit.eu +31 653 1229 57 @SteefJan nl.linkedin.com/in/steefjan
  • 4.
    What can youexpected in this session • Serverless messaging • Messaging In Azure • Messaging scenarios • Demo’s
  • 5.
  • 6.
    Evolution to Serverless •You can choose: • VM → • Containers → • Orchestrators → • PaaS → • Serverless On-premise IaaS PaaS Serverless
  • 7.
    “Serverless computing isa cloud-computing execution model in which the cloud provider dynamically manages the allocation of machine resources. Pricing is based on the actual amount of resources consumed by an application, rather than on pre- purchased units of capacity. It is a form of utility computing.” Source (Wikipedia Serverless Computing): https://en.wikipedia.org/wiki/Serverless_computing Defining Serverless
  • 8.
    Thoughts on Serverless -Infinite scale requires atomic workloads and infinite resources. - Reality is resources are always finite and many workloads not atomic. - Messaging is for decoupling and transferring state or data secure and reliably.
  • 9.
    Serverless in Azure -Serverless Compute → Azure Functions - Serverless Database → Azure Cosmos DB - Serverless Events → Azure Event Grid - Serverless Realtime → Azure Signal R Service - Serverless Workflow → Logic Apps & Durable Functions - Serverless IoT → IoT Hubs - Serverless Analytics → Application Insights
  • 10.
    Serverless Messaging Message: >Down Message:>Stay Message: >Come Is this serverless messaging?
  • 11.
    What I meanis: You have options! We have various messaging capabilities in Azure! Event Hubs Telemetry stream ingestion Service Bus & Azure Queues Cloud messaging Event Grid Event distribution Logic Apps Workflow and LOB Integration IoT Hub IoT messaging and manage- ment Notification Hubs Mobile push notifications Relay Discovery, Firewall/NAT Traversal
  • 12.
  • 13.
    What do peopledo with messaging? - Order processing - Logging / telemetry - Connected devices - Notification / event driven systems
  • 14.
    Messaging Services inAzure •Azure Service Bus •Event Hubs •Event Grid •Storage Queue
  • 15.
    Service Bus EventHubs Storage Queues Big data streamingEnterprise messaging Simple task queues Event Grid Cross cloud reactive eventing Building Blocks
  • 16.
    Enterprise Messaging Business statetransition • Transfer of money and material • Central arbiter of state / truth of transition or ownership • Rich control of communication features • You know a lot ahead of time • You know the nouns: • Who • What • Where • Competitive offering from Amazon: SQS, Amazon MQ, Google Pub/Sub
  • 17.
    Reacting dynamically tothe world around you • Cross App / Service / Cloud / Organization • Light weight • Low cost • Few features, but important ones • You probably don’t know the nouns and may not care Eventing
  • 18.
    Service Bus Enterprise Messaging AzureEvent Grid Reactive Programming • Queue Semantics • Rich filters • Instantaneous consistency • Strict ordering • JMS • Geo-replication & Availability • Rich broker features • Event raised by sources • Independent individual messages • Push mechanism • Filtering and routing • Pay as you go • Fan out Push-Pull - Push-Push Cloud Pub/Sub Models
  • 19.
    Task Queue Coordinate simpletasks across compute (workers) • Low cost • Pay as you Go • Few features • Easy to use • Have as many queues as you like
  • 20.
    Cloud Queues Service BusQueues - Storage Queues Service Bus Queues Storage Queues • Queue Semantics • Rich filters • Instantaneous consistency • Strict ordering • JMS • Geo-replication & Availability • Rich broker features • Part of Storage infrastructure • Simple REST-based interface • Pull mechanism • Reliable • Persistence messaging • Only queues
  • 21.
    Big Data Streams Flowdata and telemetry in real-time • Durable re-playable buffer • Designed as a stream • Events to an event hub via AMQP or HTTP • Real-time and batch processing • Feeding Big Data and Analytics • Pre-purchase capacity in terms of throughput units • Offers an Kafka Endpoint • Competitive offering from Amazon Kinesis, Apache Kafka, and Google Pub/Sub
  • 22.
    Brought to youby Wagner Silveira - https://goo.gl/LcjMQB Kafka Endpoint - Demo Consumer Groups Producer Consumer Event Hubs
  • 23.
  • 24.
  • 25.
  • 26.
    VMSS Cloud Services VMs Logic Apps SQLDB Express Route ASA SQL DW Power BI Data Lake Cosmos DB Functions AKS ACS
  • 27.
  • 28.
  • 29.
    License plate OCR •Use the Recognition Services Computer Vision API and its built- in OCR capabilities • The image processing function can make a REST call to send the photo and read the JSON data in return Smart solution
  • 30.
    Storage blob Azure EventGrid Function Cognitive Services Vision API Smart Services - Demo
  • 31.
  • 32.
    Serverless Home Automation- Demo data data data data start/stop data start/stop start/stop Cosmos DB Event Hubs IoT Hub Functions data data notify
  • 33.
    The processing requiredby an application can easily be broken down into a set of independent steps. Pipes and filters
  • 34.
    Microservices containers Request DataProvisioningAPI Gateway Provision Request Message Storage SQL Instance in ElasticPool Key Vault Provisioning Queue Provisioning WebJob Trigger Provision Store Connection strings Provision Container Azure AppService Provision Request Message
  • 35.
    Microservice processing -Demo Request message Container ServiceBus Listener Service Get Blob Edi-file Blob Storage Json-file Blob Storage Storage Explorer message ServiceBus Listener Service Next step!
  • 36.
  • 37.
    Use Azure MessagingServices together
  • 38.
    Data and eventpipeline - Demo
  • 39.
    Summary Choose the rightmessaging service(s) Workloads Cross-platform Costs DevOps and Management Security and compliance
  • 40.
  • 41.
    Thank you! Keep intouch. Call or mail us. Ask us. Happy to help.