Building Event-Driven Integration Architectures with Azure Event Grid (GIB2019)
•Download as PPTX, PDF•
4 likes•807 views
Presentation delivered by Dan Toomey at the Global Integration Bootcamp in Melbourne, 30 March 2019
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Building Event-Driven Integration Architectures with Azure Event Grid (GIB2019)
Similar to Building Event-Driven Integration Architectures with Azure Event Grid (GIB2019) (20)
More from Daniel Toomey
More from Daniel Toomey (20)
Recently uploaded
Recently uploaded (20)
Building Event-Driven Integration Architectures with Azure Event Grid (GIB2019)
- 1. 2019 - Melbourne Global Integration Bootcamp Dan Toomey @daniel2me https://www.mindovermessaging.com Building Event-Driven Integration Architectures with Azure Event Grid
- 2. Who am I? • Senior Integration Specialist, Deloitte • Microsoft Azure MVP • MCSE, MCT, MCPD, MCTS BizTalk & Azure • Pluralsight Author • www.mindovermessaging.com • @daniel2me
- 3. What is an Event? http://www.news.com.au
- 4. Types of Events https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Discrete Independent Report state change Actionable Series Time based Context partitioned Report condition Analyzable
- 5. Types of Events https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Discrete Independent Report state change Actionable Series Time based Context partitioned Report condition Analyzable
- 6. Event Driven vs. Event Sourcing https://martinfowler.com/articles/201701-event-driven.html Event Notification Broadcast state changes No expectation from consumer Decoupled logic Event Sourcing Persist state changes Auditability Replay capability
- 7. Event Driven vs. Event Sourcing https://martinfowler.com/articles/201701-event-driven.html Event Notification Broadcast state changes No expectation from consumer Decoupled logic Event Sourcing Persist state changes Auditability Replay capability
- 13. Messaging Models Telephone/Website • Synchronous • Need information immediately
- 14. Messaging Models Telephone/Website • Synchronous • Need information immediately Order • Asynchronous • Takes time to complete a task; CQRS
- 15. Messaging Models Telephone/Website • Synchronous • Need information immediately Order • Asynchronous • Takes time to complete a task; CQRS Subscription • Publish / Subscribe • Decoupling; flexibility; extensibility; scalability
- 16. Messaging vs. Eventing https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Messaging - INTENTS Expectations Conversations Contracts Control Transfer Value Transfer Eventing - FACTS History Context Order Schema
- 17. Messaging vs. Eventing https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Messaging - INTENTS Expectations Conversations Contracts Control Transfer Value Transfer Eventing - FACTS History Context Order Schema
- 20. https://cloudevents.io/ Serverless Working Group https://github.com/cloudevents/spec CNCF CloudEvents
- 21. CNCF CloudEvents { "cloudEventsVersion" : "0.1", "eventType" : "com.example.someevent", "source" : "/mycontext", "eventID" : "A234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", "extensions" : { "comExampleExtension" : "value" }, "contentType" : "text/xml", "data" : "<much wow="xml"/>" }
- 22. Example: Logging Event Publication
- 23. Example: Logging Event Publication (Series)
- 25. Azure Event Hubs Image from https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about
- 26. Azure Event Grid
- 27. Benefits Manage all events in one place
- 29. Messaging by the numbers… 99.9998%2,200,000,000,000 2 PB/day1.3 PB/day
- 30. What these big numbers mean… 99.9998%2,200,000,000,000 4.4 million failures
- 31. aeg-sas-key header aeg-sas-token r={resource}& e={expiration}& s={signature} Microsoft.EventGrid/ EventSubscriptions/Write /subscriptions/{id}/ resourceGroups/{rg-name}/ providers/{provider}/ {res-type}/{resource-name} /subscriptions/{id}/ resourceGroups/{rg-name}/ providers/Microsoft.EventGrid/ topics/{topic-name} • Microsoft.EventGrid/*/read • Microsoft.EventGrid/*/write • Microsoft.EventGrid/*/delete • Microsoft.EventGrid/ eventSubscriptions/ getFullUrl/action • Microsoft.EventGrid/topics/ listKeys/action • Microsoft.EventGrid/topics/ regenerateKey/action + Custom roles Security https://docs.microsoft.com/en-us/azure/event-grid/security-authentication
- 32. Blob Event Sample [ { "topic": "/subscriptions/{id}/resourceGroups/{rg}/providers/Microsoft.Storage/storageAccounts/{storageacc}", "subject": "/blobServices/default/containers/monitored/blobs/blob.json", "eventType": "Microsoft.Storage.BlobCreated", "eventTime": "2017-09-20T01:12:35.8066557Z", "id": "9ac0de1a-001e-00da-23ad-311d4a06bb2f", "data": { "api": "PutBlob", "clientRequestId": "9ac0de1a-9da0-11e7-9a3b-51cc40a2ca27", "requestId": "9ac0de1a-001e-00da-23ad-311d4a000000", "eTag": "0x9AC0FC4ADA5997D", "contentType": "application/json", "contentLength": 268, "blobType": "BlockBlob", "url": "https://{storageacc}.blob.core.windows.net/monitored/blob.json", "sequencer": "00000000000019200000000000F12B37", "storageDiagnostics": { "batchId": "9ac0de1a-e1a1-44c5-a0a1-a66b2a191fc4" } } } ]
- 33. Custom Event Sample { "topic": "/subscriptions/{id}/resourcegroups/{rg}/providers/microsoft.eventgrid/topics/{topic}", "id": " 9ac0de1a-83f2-4eb7-ab1f-e2e8d2ef1728", "eventType": "CryptoCurrencyPriceObtained", "subject": "cryptoCurrency/price/BTC", "eventTime": "2017-09-19T11:37:01.1513463+00:00", "data": { "AUD": 4991.89 } } custom data
- 34. Event Domain
- 35. Product Comparison Kafka Kinesis Solace Event Hubs Event Grid Cloud / On-Prem Both Cloud Only Both Cloud Only Cloud Only Managed Service Auto-Scaling Replay Event Sourcing * Integrations Kafka AWS services Open Source connectors available Azure services, Kafka Azure services, anything via HTTP/Webhooks Protocols Kafka HTTP HTTP, AMQP,MQTT, JMS, Web Sockets, Native HTTP, AMQP, Kafka HTTP Open Source Closed source, but open protocol Max Message Size No Limit 1MB No Limit 256K 64K *via archiving
- 36. Event Grid, Event Hubs, Service Bus? https://buildazure.com/2017/09/07/a-tour-of-azure-messaging-services-queues-event-grid-iot-hub-and-more/ Service Bus : Transactions, control, pull Event Hub : Telemetry, streaming Event Grid : Event handling, reactive, push
- 38. Storage Blob Event Grid Webhook Webhook Storage Event Subscription ext = *.jpg Subscription ext = *.pdf
- 40. Microsoft Azure Resource Group Event Grid Azure Function Logic App Virtual machine Storage table Outlook Connector Azure Function Administrator
Editor's Notes
- Kafka: Not serverless Kinesis: Streaming; Optimised for Big Data processing Solace: Event distribution & messaging middleware (both cloud & on-prem)
- Simplicity - Point and click to aim events from your Azure resource to any event handler or endpoint. Advanced filtering - Filter on event type or event publish path to make sure event handlers only receive relevant events. Fan-out - Subscribe several endpoints to the same event to send copies of the event to as many places as needed. Reliability - 24-hour retry with exponential backoff to make sure events are delivered. Pay-per-event - Pay only for the amount you use Event Grid. High throughput - Build high-volume workloads on Event Grid with support for millions of events per second. Built-in Events - Get up and running quickly with resource-defined built-in events. Custom Events - Use Event Grid route, filter, and reliably deliver custom events in your app.
- Tord Glad Nordahl : “With dynamic scale and consistent performance Azure Event grid lets you focus on your app logic rather than the infrastructure around it."
- Based on Service Fabric 99.99% availability https://docs.microsoft.com/en-us/azure/event-grid/delivery-and-retry 10 seconds 30 seconds 1 minute 5 minutes 10 minutes 30 minutes 1 hour Event Grid adds a small randomization to all retry intervals.
- Publishing: Key authentication is easiest (just pass a key) SAS (recommended) includes resource, expiry and a signature RBAC for subscription & Mgmt access Webhooks: At the time of event subscription creation/update, Event Grid posts a "SubscriptionValidationEvent" event to the target endpoint. The event contains a header value "Aeg-Event-Type: SubscriptionValidation". The event body has the same schema as other Event Grid events. The event data includes a "validationCode" property with a randomly generated string. For example, "validationCode: acb13…". Only HTTPS is supported
- Contoso Construction Machinery - manufacture tractors, digging equipment, and other heavy machinery. push real-time information to customers regarding equipment maintenance, systems health, and contract updates All of this goes to various endpoints including your app, customer endpoints, and other infrastructure customers had setup. Event Domains allows you to model Contoso Construction Machinery as a single eventing entity. Each of your customers is represented as a Topic within the Domain, handling authentication and authorization using Azure AD Each of your customers can subscribe to their Topic and get the events delivered to them, but AAD and the Event Domain block them from accessing another tenant’s Topic. Provides a single endpoint which you can publish all of your customer events to, and Event Grid will take care of making sure each Topic is only aware of events scoped to its tenant.
- Service Bus: High-value enterprise messaging - Order processing and financial transactions Event Hubs: Big data pipeline - Telemetry and distributed data streaming Event Grid: Reactive programming - React to status changes