GIB2020 - Building Event-Driven Integration Architectures
•Download as PPTX, PDF•
1 like•281 views
This document summarizes a presentation about building event-driven integration solutions. It discusses what events are, different types of events, and the differences between event-driven and event sourcing approaches. It also covers why event-driven integration is useful, different messaging models, the differences between messaging and eventing, communication patterns, CloudEvents, Azure Event Grid, event domains, and choosing between Event Grid, Event Hubs and Service Bus. Examples of using Event Grid with storage events and other services are also provided.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to GIB2020 - Building Event-Driven Integration Architectures
Similar to GIB2020 - Building Event-Driven Integration Architectures (20)
More from Daniel Toomey
More from Daniel Toomey (20)
Recently uploaded
Recently uploaded (20)
GIB2020 - Building Event-Driven Integration Architectures
- 1. 2020 - Brisbane GLOBAL INTEGRATION BOOTCAMP Dan Toomey @daniel2me Building Event-Driven Integration Solutions
- 2. Who am I? • Senior Integration Specialist, Deloitte • Microsoft Azure MVP • MCSE, MCT, MCPD, MCTS BizTalk & Azure • Pluralsight Author • www.mindovermessaging.com • @daniel2me
- 3. What is an Event? http://www.news.com.au
- 4. Types of Events https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Discrete Independent Report state change Actionable Series Time based Context partitioned Report condition Analyzable
- 5. Types of Events https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Discrete Independent Report state change Actionable Series Time based Context partitioned Report condition Analyzable
- 6. Event Driven vs. Event Sourcing https://martinfowler.com/articles/201701-event-driven.html Event Notification Broadcast state changes No expectation from consumer Decoupled logic Event Sourcing Persist state changes Auditability Replay capability
- 7. Event Driven vs. Event Sourcing https://martinfowler.com/articles/201701-event-driven.html Event Notification Broadcast state changes No expectation from consumer Decoupled logic Event Sourcing Persist state changes Auditability Replay capability
- 14. Messaging Models Telephone/Website • Synchronous • Need information immediately
- 15. Messaging Models Telephone/Website • Synchronous • Need information immediately Order • Asynchronous • Takes time to complete a task; CQRS
- 16. Messaging Models Telephone/Website • Synchronous • Need information immediately Order • Asynchronous • Takes time to complete a task; CQRS Subscription • Publish / Subscribe • Decoupling; flexibility; extensibility; scalability
- 17. Messaging Models Telephone/Website • Synchronous • Need information immediately Order • Asynchronous • Takes time to complete a task; CQRS Subscription • Publish / Subscribe • Decoupling; flexibility; extensibility; scalability QUERY COMMAND EVENT
- 18. Messaging vs. Eventing https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Messaging - INTENTS Expectations Conversations Contracts Control Transfer Value Transfer Eventing - FACTS History Context Order Schema
- 19. Messaging vs. Eventing https://www.biztalk360.com/integrate-2018-resources/eventing-serverless-and-the-extensible-enterprise/ Messaging - INTENTS Expectations Conversations Contracts Control Transfer Value Transfer Eventing - FACTS History Context Order Schema
- 22. https://cloudevents.io/ Serverless Working Group https://github.com/cloudevents/spec CNCF CloudEvents
- 23. CNCF CloudEvents { "specversion" : "1.0", "type" : "com.github.pull.create", "source" : "https://github.com/cloudevents/spec/pull", "subject" : "123", "id" : "A234-1234-1234", "time" : "2018-04-05T17:31:00Z", "comexampleextension1" : "value", "comexampleothervalue" : 5, "datacontenttype" : "text/xml", "data" : "<much wow="xml"/>" }
- 24. Azure Event Grid
- 25. Manage all events in one place
- 27. Benefits Manage all events in one place
- 28. Messaging by the numbers… 99.9998%2,200,000,000,000 2 PB/day1.3 PB/day
- 29. What these big numbers mean… 99.9998%2,200,000,000,000 4.4 million failures
- 30. Azure Event Grid Limitations
- 31. aeg-sas-key header aeg-sas-token r={resource}& e={expiration}& s={signature} Microsoft.EventGrid/ EventSubscriptions/Write /subscriptions/{id}/ resourceGroups/{rg-name}/ providers/{provider}/ {res-type}/{resource-name} /subscriptions/{id}/ resourceGroups/{rg-name}/ providers/Microsoft.EventGrid/ topics/{topic-name} • Microsoft.EventGrid/*/read • Microsoft.EventGrid/*/write • Microsoft.EventGrid/*/delete • Microsoft.EventGrid/ eventSubscriptions/ getFullUrl/action • Microsoft.EventGrid/topics/ listKeys/action • Microsoft.EventGrid/topics/ regenerateKey/action + Custom roles Security https://docs.microsoft.com/en-us/azure/event-grid/security-authentication
- 32. [ { "topic": "/subscriptions/{id}/resourceGroups/{rg}/providers/Microsoft.Storage/storageAccounts/{storageacc}", "subject": "/blobServices/default/containers/monitored/blobs/blob.json", "eventType": "Microsoft.Storage.BlobCreated", "eventTime": "2017-09-20T01:12:35.8066557Z", "id": "9ac0de1a-001e-00da-23ad-311d4a06bb2f", "data": { "api": "PutBlob", "clientRequestId": "9ac0de1a-9da0-11e7-9a3b-51cc40a2ca27", "requestId": "9ac0de1a-001e-00da-23ad-311d4a000000", "eTag": "0x9AC0FC4ADA5997D", "contentType": "application/json", "contentLength": 268, "blobType": "BlockBlob", "url": "https://{storageacc}.blob.core.windows.net/monitored/blob.json", "sequencer": "00000000000019200000000000F12B37", "storageDiagnostics": { "batchId": "9ac0de1a-e1a1-44c5-a0a1-a66b2a191fc4" } } } ] Blob Event Sample
- 33. Event Domain
- 34. Event Grid, Event Hubs, Service Bus? https://buildazure.com/2017/09/07/a-tour-of-azure-messaging-services-queues-event-grid-iot-hub-and-more/ Service Bus : Transactions, control, pull Event Hub : Telemetry, streaming Event Grid : Event handling, reactive, push
- 36. Storage Blob Event Grid Webhook Webhook Storage Event Subscription ext = *.jpg Subscription ext = *.pdf
- 38. Microsoft Azure Resource Group Event Grid Azure Function Logic App Virtual machine Storage table Outlook Connector Azure Function Administrator
- 39. Summary Event-driven design allows for cost-effective, real- time, responsive applications Eventing is not the same as messaging Event Grid provides a robust serverless solution for easily building event-driven cloud applications
- 40. @BrisbaneAzureUG meetup.com/Brisbane-Azure-User-Group 2nd Wednesday each month from 5:30pm 400 George Street, Level 28 bit.ly/BAUGYouTubeChannel
Editor's Notes
- No need for context or relationship to other events to be significant; ex: file created
- Meaning is derived from a group of recorded metrics Think of a chart with a time-based X axis
- “This happens when a system sends event messages to notify other systems of a change in its domain. A key element of event notification is that the source system doesn't really care much about the response. Often it doesn't expect any answer at all, or if there is a response that the source does care about, it's indirect. There would be a marked separation between the logic flow that sends the event and any logic flow that responds to some reaction to that event.”
- Can derive the current status of a resource from its event history (ordered sequence of events) Can recover state because events are immutable “The core idea of event sourcing is that whenever we make a change to the state of a system, we record that state change as an event, and we can confidently rebuild the system state by reprocessing the events at any time in the future. The event store becomes the principal source of truth, and the system state is purely derived from it. For programmers, the best example of this is a version-control system. The log of all the commits is the event store and the working copy of the source tree is the system state.”
- APIs are abundant, data is ubiquitous. Think about all the notifications Copernican shift – how to maintain data consistency between all those APIs?
- Three different types of service interaction.
- Simplicity - Point and click to aim events from your Azure resource to any event handler or endpoint. Advanced filtering - Filter on event type or event publish path to make sure event handlers only receive relevant events. Fan-out - Subscribe several endpoints to the same event to send copies of the event to as many places as needed. Reliability - 24-hour retry with exponential backoff to make sure events are delivered. Pay-per-event - Pay only for the amount you use Event Grid. High throughput - Build high-volume workloads on Event Grid with support for millions of events per second. Built-in Events - Get up and running quickly with resource-defined built-in events. Custom Events - Use Event Grid route, filter, and reliably deliver custom events in your app.
- Based on Service Fabric 99.99% availability https://docs.microsoft.com/en-us/azure/event-grid/delivery-and-retry 10 seconds 30 seconds 1 minute 5 minutes 10 minutes 30 minutes 1 hour Event Grid adds a small randomization to all retry intervals.
- Tord Glad Nordahl : “With dynamic scale and consistent performance Azure Event grid lets you focus on your app logic rather than the infrastructure around it."
- Publishing: Key authentication is easiest (just pass a key) SAS (recommended) includes resource, expiry and a signature RBAC for subscription & Mgmt access Webhooks: At the time of event subscription creation/update, Event Grid posts a "SubscriptionValidationEvent" event to the target endpoint. The event contains a header value "Aeg-Event-Type: SubscriptionValidation". The event body has the same schema as other Event Grid events. The event data includes a "validationCode" property with a randomly generated string. For example, "validationCode: acb13…". Only HTTPS is supported
- Contoso Construction Machinery - manufacture tractors, digging equipment, and other heavy machinery. push real-time information to customers regarding equipment maintenance, systems health, and contract updates All of this goes to various endpoints including your app, customer endpoints, and other infrastructure customers had setup. Event Domains allows you to model Contoso Construction Machinery as a single eventing entity. Each of your customers is represented as a Topic within the Domain, handling authentication and authorization using Azure AD Each of your customers can subscribe to their Topic and get the events delivered to them, but AAD and the Event Domain block them from accessing another tenant’s Topic. Provides a single endpoint which you can publish all of your customer events to, and Event Grid will take care of making sure each Topic is only aware of events scoped to its tenant.
- Service Bus: High-value enterprise messaging - Order processing and financial transactions Event Hubs: Big data pipeline - Telemetry and distributed data streaming Event Grid: Reactive programming - React to status changes
- Valeriia Savenko – next Wednesday!