Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Event Grid - quiet event to revolutionize Azure and more

103 views

Published on

Azure Event Grid service overview

Published in: Technology
  • Be the first to comment

Event Grid - quiet event to revolutionize Azure and more

  1. 1. Event Grid - quiet event to revolutionize Azure and more Sean Feldman
  2. 2. So many choices, so many options
  3. 3. Storage Queues (a.k.a. Simple Queues)
  4. 4. Service Bus
  5. 5. Event Hubs
  6. 6. Event Grid
  7. 7. Receiving Events Something happens Continuously running process
  8. 8. There’s a Theme… •Event Driven •Reactive •Pub/Sub Model
  9. 9. What is Event Grid For?
  10. 10. Why Microsoft built EventGrid? • Simplicity • Fan-out with high throughput • Pay-per-event with Push model • Built-in and Custom events AWS Sample Events: https://docs.aws.amazon.com/lambda/latest/dg/eventsources.html
  11. 11. Event Grid Concepts 1. Events: what happened 2. Event Publishers: where it took place 3. Topics: where publishers send events 4. Event Subscriptions: how you receive events 5. Event Handlers: the app or service reacting to the event 6. Security:
  12. 12. Azure Functions Logic Apps Azure Automation WebHooks Event Hubs Storage Queues IoT Hub Blob Storage Resource Groups Azure Subscriptions Event Hubs Custom Topics Storage (GPv2) Container Registry Service Bus (Premium)
  13. 13. What is an Event?
  14. 14. Resource model: extension resource • ARM calls are made to a parent resource • ARM reroutes all Event Grid calls to the Event Grid Resource Provider Storage Resource Provider Queue Service Table Service File Service Event Grid Resource Provider Blob Service Event Grid Resource Extension ARM PUT /subscriptions/mysubscription/resourceGroups/myresourcegroup/providers/Microsoft.Storage/containers/myblobcontai ner/providers/Microsoft.EventGrid/eventSubscriptions/mystoragesubscription?api-version={2017-04-14} { "properties": { “destination“: { "endpointType": “WebHook", “properties”: { "endpointURL": "https://eghttpendpoint1.azurewebsites.net/api/SubscriptionTest?code=abc123"}}, “filter ": { “beginsWith": "blobContainer1 ", "endsWith ": "*.jpg", "eventTypes": [ "eventType1", "eventType2", “eventType3”] } } }
  15. 15. What an Event Subscription looks like { "properties": { "destination": { "endpointType": "webhook", "properties": { "endpointUrl": "https://dogfoodtesting.azurewebsites.net/api/HttpTriggerCSharp1? code=VXbGWce53l48Mt8wuotr0GPmyJ/nDT4hgdFj9DpBiRt38qqnnm5OFg==" } }, "filter": { "includedEventTypes": [ "blobCreated", "blobDeleted" ], "subjectBeginsWith": “/blobServices/default/containers/mycontainer/log", "subjectEndsWith": ".jpg", "subjectIsCaseSensitive": "true" } } }
  16. 16. Advanced Filters
  17. 17. Event Domains
  18. 18. Topics Subscriptions Filters Haven’t we seen this before?
  19. 19. Why is Event Grid different? • Cloud Native by design • Which makes it serverless friendly • Engineered for reliability and scale • Supports CNCF Cloud Events v0.1
  20. 20. Cloud native by design • Always available (99.99% SLA) • Near real-time event delivery (<1s e2e) • At least once delivery • Dynamic scale • 10,000,000 events per second per region • 100,000,000 subscriptions per region • Platform agnostic (WebHook) • Language agnostic (HTTP protocol)
  21. 21. Serverless friendly • Ease of setup from the Portal • Azure CLI • Powershell
  22. 22. Engineered for scale and reliability • Cascading log architecture keeps hot path clear Cascading retry logs Replicated Log 10 sec 1 min 1 hr Event Grid Cluster
  23. 23. Engineered for scale and reliability • Cascading log architecture keeps hot path clear • Each clusters has all subscriptions • Nodes can be added to or removed from cluster Event Grid Cluster
  24. 24. Engineered for scale and reliability • Cascading log architecture keeps hot path clear • Each clusters has all subs • Nodes can be added to or removed from cluster • Clusters can be added 1 min Event Grid Cluster 1 1 min Event Grid Cluster 2
  25. 25. US WestUS East Engineered for scale and reliability • Cascading log architecture keeps hot path clear • Each clusters has all subs • Nodes can be added to or removed from cluster • Clusters can be added • Traffic spans regions Event Grid Cluster R1C1 Event Grid Cluster R1C2 Event Grid Cluster R2C1
  26. 26. Engineered for scale and reliability • Retry intervals • 10 seconds • 30 seconds • 1 minute • 5 minutes • 10 minutes • 30 minutes • 1 hour • Once an hour up to 24 hours • Defaults: 30 delivery attempts / 24 hours
  27. 27. Engineered for scale and reliability • Dead-lettering • Requires Storage account + container • Dead-lettered events stored as blobs
  28. 28. Azure Functions Logic Apps Azure Automation WebHooks Event Hubs Storage Queues IoT Hub Blob Storage Resource Groups Azure Subscriptions Event Hubs Custom Topics Storage (GPv2) Container Registry Service Bus (Prem)
  29. 29. What is CNCF CloudEvents? • Event Protocol Suite developed in CNCF Serverless WG • Common metadata attributes for events • Flexibility to innovate on event semantics • Simple abstract type system mappable to different encodings • Transport options • HTTP(S) 1.1 Webhooks, also HTTP/2 (v0.1) • MQTT 3.1.1 and 5.0 (draft) • AMQP 1.0 (draft) • Encoding options • JSON (v0.1, required for all implementations) • Extensible for binary encodings: Avro, MessagePack, AMQP, etc.
  30. 30. CloudEvent Schema v0.1
  31. 31. Ubiquitous • Today there are 10+ publishers • By year end most Azure services will be publishers • Then most Microsoft services • Expect industry to embrace Cloud Events • Grid will be coming to IoT Edge • And beyond…
  32. 32. How Event Grid is sold • Publish is an operation • Delivery attempt for each destination is an operation • Advanced matching (filtering) is an operation
  33. 33. How Event Grid composes with Queues and Streams • Other messaging services can be publishers or subscribers to Event Grid • Sometimes you want WebHook • Sometimes Queue • Others Stream • Why: at high scale a queue or log can work better • Grid will give you all of them sub1 sub2 mytopic Storage queue Event Hubs
  34. 34. Security and Authentication • Validation Handshake (WebHook event delivery) • Event of type Microsoft.EventGrid.SubscriptionValidationEvent • With validation data • Prove • Echo back {validationCode: “value”} • Send GET to validationURL
  35. 35. Security and Authentication • Topic publishing • “aeg-sas-key” header • “aeg-sas-token” header
  36. 36. Remember EVENT DRIVEN PUB/SUB NOT ENTERPRISE MESSAGING SCALE THROUGHPUT DEAD-LETTERING RETRY CNCF CLOUD EVENT CROSS-CLOUD
  37. 37. Segment Simple Queuing Events & PubSub Big Data Streaming Enterprise Messaging Product Storage Queues Event Grid Event Hubs Service Bus What do you care about • Communication within an app • Simple and easy to use • Individual message • Queue semantics / polling buffer • Pay as you go • Communication between apps / orgs • Individual message • Push semantics • Filtering and routing • Pay as you go • Fan out • Many messages in a Stream (think in MBs) • Ease of use and operation • Low cost • Fan in • Strict ordering • Works with other tools (maybe Kafka?) • Instantaneous consistency • Strict ordering • Interoperability (AMQP?) • Security & Non-repudiation • Geo-Replication & Availability • Rich features achieved with compute (de-dupe, scheduling, etc.) What are you willing to sacrifice to get it • Ordering of messaging • Instantaneous consistency • Ordering of messaging • Instantaneous consistency • Server side cursor • Only Once • Reach features achieved with compute • Cost • Simplicity EnterpriseBig DataServerless Segmentation of the cloud messaging market
  38. 38. THANK YOU Q&A

×