Download to read offline
Uploaded byBradley Irby
Event Driven Architectures - Net Conf UY 2018
The document discusses event-driven architectures (EDA) as frameworks that manage the production, detection, and consumption of events, enabling more efficient microservices. It highlights the advantages of EDA for both users and developers, including improved response times, better fault tolerance, and easier scaling. Additionally, it contrasts EDA with traditional message-driven architectures, emphasizing the ability to maintain state history and provide a single source of truth for business operations.
More Related Content
![[WSO2Con Asia 2018] Building Reactive Applications Using Event-driven Archite...](https://cdn.slidesharecdn.com/ss_thumbnails/wso2conasia2018-buildingreactiveapplicationsusingevent-drivenarchitecture-180810094630-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Event Driven Architectures - Net Conf UY 2018
Event Driven Architectures - Net Conf UY 2018
- 1. Event Driven Architectures ByPresenterMedia.com Enabling Microservices … Today and Tomorrow Brad@BradIrby.com @BradIrby LinkedIn BradIrby
- 2. • .Net developerand architect for ?? Years • I create new business value from legacy applications. • Slides will be on SlideShare Who is Brad?
- 3. • Tech Targetsays: • An event-driven architecture (EDA) is a framework that orchestrates behavior around the production, detection and consumption of events as well as the responses they evoke. An event is any identifiable occurrence that has significance for system hardware or software. What is an Event Driven Architecture
- 4. MESSAGES • Requests Actionsto be Performed • Short Lived • Full Data Payload • Only Current and Future Messages may be Consumed • Expectation that something will process the message Difference Between Messages and Events
- 5. MESSAGES • Requests Actionsto be Performed • Short Lived • Full Data Payload • Only Current and Future Messages may be Consumed • Expectation that something will process the message • Reflects a completed Action • Medium or Long Lived • Partial Data Payload • Historical Events may be Consumed • No expectation on whether an event has any listeners EVENTS Difference Between Messages and Events
- 6. • State changeis initiated via a short lived message • Results of state change request not tracked • Application components communicate via messages • Expectation that messages will be processed • Messages are destroyed once processed • History is unavailable •What is a Message Driven Architecture
- 7. • Create afteran entity state change • Persist changes to the state of a business entity, such as an Order or Customer • When state changes, new event appended • History is available • Applications can read the series of events •What is an Event Driven Architecture Let’s See an Example Using Both
- 8. Process for ourexamples A second line of text could go here User Updates Order User creates new order in Online store Create and Update Data Create new Order Check Inventory Remove proper number of items from inventory Adjust for each item in the basket Ship merchandise to consumer Notify consumer that their purchase is complete Send Email with tracking number Update their order history UI Create Order Notify User Charge Card Decrement Inventory Charge consumers credit card for purchase Ensure credit available Capture funds for purchase
- 9. • User requestsan entity change • Website makes data changes • Create Order • Check Inventory • Charge Card • Decrement Inventory • Notify User • Changes saved to the DB throughout This is a Typical Non-EDA Architecture
- 10. • User requestsan entity change • Website makes data changes • Create Order • Check Inventory • Charge Card • Decrement Inventory • Notify User • Changes saved to the DB throughout • Your baby has grown up!! This is a Typical Non-EDA Architecture
- 11. • Synchronous bydefault • Must scale entire app • Unhandled Exceptions • may affect all processes • On failure, user must manually resubmit change • Failure results in potentially unknown state • No record of how we got to current entity state What is Wrong with this Architecture?
- 12. • After OrderCreated, OrderCreated event written to Event Store Introduce Event Store
- 13. • After OrderCreated, OrderCreated event written to Event Store • All other state changes also written Events Break the Application Apart
- 14. Events Break theApplication Apart
- 15. Message Queue isIdeal, but Not Today
- 16. • Devs andBusiness must embrace eventual consistency • “Eventual” usually means < 1 second, but not immediate You have to give something to get something •Not a Panacea Eventual Consistency
- 17. • Devs andBusiness must embrace eventual consistency • “Eventual” usually means < 1 second, but not immediate • Immediate feedback not guaranteed • May get email later with bad news You have to give something to get something •Not a Panacea Minor but Measurable User Impact Eventual Consistency
- 18. • Devs andBusiness must embrace eventual consistency • “Eventual” usually means < 1 second, but not immediate • Immediate feedback not guaranteed • May get email later with bad news • Devs must get accustomed to new paradigm You have to give something to get something •Not a Panacea Minor but Measurable User Impact Eventual Consistency New Dev Paradigm
- 19. • Faster userresponse times • Scaling horizontally in response to user demand provides a faster user experience Advantages of Event Driven Architecture Advantages for Users
- 20. • Faster userresponse times • User does not have to resubmit failed requests • Should a back end service fail during processing, the user does not have to resubmit a request since all necessary information is captured in the Event Source Advantages of Event Driven Architecture Advantages for Users
- 21. • Faster userresponse times • User does not have to resubmit failed requests • Request Isolation means more confidence in the system • A service failure on one request does not affect any others Advantages of Event Driven Architecture Advantages for Users
- 22. • Scale whatand when necessary • Only services under pressure need to be scaled • Implicit support of horizontal scaling • Can easily respond to temporary scaling pressures Advantages of Event Driven Architecture Advantages for Production
- 23. • Scale whatand when necessary • Service failures do not affect other requests • If a single instance of a service fails, only the currently executing request fails. Others being processed by other services continue unaffected Advantages of Event Driven Architecture Advantages for Production
- 24. • Scale whatand when necessary • Service failures do not affect other requests • If a single instance of a service fails, only the currently executing request fails. Others being processed by other services continue unaffected Advantages of Event Driven Architecture Advantages for Production
- 25. • Scale whatand when necessary • Service failures do not affect other requests • Better Fault Tolerance • Failure simply requires re-processing of event • Current place in event source is easily stored • Systems can go down and recover minutes or hours later Advantages of Event Driven Architecture Advantages for Production
- 26. • Easier Dev- Async by default • Individual devs and code as if they are single threaded • Front end submits new event and depends on back end services to process it • Enables complex operations submitted by front end, where some processing needs to be synchronous, and others not Advantages of Event Driven Architecture Advantages for Devs
- 27. • Easier Dev- Async by default • Can replay event creation • With all events in storage, we can re-live stressful events with new systems to ensure they can handle the load • We replay events in actual time, or sped up • Recreate bugs using actual production data Advantages of Event Driven Architecture Advantages for Devs
- 28. • Easier Dev- Async by default • Can replay event creation • Test with Production events • Test systems can consume production events in order to further ensure they are prepared for the rigors of a live system Advantages of Event Driven Architecture Advantages for Devs
- 29. • One Sourceof Truth • We have a record of all events that got us to where we are today Advantages of Event Driven Architecture Advantages for Business
- 30. • One Sourceof Truth • Enhanced planning data • We have a record of exactly what happened at any point in time. • We can analyze traffic after the fact to determine how best to mitigate problems in the future Advantages of Event Driven Architecture Advantages for Business
- 31. • One Sourceof Truth • Enhanced planning data • Domain Decoupling means faster responses to a changing market • Adding and releasing features is easier with smaller domains • We can break our systems into smaller domains and microservices more easily and faster Advantages of Event Driven Architecture Advantages for Business
- 32. • One Sourceof Truth • Enhanced planning data • Domain Decoupling means faster responses to a changing market • Better error recovery • Should anything go wrong with a back end system, we have all the information we need to retroactively reconstruct data and get back into operation without significant outage Advantages of Event Driven Architecture Advantages for Business
- 33. • One Sourceof Truth • Enhanced planning data • Domain Decoupling means faster responses to a changing market • Better error recovery • Easier release of special customer restricted features and canary releases • Adding a new event consumer does not affect existing production resource • Enables us to add a specific feature for a specific customer in a very short time and without an outage • New features can “go back in time” to process events that have already occurred Advantages of Event Driven Architecture Advantages for Business
- 34. • 150 PersonDevelopment Team • Small, Focused Teams • 300 Microservices • 1000+ Git Repos • >100 Prod Releases / Day • Supports 10K Requests / Sec Other Experiences - GILT
- 35. • 100% Devdone in APIs • Avg Time from Code Commit to Prod decreased 20% • Identified 41 Data Topics • Supports 70MM Visitors / Month Other Experiences - Target
- 36. • Easy –Amazon Kinesis & Azure Event Hub • Easily turned on and used • Inexpensive • Not as feature rich as Kafka Implementation Options • Hard - Apache Kafka • Self managed, Kubernetes Required • Expensive to host (Opensource) • Flexible with Query Language
- 37. • Easy –Amazon Kinesis & Azure Event Hub • Easily turned on and used • Inexpensive • Not as feature rich as Kafka • Easy – DynamoDB • Auto-create events • Convenient if already using DynamoDB Implementation Options A second line of text here • Hard - Apache Kafka • Self managed, Kubernetes Required • Expensive to host (Opensource) • Flexible with Query Language
- 38. Event Driven Architectures Brad@BradIrby.com@BradIrby LinkedIn BradIrby By PresenterMedia.com Connect with me for a Free Webinar regarding • EDA • CQRS with Mediatr • Migrating from Monolith to Modular
Editor's Notes
- #4 This is the common definition of an event, but it is not quite correct. Before moving on, we need to make sure our terminology is correct or it will breed confusion later in the discussion.
- #5 When discussing “events”, developers often lump two things into the same heading. In Event Driven Architectures, there is a distinction between a Message and an Event. A message is a request for an action to be performed. They usually contain a full data payload consisting of all information necessary to accomplish the task they are requesting. For example, you could have a ChangeMailingAddress message which would contain the full address that should be added to the system, along with the ID of the user the address belongs to, and any other pertinent information.. Messages are short lived because they are usually thrown away once they are processed. There is no need to keep them around because any effects they may have been requesting were either applied or rejected. Note that the message will not know whether its change request was accepted or rejected, so that is an additional reason not to keep the message around. Finally, there is an expectation that something somewhere will process the message. The message creator doesn’t know what that something is, but it is assumed the message will be consumed and processed, even if that processing means just throwing it away.
- #6 If Messages are the request for some action to be performed, then Messages are the other side of the coin and represent a change that has been made. In other words, Events are created due to the processing of a message. Events typically contain a partial payload – just the data that has changed. They are medium or long lived so that they can be consumed by other processes at their leisure. This is an important difference between messages and events because it means events may be processed more than once. An event is a fact. A change was made. Whether that change should result in further changes being made can be decided by other systems hours or days later, as business rules dictate. Finally note there is no expectation that an event will of concern to anything else. It’s entirely possible that an event will be created that no other system is listening for, and that’s OK.
- #7 With that in mind, let’s look at the difference between a Message Driven Architecture and an Event Driven Architecture. A message driven architecture is basically what we’ve all been building for years. In a webforms project when a user changes a data field and presses save, the postback is basically a message being sent to the server indicating the user wants to change the data. The server would process that message by saving the data to the DB, then return a response back to the browser, effectively throwing away the message. If we take that idea and throw in a Message Queue server, then we have a more robust version of message driven architecture, but at its core it is the same thing.
- #9 In the discussion to come this is the process to keep in mind. We will be using this in our examples.
- #10 This is how requests are processed currently. A request for a change in a piece of data comes in and flows through the layers to finally update the DB. This model works, and has worked for a long time, but it has shortcomings.
- #11 We are picking up the process when your baby is now all grown up. You have more than a handful of members and you need to be able to scale to meet demand. This is both a happy and sad place in the life of a parent becuase they need to let go of the old ways and let their child advance.
- #12 This process is typically synchronous so the UI is blocked waiting on the process to complete. It’s possible to make this async, but that is not the default development paradigm. In other words, the developer has to go out of their way to make this asynchrounous. In order to scale we must scale the entire app, paying the price for scaling features that don’t have the throughput necessary. Also, this type of scaling takes time to take affect and can lose data. Scaling by adding a new UI and associated business logic requires waiting for the existing requests to be handled by the current app, then subsequent requests are routed to the new server, thus losing session. If something goes wrong in the Inventory processing, it has the potential to bring the entire application down. Also if something fails during the pipeline of processing, the default action is for the entire stream of events to be rolled back (if possible). If we’ve planned ahead, on failure we show the user an error message and they must retry their transaction. If we haven’t planned well for this error, the user is left in a state where they don’t know whether the change went through or not.
- #13 Now let’s make a simple change to this architecture and introduce an event store. Each time an order is created, we write an “OrderCreated” event to the event store, then continue processing. Note there is a mistake in the graphics for this – the write to the Event Store should be coming from the Domain layer because it should be a distributed transaction with the DB write. Keep this in mind throughout the following slides.
- #14 What we’ve really done by writing events to an event store is to open the door for breaking apart the system into a potential microservice. The whole process no longer has to be synchronous. The microservice can wait until it sees an OrderCreated event come in, then process it. Now we can respond to the user, who is still waiting, much more quickly and let them get on with their day. Once we’ve don’t that, why stop there?
- #15 If we write all of our events to the event store, now all of these different domains can be asynchronous microservices. They are all triggered according to different events, and all can scale independently depending on workload. What do we do if the original box is the one that’s our hot spot? We could scale this by adding more web servers to the front end, or we can introduce a message queue.
- #16 If we really want to allow unlimited scaling, we would introduce a message queue here and let the order creation piece also be a microservice, though it would be one driven by messages not by events. This introduces some complication because the interaction with the user is synchronous by definition. They are sitting there waiting for us to repaint the screen. There are ways to handle this, but it’s too much to present this today so I’m going to gloss over this part.
- #17 This looks great and we can scale infinitely, but there are some drawbacks. Having asynchrounous microservices forces the business to embrace eventual consistency. Often I get push-back on this because business people are trained that “eventual consistency” means next day. If all goes well, it is usually less than a second or 2, but still it is not immediate and the business needs to understand that.
- #18 What that means to the user is that they may get a success message immediately, followed by an email saying something failed. Say we could not charge their card for some reason. They would get an initial “order approved” message in real time, then a few minutes later an email informing them they need to return to the site to change their credit card.
- #19 Also the developers must get used to a new programming paradigm. They will not have full control of an order anymore. They will only be able to control one small piece at a time, then let the next process take it from there. This is actually a good thing since the developers will be able to focus on one thing at a time, but it is a paradigm change they must get used to.
- #20 Making this change to EDA is a boon for the user. The majority of the time they will receive a success message faster than they did before, and everything will progress as planned. Occasionally a consumer will receive a follow up notification telling them they need to do something to fix a problem, but the average user experience will be better than it was before due to the added ability to scale all systems.
- #21 If a request fails during processing, the user will not have to resubmit the entire request again. Assuming it does not fail in the order creation stage, all necessary information is captured along the way to enable any service to fail and then recover. In the original design if a processes crashed the server, the entire request would have been lost and the user would have to resubmit. In this distributed EDA architecture, we just reboot a server and everything processes as normal.
- #22 Continuing with the idea of server failure, by hiding any failures it gives the users more confidence in the system. Let’s say we have just as many system crashes after the EDA enhancement as before. Since many of those will happen outside the view of the user, they will never know. Therefore, the perceived reliability of the system improves.
- #23 With microservices you can scale the hotspots as necessary. This results in cost savings because you’re not scaling all funtcions in order to improve the performance of one. It also enables automatic short term scaling in response to temporary pressures.
- #24 The failure of a single service will not affect any of the other services. The others continue working as designed, processing events as they are created. In the original design, with all processes in the same address space, the failure of one would bring down all the others as well.
- #25 In fact, all back end services could crash and the user would never know since the only one they are interacting with is the UI.
- #27 There are team advantages here as well. In the original monolithic design, the default coding paradigm was synchronous. The developer had to go out of their way to build something async. In this design, the default is async and the developer must go out of their way to make it synchronous. This is what we want in for our development teams, for the proper strategy to the be the easiest to implement.