Download as PDF, PPTX
![@crichardson
OO = State + Behavior
creditLimit
creditReservations : Map[OrderId, Money]
Customer
List<Event> processCommand (
Command aCommand) { … }
void applyEvent (Event anEvent) { … }
State
Behavior](https://image.slidesharecdn.com/eventsontheinsideoutsidecorejaxlondon2015-151029160857-lva1-app6891/75/Events-on-the-outside-on-the-inside-and-at-the-core-jaxlondon-67-2048.jpg)
![@crichardson
Event Store API
T is a subclass of Aggregate[T]
Rx Observable = Future++](https://image.slidesharecdn.com/eventsontheinsideoutsidecorejaxlondon2015-151029160857-lva1-app6891/75/Events-on-the-outside-on-the-inside-and-at-the-core-jaxlondon-72-2048.jpg)
Uploaded byChris Richardson
Events on the outside, on the inside and at the core (jaxlondon)
The document discusses the principles and practices of event sourcing in the context of an event-driven architecture, emphasizing how it enables data consistency and integration across microservices. It covers key concepts such as events, messaging systems, webhooks, and the design of event-centric domain models, along with their benefits and drawbacks. Additionally, it highlights the importance of using events as the central feature in modern applications for better reliability and maintainability.
More Related Content
Similar to Events on the outside, on the inside and at the core (jaxlondon)
Events on the outside, on the inside and at the core (jaxlondon)
- 1. @crichardson Events on theoutside, on the inside and at the core Chris Richardson Founder of the original CloudFoundry.com Author of POJOs in Action @crichardson chris@chrisrichardson.net http://plainoldobjects.com http://microservices.io http://eventuate.io
- 2. @crichardson Presentation goal How eventsourcing enables the event-driven enterprise
- 3.
- 4. @crichardson About Chris Consultant andtrainer focusing on microservices (http://www.chrisrichardson.net/)
- 5. @crichardson About Chris Founder ofa startup that is creating a platform that makes it easy for application developers write microservices (http://bit.ly/trialeventuate)
- 6.
- 7. @crichardson Agenda Events on theoutside Events on the inside Events at the core with event sourcing Designing event-centric domain model
- 8.
- 9.
- 10.
- 11. Who wants tobe notified? Humans via Email SMS Mobile push … Applications Applications that implement other parts of the business process, e.g. order fulfillment system Analytics dashboards, monitoring, … Applications that deliver notifications to humans: Twilio, SendGrid, … …
- 12. @crichardson How: Polling forevents HTTP Periodically poll for events AtomPub Based on HTTP Head is constantly changing Tail is immutable and can be efficiently cached High-latency, inefficient
- 13. @crichardson Using WebSockets Browser Web Socket STOMP Client application Service Web Socket STOMP Application SUBSCRIBE MESSAGE MESSAGE Low latency,more efficient, but what about past events?
- 14. @crichardson Webhooks = user-defined HTTPcallback Client Service register(events, callbackUrl) POST callbackUrl POST callbackUrl … https://en.wikipedia.org/wiki/Webhook Low latency, more efficient, but what about past events?
- 15. @crichardson Github webhooks https://developer.github.com/webhooks/ Installed onan organization or repository e.g. POST /repos/:owner/:repo/hooks Available events: push - push to a repository fork - repository is forked pull_request - assigned, unassigned, … push - push to a repository …
- 16. Twilio - Telephonyand SMS as a service REST API Allocate phone numbers Make and receive phone calls Send and receive SMS messages Pay per use: Phone calls - per-minute SMS – per SMS sent or received Phone number – per month Examples OpenVBX is a web-based, open source phone system StubHub – notifies sellers of a pending sale via phone SurveyMonkey – interactive polling Salesforce – SMS-based voting for 19,000 conference attendees
- 17. @crichardson Using Twilio Twilio Your Application TwiMLdoc HTTP GET/ POST REST API Manage resources Send SMS Initiate voice calls Webhooks handle incoming SMS and voice calls Voice SMS Phone number SMS URL +VOICE URL
- 18. @crichardson Integration hubs -Zapier, IFTTT Application abstraction: Triggers - events published by application: polling or Webhooks Action - operation supported by application, e.g. REST API end points
- 19.
- 20.
- 21. @crichardson messaging system Messaging-based IPC SenderRecipientChannel message
- 22.
- 23. @crichardson The event-driven enterprise AppA App BApp X App Y Firewall Message Broker webhooks WebSockets AtomPub App Z App C Msg Broker Client Msg Broker Client Msg Broker Client App D Msg Broker Client Inside the firewallOutside the firewall
- 24. @crichardson Agenda Events on theoutside Events on the inside Events at the core with event sourcing Designing event-centric domain model
- 25. @crichardson Traditional monolithic architecture Simpleto develop test deploy scale Shopping Cart Customers Orders Mobile client REST API Browser Web UI Message Broker Adapter MySQL Adapter RDBMS … Message Broker
- 26. @crichardson But that leads*to monolithic hell For large and/or complex applications…
- 27. @crichardson Today: use amicroservice, polyglot architecture Orders Customers … Shopping UI Mobile Client Browser API Gateway Order management REST API Customer Management REST API …. REST API Order Database (MongoDB) MongoDB Adapter Customer Database (Sharded MySQL) MySQL Adapter
- 28. @crichardson But now wehave distributed data management problems
- 29. @crichardson Example: placing anorder Order Service Customer Service Order Database Customer Database Order #1 Customer #1 No 2PC No ACID NoSQL SQL
- 30. @crichardson Customer management How tomaintain invariants? Order management Order Service placeOrder() Customer Service updateCreditLimit() Customer creditLimit ... has ordersbelongs toOrder total Invariant: sum(open order.total) <= customer.creditLimit ?
- 31. @crichardson Use an event-driven architecture Servicespublish events when something important happens, e.g. state changes Services subscribe to events and update their state Maintain eventual consistency across multiple aggregates (in multiple datastores) Synchronize replicated data
- 32. @crichardson Event-driven application architecture Application ServiceA Service B Message Broker Service C Msg Broker Client Msg Broker Client Msg Broker Client Service D Msg Broker Client
- 33. @crichardson Order Management Order id :4567 total: 343 state = CREATED Customer Management Customer creditLimit : 12000 creditReservations: {} Customer creditLimit : 12000 creditReservations: { 4567 -> 343} Order id : 4567 total: 343 state = OPEN Eventually consistent credit checking Message Bus createOrder() Publishes: Subscribes to: Subscribes to: publishes: OrderCreatedEvent CreditReservedEvent OrderCreatedEvent CreditReservedEvent
- 34. @crichardson Now there aretwo problems to solve….
- 35. @crichardson Problem #1: Howto design eventually consistent business logic? More on that later….
- 36. @crichardson Problem #2: Howto atomically update database and publish an event Order Service Order Database Message Broker insert Order publish OrderCreatedEvent dual write problem ?
- 37. @crichardson Update and publishusing 2PC Guaranteed atomicity BUT Need a distributed transaction manager Database and message broker must support 2PC Impacts reliability Not fashionable 2PC is best avoided
- 38. @crichardson Transaction log tailing How: Readthe database “transaction log” = single source of truth Publish events to message broker LinkedIn databus https://github.com/linkedin/databus Supports Oracle and MySQL Publish as events AWS DynamoDB streams Ordered sequence of creates, updates, deletes made to a DynamoDB table Last 24 hours Subscribe to get changes MongoDB Read the oplog
- 39. Transaction log tailing:benefits and drawbacks Benefits No 2PC No application changes required Guaranteed to be accurate Drawbacks Immature Database specific solutions Low-level DB changes rather business level events = need to reverse engineer domain events
- 40. @crichardson Use database triggers Trackchanges to tables Insert events into an event table Use datastore as a message queue Pull events from event table and write to message broker
- 41. Database triggers: benefits anddrawbacks Benefits No 2PC No application changes required Drawbacks Requires the database to support them Database specific solutions Low-level DB changes rather business level events = need to reverse engineer domain events Error-prone, e.g. missing trigger
- 42. @crichardson Application created events Usedatastore as a message queue Txn #1: Update database: new entity state & event Txn #2: Consume event Txn #3: Mark event as consumed Eventually consistent mechanism (used by eBay) See BASE: An Acid Alternative, http://bit.ly/ebaybase
- 43. Application created events Benefits High-leveldomain events No 2PC Drawbacks Requires changes to the application Only works for SQL and some NoSQL databases Error-prone
- 44. @crichardson Agenda Events on theoutside Events on the inside Events at the core with event sourcing Designing event-centric domain model
- 45. @crichardson Just publish events Application Database MessageBroker update publish X
- 46. @crichardson Event sourcing For eachaggregate (business entity): Identify (state-changing) domain events Define Event classes For example, ShoppingCart: ItemAddedEvent, ItemRemovedEvent, OrderPlacedEvent Order: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped
- 47. @crichardson Persists events NOT currentstate Order status …. Event table 101 ACCEPTED Order table X OrderCreated101 901 … OrderApproved101 902 … OrderShipped101 903 …
- 48. @crichardson Replay events torecreate state Order state OrderCreated(…) OrderAccepted(…) OrderShipped(…) Events Periodically snapshot to avoid loading all events
- 49. @crichardson The present isa fold over history currentState = foldl(applyEvent, initialState, events)
- 50. @crichardson Domain logic =event-driven aggregates Customer AggregateOrder Aggregate Order Created Event Check Credit Command Credit Reserved Event Approve Order Command Create Order Command External request Emits events Event Command
- 51. @crichardson Request handling inan event-sourced application HTTP Handler Event Store pastEvents = findEvents(entityId) Order new() applyEvents(pastEvents) newEvents = processCmd(SomeCmd) saveEvents(newEvents) Microservice A (optimistic locking)
- 52. @crichardson Event Store publishesevents - consumed by other services Event Store Event Subscriber subscribe(EventTypes) publish(event) publish(event) Aggregate CQRS View update() update() Microservice B send notifications …
- 53. Event store =database + message broker Hybrid database and message broker Implementations: Home-grown/DIY geteventstore.com by Greg Young http://eventuate.io (mine) Event Store Save aggregate events Get aggregate events Subscribe to events
- 54. @crichardson Benefits of eventsourcing Solves data consistency issues in a Microservice/NoSQL-based architecture Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,… Eliminates O/R mapping problem (mostly) Reifies state changes: Built-in, reliable audit log, temporal queries Preserved history More easily implement future requirements
- 55. @crichardson Drawbacks of eventsourcing Weird and unfamiliar Events = a historical record of your bad design decisions Handling duplicate events can be tricky Application must handle eventually consistent data Event store only directly supports PK-based lookup => use Command Query Responsibility Segregation (CQRS) to handle queries
- 56. @crichardson Agenda Events on theoutside Events on the inside Events at the core with event sourcing Designing event-centric domain model
- 57. @crichardson Use the familiarbuilding blocks of DDD Entity Value object Services Repositories Aggregates ⟸ essential
- 58. Aggregate design Graph consistingof a root entity and one or more other entities and value objects Each core business entity = Aggregate: e.g. customer, Account, Order, Product, …. Reference other aggregate roots via primary key Often contains partial copy of other aggregates’ data Order OrderLine Item quantity productId productName productPrice customerId Address street city …
- 59. @crichardson Partition the domainmodel into Aggregates Order OrderLine Item quantity … Address street city … Customer Product name price
- 60. @crichardson Transaction = processingone command by one aggregate No opportunity to update multiple aggregates within a transaction Aggregate granularity is important If an update must be atomic (i.e. no compensating transaction) then it must be handled by a single aggregate e.g. scanning boarding pass at security checkpoint or when entering jetway
- 61.
- 62. Designing domain events Recordstate changes for an aggregate Part of the public API of the domain model ProductAddedToCart id : TimeUUID senderId: UUID productId productName productPrice … Required by aggregate Enrichment: Useful for consumers Event metadata
- 63.
- 64. @crichardson Designing commands Created bya service from incoming request Processed by an aggregate Immutable Contains value objects for Validating request Creating event Auditing user activity
- 65.
- 66.
- 67. @crichardson OO = State+ Behavior creditLimit creditReservations : Map[OrderId, Money] Customer List<Event> processCommand ( Command aCommand) { … } void applyEvent (Event anEvent) { … } State Behavior
- 68. @crichardson Aggregate traits Map Commandto Events Apply event returning updated Aggregate Used by Event Store to reconstitute aggregate
- 69.
- 70. @crichardson Customer - commandprocessing
- 71.
- 72. @crichardson Event Store API Tis a subclass of Aggregate[T] Rx Observable = Future++
- 73. @crichardson Creating an order save()concisely specifies: 1.Creates Customer aggregate 2.Processes command 3.Applies events 4.Persists events
- 74. @crichardson Event handling inCustomers 1.Load Customer aggregate 2.Processes command 3.Applies events 4.Persists events Triggers BeanPostProcessor Durable subscription name
- 75. @crichardson Summary Events are acentral to modern applications Events integrate applications Events maintain data consistency in a microservices architecture Build events into the core of your application using event sourcing
- 76.