Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Exploring Lightweight   Event SourcingErik Rozendaal <erozendaal@zilverline.com>              @erikrozendaal
Goals• Understand event sourcing• Show why Scala is well-suited as  implementation language              Exploring lightwe...
Event Sourcing• Documented by Martin Fowler at http://  martinfowler.com/eaaDev/  EventSourcing.html (2005)• Made practica...
•   Domain-Driven Design An approach to software development that suggests    that (1) For most software projects, the pri...
Durability Exploring lightweight event sourcing   5
Durability• Most applications require durability of data               Exploring lightweight event sourcing   5
Durability• Most applications require durability of data• UPDATE invoice WHERE id = 1234     SET total_amount = 230       ...
Durability• Most applications require durability of data• UPDATE invoice WHERE id = 1234     SET total_amount = 230  • Wha...
Durability• Most applications require durability of data• UPDATE invoice WHERE id = 1234     SET total_amount = 230  • Wha...
ORMs• Query and persist current state in DB• Tightly couples domain and data model• Leaky abstraction• Still “lossy” and t...
Behavior?Exploring lightweight event sourcing   7
Behavior?Status:"Draft"          Total:Invoice          0.00                   Exploring lightweight event sourcing   7
Behavior?Status:                  Status:          Recipient:"Draft"                  "Draft"            "Erik"          T...
Behavior?Status:                  Status:          Recipient:        Status:      Recipient:"Draft"                  "Draf...
It’s just data mutationStatus:                  Status:          Recipient:        Status:      Recipient:"Draft"         ...
It’s just data mutation             Status:          Recipient:             "Draft"            "Erik"                     ...
ORM implementation“Inexperienced programmers love magic becauseit saves their time. Experienced programmers hatemagic beca...
Event Sourcing• All state changes are explicitly captured  using domain events• Events represent the outcome of behavior• ...
Domain Behavior    Exploring lightweight event sourcing   11
Domain Behavior createDraft Invoice  Created                    Exploring lightweight event sourcing   11
Domain Behavior createDraft Invoice  Created                apply                        Status:                        "D...
Domain Behavior createDraft Invoice                          Invoice Recipient  Created                                  C...
Domain Behavior createDraft Invoice                          Invoice Recipient  Created                                  C...
Domain Behavior createDraft Invoice                          Invoice Recipient                             Invoice Item  C...
Domain Behavior createDraft Invoice                          Invoice Recipient                             Invoice Item  C...
Only the events are                 stored on diskDraft Invoice              Invoice Recipient                         Inv...
Reloading from history       Exploring lightweight event sourcing   13
Reloading from historyDraft Invoice  Created                apply                        Status:                        "D...
Reloading from historyDraft Invoice                     Invoice Recipient  Created                             Changed    ...
Reloading from historyDraft Invoice                     Invoice Recipient                         Invoice Item  Created   ...
Reloading from historyDraft Invoice                     Invoice Recipient                         Invoice Item  Created   ...
Aggregates• A cluster of associated objects that are  treated as a unit for the purpose of data  changes• Each aggregate h...
Aggregates       Status:          Recipient:       "Draft"            "Erik"                          Total:       Invoice...
Aggregates       Status:           Recipient:       "Draft"             "Erik"                            Total:       Inv...
Event Store• Stores sequence of events per aggregate• Dispatches events when successfully  committed• Replays events on st...
Event Store• No good for queries!• Use separate views or reports that use the  domain events to capture answers to  querie...
Domain Codesealed trait InvoiceEventcase class InvoiceCreated() extends InvoiceEventcase class InvoiceRecipientChanged(rec...
Domain Codesealed trait InvoiceEventcase class InvoiceCreated() extends InvoiceEventcase class InvoiceRecipientChanged(rec...
Domain Codecase class DraftInvoice(  recipient: Option[String] = None,  nextItemId: Int = 1,  items: Map[Int, InvoiceItem]...
Domain Codecase class DraftInvoice(  recipient: Option[String] = None,  nextItemId: Int = 1,                              ...
Domain Codecase class DraftInvoice(  recipient: Option[String] = None,  nextItemId: Int = 1,                              ...
Domain Codecase class DraftInvoice(  recipient: Option[String] = None,  nextItemId: Int = 1,  items: Map[Int, InvoiceItem]...
Domain Codecase class DraftInvoice(  recipient: Option[String] = None,  nextItemId: Int = 1,  items: Map[Int, InvoiceItem]...
Minimal Infrastructure• Store only the domain events, replay these  on application startup to reconstruct  current state• ...
Simple functionality        example• “CRUD”: no domain logic, so no aggregate• So we’ll persist events directly from the U...
Aggregate Example     Exploring lightweight event sourcing   24
Implementation            Limitations•   Serializing (“big lock”) event store    •   ~ 50 transactions per second•   Numbe...
But ready to scale•   Advanced event store implementations support    SQL, MongoDB, Amazon SimpleDB, etc.•   Use persisted...
Conclusion• Event Sourcing captures full historical  information• Fully encapsulated domain code highly  decoupled from pe...
Thanks!
References•   Example code https://github.com/erikrozendaal/scala-event-sourcing-example•   CQRS http://cqrsinfo.com/•   G...
Upcoming SlideShare
Loading in …5
×

Exloring Lightweight Event Sourcing - Scala Days 2011

6,886 views

Published on

Slides of my talk on using Scala to implement event sourcing given at Scala Days 2011.

Published in: Technology
  • Be the first to comment

Exloring Lightweight Event Sourcing - Scala Days 2011

  1. 1. Exploring Lightweight Event SourcingErik Rozendaal <erozendaal@zilverline.com> @erikrozendaal
  2. 2. Goals• Understand event sourcing• Show why Scala is well-suited as implementation language Exploring lightweight event sourcing 2
  3. 3. Event Sourcing• Documented by Martin Fowler at http:// martinfowler.com/eaaDev/ EventSourcing.html (2005)• Made practical by Greg Young (QCon 2008)• Applied to complex, high volume systems (financial trading)• But what about small systems? Exploring lightweight event sourcing 3
  4. 4. • Domain-Driven Design An approach to software development that suggests that (1) For most software projects, the primary focus should be on the domain and domain logic; and (2) Complex domain designs should be based on a model.• Domain Expert A member of a software project whose field is the domain of the application, rather than software development. Not just any user of the software, the domain expert has deep knowledge of the subject.• Ubiquitous Language A language structured around the domain model and used by all team members to connect all the activities of the team with the software. Exploring lightweight event sourcing 4
  5. 5. Durability Exploring lightweight event sourcing 5
  6. 6. Durability• Most applications require durability of data Exploring lightweight event sourcing 5
  7. 7. Durability• Most applications require durability of data• UPDATE invoice WHERE id = 1234 SET total_amount = 230 Exploring lightweight event sourcing 5
  8. 8. Durability• Most applications require durability of data• UPDATE invoice WHERE id = 1234 SET total_amount = 230 • What happened to previous order amount? Exploring lightweight event sourcing 5
  9. 9. Durability• Most applications require durability of data• UPDATE invoice WHERE id = 1234 SET total_amount = 230 • What happened to previous order amount? • What was the reason for the change? Exploring lightweight event sourcing 5
  10. 10. ORMs• Query and persist current state in DB• Tightly couples domain and data model• Leaky abstraction• Still “lossy” and the intent of the user is not captured Exploring lightweight event sourcing 6
  11. 11. Behavior?Exploring lightweight event sourcing 7
  12. 12. Behavior?Status:"Draft" Total:Invoice 0.00 Exploring lightweight event sourcing 7
  13. 13. Behavior?Status: Status: Recipient:"Draft" "Draft" "Erik" Total: Total:Invoice Invoice 0.00 0.00 Exploring lightweight event sourcing 7
  14. 14. Behavior?Status: Status: Recipient: Status: Recipient:"Draft" "Draft" "Erik" "Draft" "Erik" Total: Total: Total:Invoice Invoice Invoice 0.00 0.00 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 7
  15. 15. It’s just data mutationStatus: Status: Recipient: Status: Recipient:"Draft" "Draft" "Erik" "Draft" "Erik" Total: Total: Total:Invoice Invoice Invoice 0.00 0.00 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 8
  16. 16. It’s just data mutation Status: Recipient: "Draft" "Erik" Total: Invoice 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 8
  17. 17. ORM implementation“Inexperienced programmers love magic becauseit saves their time. Experienced programmers hatemagic because it wastes their time.”– @natpryce Exploring lightweight event sourcing 9
  18. 18. Event Sourcing• All state changes are explicitly captured using domain events• Events represent the outcome of behavior• Capture the intent of the user and the related data• The domain expert understands and can explain the meaning of the domain events Exploring lightweight event sourcing 10
  19. 19. Domain Behavior Exploring lightweight event sourcing 11
  20. 20. Domain Behavior createDraft Invoice Created Exploring lightweight event sourcing 11
  21. 21. Domain Behavior createDraft Invoice Created apply Status: "Draft" Total: Invoice 0.00 Exploring lightweight event sourcing 11
  22. 22. Domain Behavior createDraft Invoice Invoice Recipient Created Changed Recipient: "Erik" apply create Status: "Draft" Total: Invoice 0.00 Exploring lightweight event sourcing 11
  23. 23. Domain Behavior createDraft Invoice Invoice Recipient Created Changed Recipient: "Erik" apply create apply Status: Status: Recipient: "Draft" "Draft" "Erik" Total: Total: Invoice Invoice 0.00 0.00 Exploring lightweight event sourcing 11
  24. 24. Domain Behavior createDraft Invoice Invoice Recipient Invoice Item Created Changed Added Recipient: "Erik" Item: "Food" Item amount: 9.95 Total amount: 9.95 apply create apply create Status: Status: Recipient: "Draft" "Draft" "Erik" Total: Total: Invoice Invoice 0.00 0.00 Exploring lightweight event sourcing 11
  25. 25. Domain Behavior createDraft Invoice Invoice Recipient Invoice Item Created Changed Added Recipient: "Erik" Item: "Food" Item amount: 9.95 Total amount: 9.95 apply create apply create apply Status: Status: Recipient: Status: Recipient: "Draft" "Draft" "Erik" "Draft" "Erik" Total: Total: Total: Invoice Invoice Invoice 0.00 0.00 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 11
  26. 26. Only the events are stored on diskDraft Invoice Invoice Recipient Invoice Item Created Changed Added Recipient: "Erik" Item: "Food" Item amount: 9.95 Total amount: 9.95 Status: Status: Recipient: Status: Recipient: "Draft" "Draft" "Erik" "Draft" "Erik" Total: Total: Total: Invoice Invoice Invoice 0.00 0.00 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 12
  27. 27. Reloading from history Exploring lightweight event sourcing 13
  28. 28. Reloading from historyDraft Invoice Created apply Status: "Draft" Total: Invoice 0.00 Exploring lightweight event sourcing 13
  29. 29. Reloading from historyDraft Invoice Invoice Recipient Created Changed Recipient: "Erik" apply apply Status: Status: Recipient: "Draft" "Draft" "Erik" Total: Total: Invoice Invoice 0.00 0.00 Exploring lightweight event sourcing 13
  30. 30. Reloading from historyDraft Invoice Invoice Recipient Invoice Item Created Changed Added Recipient: "Erik" Item: "Food" Item amount: 9.95 Total amount: 9.95 apply apply apply Status: Status: Recipient: Status: Recipient: "Draft" "Draft" "Erik" "Draft" "Erik" Total: Total: Total: Invoice Invoice Invoice 0.00 0.00 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 13
  31. 31. Reloading from historyDraft Invoice Invoice Recipient Invoice Item Created Changed Added Recipient: "Erik" Item: "Food" Item amount: 9.95 Total amount: 9.95 apply apply apply Status: Status: Recipient: Status: Recipient: "Draft" "Draft" "Erik" "Draft" "Erik" Total: Total: Total: Invoice Invoice Invoice 0.00 0.00 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 13
  32. 32. Aggregates• A cluster of associated objects that are treated as a unit for the purpose of data changes• Each aggregate has its own stream of events• Consistency boundary Exploring lightweight event sourcing 14
  33. 33. Aggregates Status: Recipient: "Draft" "Erik" Total: Invoice 9.95 Item: Invoice Item "Food" Amount: 9.95 Exploring lightweight event sourcing 15
  34. 34. Aggregates Status: Recipient: "Draft" "Erik" Total: Invoice 9.95 Item: Invoice Item "Food" Amount: 9.95 Draft Invoice Invoice Recipient Invoice Item Created Changed Added Recipient: "Erik" Item: "Food" Item amount: 9.95 Total amount: 9.95 Exploring lightweight event sourcing 15
  35. 35. Event Store• Stores sequence of events per aggregate• Dispatches events when successfully committed• Replays events on startup• It’s your application’s transaction log Exploring lightweight event sourcing 16
  36. 36. Event Store• No good for queries!• Use separate views or reports that use the domain events to capture answers to queries Exploring lightweight event sourcing 17
  37. 37. Domain Codesealed trait InvoiceEventcase class InvoiceCreated() extends InvoiceEventcase class InvoiceRecipientChanged(recipient: String) extends InvoiceEventcase class InvoiceItemAdded(item: InvoiceItem, totalAmount: BigDecimal) extends InvoiceEventcase class InvoiceSent(sentOn: LocalDate, paymentDueOn: LocalDate) extends InvoiceEvent Exploring lightweight event sourcing 18
  38. 38. Domain Codesealed trait InvoiceEventcase class InvoiceCreated() extends InvoiceEventcase class InvoiceRecipientChanged(recipient: String) extends InvoiceEventcase class InvoiceItemAdded(item: InvoiceItem, totalAmount: BigDecimal) extends InvoiceEventcase class InvoiceSent(sentOn: LocalDate, paymentDueOn: LocalDate) extends InvoiceEventsealed trait Invoice extends AggregateRoot { protected[this] type Event = InvoiceEvent} Exploring lightweight event sourcing 18
  39. 39. Domain Codecase class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice { def changeRecipient(recipient: String): Behavior[DraftInvoice] = ... def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = ... def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = ... ... private def recipientChanged = when[InvoiceRecipientChanged] { event => copy(recipient = Some(event.recipient)) } private def itemAdded = when[InvoiceItemAdded] { event => copy(nextItemId = event.item.id + 1, items = items + (event.item.id -> event.item)) } private def sent = when[InvoiceSent] { event => SentInvoice(event.sentOn, event.paymentDueOn) }} Exploring lightweight event sourcing 19
  40. 40. Domain Codecase class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, The “Brains” items: Map[Int, InvoiceItem] = Map.empty) extends Invoice { def changeRecipient(recipient: String): Behavior[DraftInvoice] = ... def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = ... def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = ... ... private def recipientChanged = when[InvoiceRecipientChanged] { event => copy(recipient = Some(event.recipient)) } private def itemAdded = when[InvoiceItemAdded] { event => copy(nextItemId = event.item.id + 1, items = items + (event.item.id -> event.item)) } private def sent = when[InvoiceSent] { event => SentInvoice(event.sentOn, event.paymentDueOn) }} Exploring lightweight event sourcing 19
  41. 41. Domain Codecase class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, The “Brains” items: Map[Int, InvoiceItem] = Map.empty) extends Invoice { def changeRecipient(recipient: String): Behavior[DraftInvoice] = ... def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = ... def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = ... ... The “Muscle” private def recipientChanged = when[InvoiceRecipientChanged] { event => copy(recipient = Some(event.recipient)) } private def itemAdded = when[InvoiceItemAdded] { event => copy(nextItemId = event.item.id + 1, items = items + (event.item.id -> event.item)) } private def sent = when[InvoiceSent] { event => SentInvoice(event.sentOn, event.paymentDueOn) }} Exploring lightweight event sourcing 19
  42. 42. Domain Codecase class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice { def changeRecipient(recipient: String): Behavior[DraftInvoice] = recipientChanged(InvoiceRecipientChanged(recipient)) def addItem(description: String, amount: BigDecimal): Behavior[DraftInvoice] = itemAdded(InvoiceItemAdded(InvoiceItem(nextItemId, description, amount), totalAmount + amount)) def send(sentOn: LocalDate): Behavior[Either[String, SentInvoice]] = if (readyToSend_?) for (updated <- sent(InvoiceSent(sentOn, paymentDueOn = sentOn.plusDays(14)))) yield Right(updated) else pure(Left("invoice not ready to send")) private def totalAmount = items.values.map(_.amount).sum private def readyToSend_? = recipient.isDefined && items.nonEmpty ...} Exploring lightweight event sourcing 20
  43. 43. Domain Codecase class DraftInvoice( recipient: Option[String] = None, nextItemId: Int = 1, items: Map[Int, InvoiceItem] = Map.empty) extends Invoice { ... protected[this] def applyEvent = recipientChanged orElse itemAdded orElse sent ...} Exploring lightweight event sourcing 21
  44. 44. Minimal Infrastructure• Store only the domain events, replay these on application startup to reconstruct current state• Keep the current state in-memory• Use immutability data to reduce need for cloning or locking Exploring lightweight event sourcing 22
  45. 45. Simple functionality example• “CRUD”: no domain logic, so no aggregate• So we’ll persist events directly from the UI• Useful to get started• ... and even complex applications still contain simple functionality Exploring lightweight event sourcing 23
  46. 46. Aggregate Example Exploring lightweight event sourcing 24
  47. 47. Implementation Limitations• Serializing (“big lock”) event store • ~ 50 transactions per second• Number of events to replay on startup • ~ 30.000 JSON serialized events per second• Total number of objects to store in main memory • JVM can run with large heaps Exploring lightweight event sourcing 25
  48. 48. But ready to scale• Advanced event store implementations support SQL, MongoDB, Amazon SimpleDB, etc.• Use persisted view model for high volume objects (fixes startup time and memory usage)• Easy partitioning of aggregates (consistency boundary with unique identifier)• Load aggregates on-demand, use snapshotting Exploring lightweight event sourcing 26
  49. 49. Conclusion• Event Sourcing captures full historical information• Fully encapsulated domain code highly decoupled from persistence mechanism• Simpler to implement and fully understand than traditional ORM based approach• “Paradigm” shift towards Event-Driven Exploring lightweight event sourcing 27
  50. 50. Thanks!
  51. 51. References• Example code https://github.com/erikrozendaal/scala-event-sourcing-example• CQRS http://cqrsinfo.com/• Greg Young, “Unshackle Your Domain” http://www.infoq.com/presentations/greg-young-unshackle-qcon08• Pat Helland, “Life Beyond Distributed Transactions: An Apostates Opinion” http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf• Erik Rozendaal, “Towards an immutable domain model” http:// blog.zilverline.com/2011/02/10/towards-an-immutable-domain-model- monads-part-5/• Frameworks: http://www.axonframework.org/ (Java) and http://ncqrs.org/ (.NET)• Event store: https://github.com/joliver/EventStore (.NET) with over 20 supported data stores Exploring lightweight event sourcing 29

×