Akka persistence == event sourcing in 30 minutes

26,063 views

Published on

Akka 2.3 introduces akka-persistence, a wonderful way of implementing event-sourced applications. Let's give it a shot and see how DDD and Akka are a match made in heaven :-)

Published in: Technology, Design
0 Comments
92 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
26,063
On SlideShare
0
From Embeds
0
Number of Embeds
412
Actions
Shares
0
Downloads
426
Comments
0
Likes
92
Embeds 0
No embeds

No notes for slide

Akka persistence == event sourcing in 30 minutes

  1. 1. Akka persistence (message sourcing in 30 minutes) Konrad 'ktoso' Malawski Scalar 2014 @ Warsaw, PL
  2. 2. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  3. 3. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  4. 4. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  5. 5. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  6. 6. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  7. 7. Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow hAkker @
  8. 8. mainly by Martin Krasser ! ! (as contractor for Typesafe) ! inspired by: akka-persistence https://github.com/krasserm https://github.com/eligosource/eventsourced
  9. 9. dependencies libraryDependencies ++= Seq(! "com.typesafe.akka" %% “akka-actor" % "2.3.0",! "com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.0"! )
  10. 10. Show of hands!
  11. 11. Show of hands!
  12. 12. Show of hands!
  13. 13. Show of hands!
  14. 14. sourcing styles Command Sourcing Event Sourcing msg: DoThing msg persisted before receive imperative, “do the thing” business logic change, can be reflected in reaction Processor
  15. 15. sourcing styles Command Sourcing Event Sourcing msg: DoThing msg: ThingDone msg persisted before receive commands converted to events, must be manually persisted imperative, “do the thing” past tense, “happened” business logic change, can be reflected in reaction business logic change, won’t change previous events Processor EventsourcedProcessor
  16. 16. Plain Actors
  17. 17. count: 0 ! ! Actor
  18. 18. count: 0 ! ! Actor An Actor that keeps count of messages it processed
  19. 19. count: 0 ! ! Actor An Actor that keeps count of messages it processed
  20. 20. count: 0 ! ! Actor An Actor that keeps count of messages it processed Let’s send 2 messages to it
  21. 21. count: 0 ! ! Actor An Actor that keeps count of messages it processed Let’s send 2 messages to it (it’s “commands”)
  22. 22. Actor ! ! class Counter extends Actor {! var count = 0! 
 def receive = {! case _ => count += 1! }! }
  23. 23. count: 0 ! ! Actor
  24. 24. count: 0 ! ! Actor
  25. 25. count: 1 ! ! Actor
  26. 26. count: 1 ! ! Actor crash!
  27. 27. Actor crash!
  28. 28. Actor restart
  29. 29. count: 0 ! ! Actor restart
  30. 30. count: 0 ! ! Actor restarted
  31. 31. count: 1 ! ! Actor restarted
  32. 32. count: 1 ! ! Actor restarted
  33. 33. count: 1 ! ! wrong! expected count == 2! Actor restarted
  34. 34. Processor
  35. 35. var count = 0 ! def processorId = “a” ! Journal (DB) ! ! ! Processor
  36. 36. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” !
  37. 37. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” !
  38. 38. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” !
  39. 39. Journal (DB) ! ! ! Processor var count = 1 ! def processorId = “a” !
  40. 40. Journal (DB) ! ! ! Processor var count = 1 ! def processorId = “a” !
  41. 41. Journal (DB) ! ! ! Processor var count = 1 ! def processorId = “a” ! crash!
  42. 42. Journal (DB) ! ! ! Processor
  43. 43. Journal (DB) ! ! ! Processor restart
  44. 44. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” ! restart
  45. 45. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” ! replay! restart
  46. 46. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” !
  47. 47. Journal (DB) ! ! ! Processor var count = 0 ! def processorId = “a” ! replay!
  48. 48. Journal (DB) ! ! ! Processor var count = 1 ! def processorId = “a” !
  49. 49. Journal (DB) ! ! ! Processor var count = 1 ! def processorId = “a” ! replay!
  50. 50. Journal (DB) ! ! ! Processor var count = 1 ! def processorId = “a” !
  51. 51. Journal (DB) ! ! ! Processor var count = 2 ! def processorId = “a” !
  52. 52. Journal (DB) ! ! ! Processor var count = 2 ! def processorId = “a” ! yay!
  53. 53. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }! }
  54. 54. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }! } counter ! Persistent(payload)
  55. 55. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }! } counter ! Persistent(payload)
  56. 56. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }! } counter ! Persistent(payload)
  57. 57. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }! } counter ! Persistent(payload) is already persisted!
  58. 58. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }! } counter ! Persistent(payload) sequenceNr (generated by akka) is already persisted!
  59. 59. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }! } counter ! payload
  60. 60. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }! } counter ! payload won’t persist
  61. 61. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }! } counter ! payload won’t persist won’t replay
  62. 62. Processor import akka.persistence._! ! class CounterProcessor extends Processor {! var count = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! ! case notPersistentMsg =>! // msg not persisted - like in normal Actor! count += 1! }! }
  63. 63. Processor Upsides • Persistent Command Sourcing “out of the box” • Pretty simple, persist handled for you • Once you get the msg, it’s persisted • Pluggable Journals (HBase, Cassandra, Mongo, …) • Can replay to given seqNr (post-mortem etc!)
  64. 64. Processor Downsides • Exposes Persistent() to Actors who talk to you • No room for validation before persisting • There’s one Model, we act on the incoming msg • Lower throughput than plain Actor (limited by DB)
  65. 65. EventsourcedProcessor
  66. 66. EventsourcedProcessor (longer name == more flexibility) ;-)
  67. 67. super quick domain modeling!
  68. 68. super quick domain modeling! sealed trait Command! case class ManyCommand(nums: List[Int]) extends Command Commands - input from user,“send emails”, not persisted
  69. 69. super quick domain modeling! sealed trait Command! case class ManyCommand(nums: List[Int]) extends Command Commands - input from user,“send emails”, not persisted sealed trait Event! case class AddOneEvent(num: Int) extends Event! Events - business events emitted by the processor, persisted
  70. 70. super quick domain modeling! sealed trait Command! case class ManyCommand(nums: List[Int]) extends Command Commands - input from user,“send emails”, not persisted case class State(count: Int) {! def updated(more: Int) = State(count + more)! } State - internal actor state, kept in var sealed trait Event! case class AddOneEvent(num: Int) extends Event! Events - business events emitted by the processor, persisted
  71. 71. var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal C1
  72. 72. var count = 0 ! def processorId = “a” ! EventsourcedProcessor Command ! ! Journal C1
  73. 73. var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal C1
  74. 74. var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal C1 Generate Events
  75. 75. C1 var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal E1
  76. 76. C1 var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal E1 Event
  77. 77. C1 var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal E1
  78. 78. C1 var count = 0 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal E1 ACK “persisted”
  79. 79. C1 var count = 1 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal E1 E1
  80. 80. C1 var count = 1 ! def processorId = “a” ! EventsourcedProcessor ! ! Journal E1 E1 “Apply” event
  81. 81. EventsourcedProcessor class MultiCounter extends EventsourcedProcessor {! ! var state = State(count = 0)! ! def updateState(e: Event): State = {! case event: AddOneEvent => state.updated(event.num)! }! ! // API:! ! def receiveCommand = ??? // TODO! ! def receiveRecover = ??? // TODO! ! }!
  82. 82. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! }
  83. 83. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! }
  84. 84. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! }
  85. 85. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! } async persist that event
  86. 86. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! } async persist that event async called after successful persist
  87. 87. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! } async persist that event async called after successful persist It is guaranteed that no new commands will be received by a processor between a call to `persist` and the execution of its `handler`.
  88. 88. EventsourcedProcessor def receiveCommand = {! case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }! ! case command: Command =>! persist(AddOneEvent(command)) { state = updateState(_) }! }
  89. 89. EventsourcedProcessor case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) }
  90. 90. EventsourcedProcessor case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) } “style fix”
  91. 91. EventsourcedProcessor case ManyCommand(many) =>! for (event <- many map AddOneEvent)! persist(event) { state = updateState(_) } case ManyCommand(many) =>! persist(many map AddOneEvent) { state = updateState(_) } “style fix”
  92. 92. Eventsourced, recovery /** MUST NOT SIDE-EFFECT! */! def receiveRecover = {! case replayedEvent: Event => ! updateState(replayedEvent)! }
  93. 93. Eventsourced, snapshots
  94. 94. Eventsourced, snapshots /** MUST NOT SIDE-EFFECT! */! def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state! ! case replayedEvent: Event => ! updateState(replayedEvent)! }
  95. 95. Eventsourced, snapshots /** MUST NOT SIDE-EFFECT! */! def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state! ! case replayedEvent: Event => ! updateState(replayedEvent)! }
  96. 96. Eventsourced, snapshots /** MUST NOT SIDE-EFFECT! */! def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state! ! case replayedEvent: Event => ! updateState(replayedEvent)! } snapshot!? how?
  97. 97. Eventsourced, snapshots def receiveCommand = {! case command: Command =>! saveSnapshot(state) // async!! } /** MUST NOT SIDE-EFFECT! */! def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state! ! case replayedEvent: Event => ! updateState(replayedEvent)! } snapshot!? how?
  98. 98. Snapshots
  99. 99. Snapshots (in SnapshotStore)
  100. 100. …sum of states… Snapshots ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  101. 101. state until [E8] Snapshots S8 ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  102. 102. state until [E8] Snapshots S8 ! ! Snapshot Store snapshot! ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  103. 103. state until [E8] Snapshots S8 ! ! Snapshot Store ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8 S8
  104. 104. state until [E8] Snapshots S8 ! ! Snapshot Store ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8 S8 crash!
  105. 105. Snapshots ! ! Snapshot Store ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8 S8 crash!
  106. 106. “bring me up-to-date!” Snapshots ! ! Snapshot Store ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8 S8 restart!
  107. 107. “bring me up-to-date!” Snapshots ! ! Snapshot Store ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8 S8 restart! replay!
  108. 108. “bring me up-to-date!” Snapshots ! ! Snapshot Store S8 restart! replay! ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  109. 109. “bring me up-to-date!” Snapshots ! ! Snapshot Store S8 restart! replay! ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  110. 110. “bring me up-to-date!” Snapshots ! ! Snapshot Store S8 restart! replay! S8 ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  111. 111. state until [E8] Snapshots ! ! Snapshot Store S8 restart! replay! S8 ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  112. 112. state until [E8] Snapshots ! ! Snapshot Store S8 S8 ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8
  113. 113. state until [E8] Snapshots ! ! Snapshot Store S8 S8 ! ! Journal E1 E2 E3 E4 E5 E6 E7 E8 We could delete these!
  114. 114. trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }! }! Snapshots, save
  115. 115. trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }! }! Snapshots, save Async!
  116. 116. trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }! }! Snapshots, success
  117. 117. trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }! }! Snapshots, success final case class SnapshotMetadata(! processorId: String, sequenceNr: Long, ! timestamp: Long)
  118. 118. trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }! }! Snapshots, success
  119. 119. Snapshot Recovery class Counter extends Processor {! var total = 0! ! def receive = {! case SnapshotOffer(metadata, snap: Int) => ! total = snap! ! case Persistent(payload, sequenceNr) => // ...! }! }
  120. 120. Snapshots Upsides • Simple! • Faster recovery (!) • Allows to delete “older” events • “known state at point in time”
  121. 121. Snapshots Downsides • More logic to write • Maybe not needed if events replay “fast enough” • Possibly “yet another database” to pick • snapshots are different than events, may be big!
  122. 122. Views
  123. 123. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” !
  124. 124. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” ! ! View ! def processorId = “a” ! ! !
  125. 125. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” ! ! View ! def processorId = “a” ! ! ! pooling
  126. 126. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” ! ! View ! def processorId = “a” ! ! ! pooling ! View ! def processorId = “a” ! ! ! pooling
  127. 127. Journal (DB) ! ! ! Views ! Processor ! def processorId = “a” ! ! View ! def processorId = “a” ! ! ! pooling ! View ! def processorId = “a” ! ! ! pooling different ActorPath, same processorId
  128. 128. View class DoublingCounterProcessor extends View {! var state = 0! 
 override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // “state += 2 * payload” ! ! }! }
  129. 129. Akka Persistence Plugins
  130. 130. Akka Persistence Plugins Plugins are Community maintained!
 (“not sure how production ready”) http://akka.io/community/#journal_plugins
  131. 131. Akka Persistence Plugins Plugins are Community maintained!
 (“not sure how production ready”) http://akka.io/community/#journal_plugins • Journals - Cassandra, HBase, Mongo …
  132. 132. Akka Persistence Plugins Plugins are Community maintained!
 (“not sure how production ready”) http://akka.io/community/#journal_plugins • Journals - Cassandra, HBase, Mongo … • SnapshotStores - Cassandra, HDFS, HBase, Mongo …
  133. 133. SnapshotStore Plugins! http://akka.io/community/#journal_plugins
  134. 134. Links • Official docs: http://doc.akka.io/docs/akka/2.3.0/scala/persistence.html • Patrik’s Slides & Webinar: http://www.slideshare.net/patriknw/akka- persistence-webinar • Papers: • Sagas http://www.cs.cornell.edu/andru/cs711/2002fa/reading/ sagas.pdf • Life beyond Distributed Transactions: http://www-db.cs.wisc.edu/ cidr/cidr2007/papers/cidr07p15.pdf • Pics: • http://misaspuppy.deviantart.com/art/Messenger-s-Cutie-Mark-A- Flying-Envelope-291040459
  135. 135. Mailing List groups.google.com/forum/#!forum/akka-user
  136. 136. Links Eric Evans - “the DDD book” http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215 ! ! ! ! ! ! VaughnVernon’s Book and Talk http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577 video: https://skillsmatter.com/skillscasts/4698-reactive-ddd-with-scala-and-akka ! ! !
  137. 137. ktoso @ typesafe.com twitter: ktosopl github: ktoso blog: project13.pl team blog: letitcrash.com Scalar 2014 @ Warsaw, PL ! Dzięki! Thanks! ありがとう! ! ! ping me:

×