Downloaded 428 times
![View, as ReactiveStream
// 1 producer and 2 consumers:!
val p1: Producer[Persistent] = PersistentFlow.!
fromProcessor(“processor-1").!
toProducer(materializer)!
!
Flow(p1).!
foreach(p => println(s"consumer-1: ${p.payload}”)).!
consume(materializer)!
!
Flow(p1).!
foreach(p => println(s"consumer-2: ${p.payload}”)).!
consume(materializer)
pull request
by krasserm
early preview
Akka
Persistence
ScalaDays](https://image.slidesharecdn.com/akka-persistence-scaladays-2014-140621085129-phpapp02/85/Resilient-Applications-with-Akka-Persistence-Scaladays-2014-55-320.jpg)
![View, as ReactiveStream
// 2 producers (merged) and 1 consumer:!
val p2: Producer[Persistent] = PersistentFlow.!
fromProcessor(“processor-2").!
toProducer(materializer)!
val p3: Producer[Persistent] = PersistentFlow.!
fromProcessor(“processor-3").!
toProducer(materializer)!
!
Flow(p2).merge(p3). // triggers on “either”!
foreach { p => println(s"consumer-3: ${p.payload}") }.!
consume(materializer)!
pull request
by krasserm
early preview
Akka
Persistence
ScalaDays](https://image.slidesharecdn.com/akka-persistence-scaladays-2014-140621085129-phpapp02/85/Resilient-Applications-with-Akka-Persistence-Scaladays-2014-56-320.jpg)
Uploaded byBjörn Antonsson
Resilient Applications with Akka Persistence - Scaladays 2014
This document discusses Akka persistence and command/event sourcing. It introduces key concepts like command and event sourcing where state is represented as a log of events, and commands generate events. It describes how Akka persistence allows storing commands/events to a journal and replaying them on recovery. It also discusses views which allow querying the event log, and using Akka persistence in a clustered environment with techniques like cluster singleton and cluster sharding.
More Related Content
![[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)](https://cdn.slidesharecdn.com/ss_thumbnails/2014-akka-streams-tokyo-140914201812-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Resilient Applications with Akka Persistence - Scaladays 2014
Resilient Applications with Akka Persistence - Scaladays 2014
- 1. Resilient Applications withAkka Persistence Patrik Nordwall @patriknw Konrad Malawski @ktosopl Björn Antonsson @bantonsson
- 2. Reactive Applications Akka Persistence ScalaDays 2014
- 3. Resilient • Embrace Failure •Failure is a normal part of the application lifecycle • Self Heal • Failure is detected, isolated, and managed Akka Persistence ScalaDays 2014
- 4. The Naïve Way •Write State to Database • Transactions Everywhere • Problem Solved? • Not Scalable, Responsive, Event-Driven! Akka Persistence ScalaDays 2014
- 5. Command and EventSourcing
- 6. Command and EventSourcing • State is the sum of Events • Events are persisted to Store • Append only • Scales well Akka Persistence ScalaDays 2014
- 7. Command v.s. Event •Command • What someone wants me to do • Can be rejected • Event • Something that has already happened • An immutable fact Akka Persistence ScalaDays 2014
- 8. Commands can GenerateEvents • If I accept a Command and change State • Persist Event to Store • If I crash • Replay Events to recover State Akka Persistence ScalaDays 2014
- 9. Persist All Commands? •If I crash on a Command • I will likely crash during recovery • Like the Army • Don't question orders • Repeat until success Akka Persistence ScalaDays 2014
- 10. Only Persist Events •Only accepted Commands generate Events • No surprises during recovery • Like a dieting method • You are what you eat Akka Persistence ScalaDays 2014
- 11. Achievement Unlocked? • Resilient •State is recoverable • Scalable • Append only writes • Something Missing? • Queries Akka Persistence ScalaDays 2014
- 12. CQRS Command Query ResponsibilitySegregation
- 13. CQRS • Separate Models •Command Model • Optimized for command processing • Query Model • Optimized data presentation Akka Persistence ScalaDays 2014
- 14. Query Model fromEvents • Source the Events • Pick what fits • In Memory • SQL Database • Graph Database • Key Value Store Akka Persistence ScalaDays 2014
- 15. Akka Persistence ScalaDays 2014 Client Service Query Model Command Store Query Store Command Model
- 16.
- 17. PersistentActor Processor & EventsourcedProcessor Replaces: in Akka 2.3.4+
- 18. super quick domainmodelling! sealed trait Command! case class GiveMe(coins: Int) extends Command! case class TakeMy(coins: Int) extends Command Commands - what others “tell” us; not persisted case class Wallet(coins: Int) {! def updated(diff: Int) = State(coins + diff)! } State - reflection of a series of events sealed trait Event! case class BalanceChangedBy(coins: Int) extends Event! Events - reflect effects, past tense; persisted
- 19. var state =S0 ! def processorId = “a” ! PersistentActor Command ! ! Journal
- 20. PersistentActor var state =S0 ! def processorId = “a” ! ! ! Journal Generate Events
- 21. PersistentActor var state =S0 ! def processorId = “a” ! ! ! Journal Generate Events E1
- 22. PersistentActor ACK “persisted” ! ! Journal E1 var state =S0 ! def processorId = “a” !
- 23. PersistentActor “Apply” event ! ! Journal E1 var state =S1 ! def processorId = “a” ! E1
- 24. PersistentActor ! ! Journal E1 var state =S1 ! def processorId = “a” ! E1 Okey!
- 25. PersistentActor ! ! Journal E1 var state =S1 ! def processorId = “a” ! E1 Okey!
- 26. PersistentActor ! ! Journal E1 var state =S1 ! def processorId = “a” ! E1 Ok, he got my $.
- 27. PersistentActor class BitCoinWallet extendsPersistentActor {! ! var state = Wallet(coins = 0)! ! def updateState(e: Event): State = {! case BalanceChangedBy(coins) => state.updatedWith(coins)! }! ! // API:! ! def receiveCommand = ??? // TODO! ! def receiveRecover = ??? // TODO! ! }!
- 28. persist(e) { e=> }
- 29. PersistentActor def receiveCommand ={! ! case TakeMy(coins) =>! persist(BalanceChangedBy(coins)) { changed =>! state = updateState(changed) ! }! ! ! ! ! ! ! } async callback
- 30. PersistentActor: persist(){} def receiveCommand= {! ! ! ! ! ! ! case GiveMe(coins) if coins <= state.coins =>! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }! } async callback Safe to mutate the Actor’s state
- 31. PersistentActor def receiveCommand ={! ! ! ! ! ! ! case GiveMe(coins) if coins <= state.coins =>! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }! } Safe to access sender here
- 32. persist(){} - Orderingguarantees ! ! Journal E1 var state = S0 ! def processorId = “a” ! C1 C2 C3
- 33. ! ! Journal E1 var state =S0 ! def processorId = “a” ! C1 C2 C3 Commands get “stashed” until processing C1’s events are acted upon. persist(){} - Ordering guarantees
- 34. ! ! Journal var state =S0 ! def processorId = “a” ! C1 C2 C3 E1 E2 E2E1 events get applied in-order persist(){} - Ordering guarantees
- 35. C2 ! ! Journal var state =S0 ! def processorId = “a” ! C3 E1 E2 E2E1 and the cycle repeats persist(){} - Ordering guarantees
- 36.
- 37. persistAsync(e) { e=> } + defer(e) { e => }
- 38. def receiveCommand ={! ! ! ! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }! ! ! ! ! } persistAsync PersistentActor: persistAsync(){} will NOT force stashing of commands
- 39. PersistentActor: persistAsync(){} def receiveCommand= {! ! ! ! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }! ! defer(Marked(id)) { marked =>! sender() ! marked! }! } execute once all persistAsync handlers done NOT persisted
- 40. persistAsync(){} - Orderingguarantees ! ! Journal var state = S0 ! def processorId = “a” ! C1 C2 C3
- 41. persistAsync(){} - Orderingguarantees ! ! Journal var state = S0 ! def processorId = “a” !C2 C3
- 42. persistAsync(){} - Orderingguarantees ! ! Journal var state = S0 ! def processorId = “a” ! C3
- 43. persistAsync(){} - Orderingguarantees ! ! Journal var state = S0 ! def processorId = “a” ! C3 E1 E2
- 44. persistAsync(){} - Orderingguarantees var state = S0 ! def processorId = “a” ! C3 E1 Akka Persistence ScalaDays ! ! Journal E1 E2
- 45. persistAsync(){} - Orderingguarantees E1 var state = S1 ! def processorId = “a” ! E2 E1 E2 ! ! Journal Akka Persistence ScalaDays E2 E3E1
- 46. persistAsync(){} - Orderingguarantees E1 var state = S2 ! def processorId = “a” ! E2 E1 E2 deferred handlers triggered M1 M2 ! ! Journal Akka Persistence ScalaDays E2 E3E1
- 47.
- 48. Eventsourced, recovery /** MUSTNOT SIDE-EFFECT! */! def receiveRecover = {! case replayedEvent: Event => ! state = updateState(replayedEvent)! } re-using updateState, as seen in receiveCommand Akka Persistence ScalaDays
- 49.
- 50. Journal (DB) ! ! ! Views ! Processor ! def processorId =“a” ! polling Akka Persistence ScalaDays ! View ! def processorId = “a” ! ! !
- 51. Journal (DB) ! ! ! Views ! Processor ! def processorId =“a” ! polling ! View ! def processorId = “a” ! ! ! polling different ActorPath, same processorId Akka Persistence ScalaDays ! View ! def processorId = “a” ! ! !
- 52. View class DoublingCounterProcessor extendsView {! var state = 0! override val processorId = "counter"! ! def receive = {! case Persistent(payload, seqNr) =>! // “state += 2 * payload” ! ! }! } subject to change! Akka Persistence ScalaDays
- 53. Views, as Reactive Streams Akka Persistence ScalaDays
- 54. View, as ReactiveStream //Imports ...! ! import org.reactivestreams.api.Producer! ! import akka.stream._! import akka.stream.scaladsl.Flow! ! import akka.persistence._! import akka.persistence.stream._! val materializer = FlowMaterializer(MaterializerSettings())! pull request by krasserm early preview Akka Persistence ScalaDays
- 55. View, as ReactiveStream //1 producer and 2 consumers:! val p1: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-1").! toProducer(materializer)! ! Flow(p1).! foreach(p => println(s"consumer-1: ${p.payload}”)).! consume(materializer)! ! Flow(p1).! foreach(p => println(s"consumer-2: ${p.payload}”)).! consume(materializer) pull request by krasserm early preview Akka Persistence ScalaDays
- 56. View, as ReactiveStream //2 producers (merged) and 1 consumer:! val p2: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-2").! toProducer(materializer)! val p3: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-3").! toProducer(materializer)! ! Flow(p2).merge(p3). // triggers on “either”! foreach { p => println(s"consumer-3: ${p.payload}") }.! consume(materializer)! pull request by krasserm early preview Akka Persistence ScalaDays
- 57. Akka Persistence ScalaDays 2014 Usage in a Cluster • distributed journal (http://akka.io/community/) • Cassandra • DynamoDB • HBase • MongoDB • shared LevelDB journal for testing • single writer • cluster singleton • cluster sharding
- 58. Akka Persistence ScalaDays 2014 Cluster Singleton A B C D
- 59. Akka Persistence ScalaDays 2014 Cluster Singleton A B C D role: backend-1 role: backend-1 role: backend-2 role: backend-2
- 60. Akka Persistence ScalaDays 2014 Cluster Sharding A B C D
- 61. Akka Persistence ScalaDays 2014 Cluster Sharding sender id:17 region node-‐1 coordinator region node-‐2 region node-‐3 GetShardHome:17 id:17 ShardHome:17 -‐> node2 17 -‐> node2
- 62. Akka Persistence ScalaDays 2014 Cluster Sharding sender region node-‐1 coordinator region node-‐2 region node-‐3 id:17 id:17 GetShardHome:17 ShardHome:17 -‐> node2 id:17 17 -‐> node2 17 -‐> node2
- 63. Akka Persistence ScalaDays 2014 Cluster Sharding 17 sender region node-‐1 coordinator region node-‐2 region node-‐3 id:17 id:17 17 -‐> node2 17 -‐> node2 17 -‐> node2
- 64. Akka Persistence ScalaDays 2014 Cluster Sharding 17 sender region node-‐1 coordinator region node-‐2 region node-‐3 17 -‐> node2 17 -‐> node2 17 -‐> node2 id:17
- 65. Akka Persistence ScalaDays 2014 Cluster Sharding 17 sender region node-‐1 coordinator region node-‐2 region node-‐3 17 -‐> node2 17 -‐> node2 17 -‐> node2 id:17
- 66. Cluster Sharding val idExtractor:ShardRegion.IdExtractor = { case cmd: Command => (cmd.postId, cmd) } ! val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => (math.abs(cmd.postId.hashCode) % 100).toString } ClusterSharding(system).start( typeName = BlogPost.shardName, entryProps = Some(BlogPost.props()), idExtractor = BlogPost.idExtractor, shardResolver = BlogPost.shardResolver) val blogPostRegion: ActorRef = ClusterSharding(context.system).shardRegion(BlogPost.shardName) ! val postId = UUID.randomUUID().toString blogPostRegion ! BlogPost.AddPost(postId, author, title)
- 67. Akka Persistence ScalaDays 2014 Lost messages sender destination $
- 68. Akka Persistence ScalaDays 2014 At-least-once delivery - duplicates sender destination $ ok $ $ $ ok Re-‐send
- 69. Akka Persistence ScalaDays 2014 M2 At-least-once delivery - unordered sender destination M1 ok 1 ok 2 M2 ok 3 M3 M1M3 M2 Re-‐send
- 70. Akka Persistence ScalaDays 2014 M2 At-least-once delivery - crash sender destination M1 ok 1 ok 2 M2 ok 3 M3 1. Sent M1 2. Sent M2 3. Sent M3 M3 5. M2 Confirmed 6. M3 Confirmed 4. M1 Confirmed sender M1M2 M3
- 71. PersistentActor with AtLeastOnceDelivery caseclass Msg(deliveryId: Long, s: String) case class Confirm(deliveryId: Long) sealed trait Evt case class MsgSent(s: String) extends Evt case class MsgConfirmed(deliveryId: Long) extends Evt class Sender(destination: ActorPath) extends PersistentActor with AtLeastOnceDelivery { ! def receiveCommand: Receive = { case s: String => persist(MsgSent(s))(updateState) case Confirm(deliveryId) => persist(MsgConfirmed(deliveryId))(updateState) } ! def receiveRecover: Receive = { case evt: Evt => updateState(evt) } ! def updateState(evt: Evt): Unit = evt match { case MsgSent(s) => deliver(destination, deliveryId => Msg(deliveryId, s)) ! case MsgConfirmed(deliveryId) => confirmDelivery(deliveryId) } }
- 72. Akka Persistence ScalaDays 2014 Next step • Documentation • http://doc.akka.io/docs/akka/2.3.3/scala/persistence.html • http://doc.akka.io/docs/akka/2.3.3/java/persistence.html • http://doc.akka.io/docs/akka/2.3.3/contrib/cluster-sharding.html • Typesafe Activator • https://typesafe.com/activator/template/akka-sample-persistence-scala • https://typesafe.com/activator/template/akka-sample-persistence-java • http://typesafe.com/activator/template/akka-cluster-sharding-scala • Mailing list • http://groups.google.com/group/akka-user • Migration guide from Eventsourced • http://doc.akka.io/docs/akka/2.3.3/project/migration-guide-eventsourced-2.3.x.html
- 73. ©Typesafe 2014 –All Rights Reserved