Event Sourcing using Akka on AWS

507 views

Published on

"'Capture all changes to an application state as a sequence of events' is what Martin Fowler said about Event Sourcing in 2005 and what is the starting point into that topic for this talk.

I will demonstrate how you can store events using Akka Persistence and then distribute them via AWS to be consumed by your other services.

An event based architecture has lots of technical and organisational benefits for your development team. It can be a huge gain for your development process, but can also be difficult to implement as there are lots of challenges.

I will discuss the good as well as the bad things and provide solutions to overcome common pitfalls and aforementioned challenges."

Published in: Software
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
507
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Event Sourcing using Akka on AWS

  1. 1. EVENT‐DRIVEN‐ ARCHITECTURE ON AWS USING AKKA
  2. 2. Software Engineer @firstbird Contact me: @pfeiffer_d_ DANIEL PFEIFFER
  3. 3. THE JOURNEY OF EVENTS that are stored with Akka Persistence to be then distributed via AWS to be consumed with Akka Stream what comes after that
  4. 4. THE EXAMPLE
  5. 5. DOMAIN DRIVEN DESIGN We are talking about Aggregates and Events
  6. 6. EVENT SOURCING "Capture all changes to an application state as a sequence of events" - Martin Fowler, 2005
  7. 7. CQRS "Asking a question should not change the answer." - Bertrand Meyer, 2012
  8. 8. THE DIFFERENCE
  9. 9. PERSISTENTACTOR class TimeEntryAggregate(id: String) extends PersistentActor{ override def persistenceId: String = ??? override def receiveCommand: Receive = ??? override def receiveRecover: Receive = ??? }
  10. 10. A PersistentActorclass represents one DDD aggregate. class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def persistenceId: String = s"time-entry-$id" ... }
  11. 11. A PersistentActorreceives commands and persists events. case class CreateTimeEntry( begin: DateTime, end: DateTime ) case class TimeEntryCreated( begin: DateTime, end: DateTime, sequenceNr: Long )
  12. 12. A PersistentActorreceives commands and persists events. class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def receiveCommand: Receive = case CreateTimeEntry(start, end) => val event = TimeEntryCreated(start,end, lastSequenceNr + 1) persist(event){e => // update internal state // here we go after the event is persisted } } ... }
  13. 13. A PersistentActorrecovers from the journal class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def receiveRecover: Receive = { case TimeEntryCreated(start,end) => // update internal state } ... }
  14. 14. STORING EVENTS TO A JOURNAL IS NICE, BUT ... others may be as well interested, so we have to distribute them.
  15. 15. WITHIN ONE JVM WE COULD... use Akka EventStreamto publish events ... class TimeEntryActor() extends PersistentActor{ ... persist(event){e => context.system.eventStream.publish(e) } ... }
  16. 16. ... and subscribe from interested actors class EventConsumer extends Actor{ override def preStart: Unit = { context.system.eventStream.subscribe(classOf[TimeEntryActor.TimeEntry } override def receive: Receive = { case e: TimeEntryCreated => //do something with that event } }
  17. 17. HOW DO I INTERACT WITH MY AGGREGATES?
  18. 18. DON'T CALL ME! CALL MY OFFICE!
  19. 19. EXAMPLE 1
  20. 20. BUT WE WANT TO BE COOL
  21. 21. DISTRIBUTED SYSTEMS ARE HARD
  22. 22. WE WANT TO MAKE THEM EASIER
  23. 23. INTRODUCE A MESSAGE BROKER, PUB/SUB ...
  24. 24. THE EXAMPLE ON AWS STEROIDS
  25. 25. TIME ENTRY SERVICE
  26. 26. EMAIL SERVICE
  27. 27. SQS FLOW SqsSource(...) .map(msg => Envelope(msg.receiptHandle, msg.body)) .via(unmarshal()) .via(process()) .runWith(ack(...))
  28. 28. SQS MESSAGE { "messageId" : "", "receiptHandle" : "", "md5OfBody" : "", "body" : "" }
  29. 29. SNS NOTIFICATION ENVELOPE { "Type" : "Notification", "MessageId" : "", "TopicArn" : "", "Subject" : "time_entry.approved", "Message" : "", "Timestamp" : "", "SignatureVersion" : "", "Signature" : "", "SigningCertURL" : "", "UnsubscribeURL" : "" }
  30. 30. CHALLENGES
  31. 31. EVENT SCHEMA EVOLUTION adding a field to an event type, remove or rename field in event type, remove event type, split event into multiple smaller events.
  32. 32. case class TimeEntryCreated( id: UUID, begin: DateTime, end: DateTime, timeEntryUserId: UUID, userId: UUID ) gets marshalled to { "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000" }
  33. 33. Then something within our schema changes case class TimeEntryCreated( ... description: String ... ) and we will have fun with that one { "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000" }
  34. 34. { "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000" } needs to be transformed to that { "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000", "description" : "N/A" }
  35. 35. class Evolution(system: ExtendedActorSystem) extends EventAdapter{} override def fromJournal(event: Any, manifest: String): EventSeq = { val es = ... //doing evolution on event EventSeq(es) } override def manifest(event: Any): String = { val version = ??? event.getClass.getSimpleName + ":" + version } override def toJournal(event: Any): Any = ??? //write to the journal }
  36. 36. SCALING WITH AKKA CLUSTER ON AWS or "Who am I, and where are all the others?"
  37. 37. JOINING A CLUSTER gives you information about your instance http://169.254.169.254/latest/meta-data/instance-id
  38. 38. PRESERVING MESSAGE ORDER
  39. 39. The query side could look like that | id | ... | version | | --- | --- | ------- | | 1 | ... | 45 | | 2 | ... | 3 | | 3 | ... | 58 |
  40. 40. CHOOSING THE RIGHT DATASTORE you will make mistakes so plan for it
  41. 41. ASYNCHRONOUS MEANS NOT IMMEDIATE
  42. 42. WHY DO WE REALLY WANT TO DO THAT?
  43. 43. WE WANT TO SLEEP BETTER
  44. 44. WE WANT TO GET RID OF OUR MONOLITH
  45. 45. OUR DEVELOPMENT PROCESS BENEFITS
  46. 46. WE CAN ALWAYS AGGREGATE OUR EVENTS
  47. 47. IF SOMEONE STARTS TO ASK QUESTIONS... An audit log is included in your application for free!
  48. 48. SUMMARY
  49. 49. THE FIRST PROTOTYPE IS EASY BUT TO TACKLE THE CHALLENGES NEEDS EFFORT!
  50. 50. CONFIGURE YOUR SQS QUEUES PROPERLY
  51. 51. LET EACH SERVICE MANAGE IT'S RESOURCES ITSELF.
  52. 52. ONE TOPIC PER AGGREGATE
  53. 53. LITERATURE https://www.infoq.com/articles/AmazonPubSub http://martinfowler.com/eaaDev/EventCollaboration.html http://martinfowler.com/bliki/CQRS.html https://github.com/dpfeiffer/event-sourcing-aws- akka-showcase http://www.oreilly.com/programming/free/reactive- microservices-architecture.html http://doc.akka.io/docs/akka/snapshot/scala/persistence- schema-evolution.html#persistence-schema- evolution-scala http://chrisloy.net/2014/05/11/akka- cluster-ec2-autoscaling.html

×