Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing Events-first Microservices

324 views

Published on

In this talk, we will explore the nature of events, what it means to be event-driven, and how we can unleash the power of events and commands by applying an events-first domain-driven design to microservices-based architectures.

We will start by developing a solid theoretical understanding of how to design systems of event-driven microservices. Then we will discuss the practical tools and techniques you can use to reap the most benefit from that design, as well as, most importantly, what to avoid along the way.

We’ll discuss how an events-first design approach to building microservices can improve the following characteristics over competing techniques:
- increase certainty
- increase resilience
- increase scalability
- increase traceability
- increase loose coupling
- reduce risk

Skeptics should definitely attend.

Published in: Software
  • Be the first to comment

Designing Events-first Microservices

  1. 1. Designing Events First Microservices Jonas Bonér @jboner
  2. 2. 1. Events drive autonomy 2. Events help reduce risk 3. Events help you move faster 4. Events Increase Loose coupling 5. Events increase stability 6. Events increase scalability 7. Events increase resilience 8. Events increase traceability 9. Events allow for time-travel Why Should you care about Events?
  3. 3. Why Now? 1. Cloud and multicore architectures 2. Microservices and distributed systems 3. Data-centric applications - Data in motion 4. “We want more, of everything, and we want it now.” -Your Customers
  4. 4. So, you want to do microservices?
  5. 5. Make sure you don’t end up with Microliths
  6. 6. Make sure you don’t end up with Microliths We can do better than this
  7. 7. Events First Domain Driven Design
  8. 8. “When you start modeling events, it forces you to think about the behaviour of the system. As opposed to thinking about the structure of the system.” - Greg Young A Decade of DDD, CQRS, Event Sourcing, Greg Young (Presentation from 2016)
  9. 9. ✴ Don’t focus on the things The Nouns The Domain Objects
  10. 10. ✴ Don’t focus on the things The Nouns The Domain Objects ✴ Focus on what happens The Verbs The Events
  11. 11. What is an Event?
  12. 12. The Nature of Events
  13. 13. ✴Events represent Facts of information ➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow The Nature of Events
  14. 14. ✴Events represent Facts of information ➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow ✴ Events/Facts can be disregarded/Ignored The Nature of Events
  15. 15. ✴Events represent Facts of information ➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow ✴ Events/Facts can be disregarded/Ignored ✴ Events/Facts Can not be retracted (once accepted) The Nature of Events
  16. 16. ✴Events represent Facts of information ➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow ✴ Events/Facts can be disregarded/Ignored ✴ Events/Facts Can not be retracted (once accepted) ✴ Events/Facts Can not be deleted (once accepted) ➡ Might be needed for legal or moral reasons The Nature of Events
  17. 17. ✴Events represent Facts of information ➡ Facts are immutable ➡ Facts Accrue - Knowledge can only grow ✴ Events/Facts can be disregarded/Ignored ✴ Events/Facts Can not be retracted (once accepted) ✴ Events/Facts Can not be deleted (once accepted) ➡ Might be needed for legal or moral reasons ✴ Events/Facts (new) can invalidate existing Facts The Nature of Events
  18. 18. Mine the Facts
  19. 19. Event Storming
  20. 20. Event Driven Design
  21. 21. ✴ IntentS ➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer Event Driven Design
  22. 22. ✴ IntentS ➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer Event Driven Design ✴ Facts ➡ State ➡ History ➡ Causality ➡ Notifications ➡ State Transfer
  23. 23. ✴ IntentS ➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer Event Driven Design ✴ Facts ➡ State ➡ History ➡ Causality ➡ Notifications ➡ State Transfer Commands
  24. 24. ✴ IntentS ➡ Communication ➡ Conversations ➡ Expectations ➡ Contracts ➡ Control Transfer Event Driven Design ✴ Facts ➡ State ➡ History ➡ Causality ➡ Notifications ➡ State Transfer Commands Events
  25. 25. Event Driven Design
  26. 26. ✴Commands ➡ Object form of method/Action request ➡ Imperative: CreateOrder, ShipProduct Event Driven Design
  27. 27. ✴Commands ➡ Object form of method/Action request ➡ Imperative: CreateOrder, ShipProduct ✴Reactions ➡ Represents side-effects Event Driven Design
  28. 28. ✴Commands ➡ Object form of method/Action request ➡ Imperative: CreateOrder, ShipProduct ✴Reactions ➡ Represents side-effects ✴Events ➡ Represents something that has happened ➡ Past-tense: OrderCreated, ProductShipped Event Driven Design
  29. 29. Commands Eventsvs
  30. 30. Commands Eventsvs 1. All about intent 1. Intentless
  31. 31. Commands Eventsvs 1. All about intent 2. Directed 1. Intentless 2. Anonymous
  32. 32. Commands Eventsvs 1. All about intent 2. Directed 3. Single addressable destination 1. Intentless 2. Anonymous 3. Just happens - for others (0-N) to observe
  33. 33. Commands Eventsvs 1. All about intent 2. Directed 3. Single addressable destination 4. Models personal communication 1. Intentless 2. Anonymous 3. Just happens - for others (0-N) to observe 4. Models broadcast (speakers corner)
  34. 34. Commands Eventsvs 1. All about intent 2. Directed 3. Single addressable destination 4. Models personal communication 5. Distributed focus 1. Intentless 2. Anonymous 3. Just happens - for others (0-N) to observe 4. Models broadcast (speakers corner) 5. Local focus
  35. 35. Commands Eventsvs 1. All about intent 2. Directed 3. Single addressable destination 4. Models personal communication 5. Distributed focus 6. Command & Control 1. Intentless 2. Anonymous 3. Just happens - for others (0-N) to observe 4. Models broadcast (speakers corner) 5. Local focus 6. Autonomy
  36. 36. Let the Events Define the Bounded Context
  37. 37. Event Driven Services
  38. 38. 1. REceive & react (or not) 
 to facts* that are coming its way Event Driven Services
  39. 39. 1. REceive & react (or not) 
 to facts* that are coming its way 2. Publish new facts* asynchronously 
 to the rest of the world Event Driven Services
  40. 40. 1. REceive & react (or not) 
 to facts* that are coming its way 2. Publish new facts* asynchronously 
 to the rest of the world 3. Invert the control flow
 to minimize coupling and increase autonomy Event Driven Services
  41. 41. 1. REceive & react (or not) 
 to facts* that are coming its way 2. Publish new facts* asynchronously 
 to the rest of the world 3. Invert the control flow
 to minimize coupling and increase autonomy Event Driven Services *Facts == Immutable Events
  42. 42. Mutable State Is Fine But Needs To Be Contained And Non Observable
  43. 43. Publish Facts To Outside World
  44. 44. ✴ Maintains Integrity & consistency ✴ Is our Unit of Consistency ✴ Is our Unit of Failure ✴ Is our Unit of Determinism ✴ Is fully Autonomous The Aggregate
  45. 45. Event Driven Services
  46. 46. Event Driven Services
  47. 47. Event Driven Services Command
  48. 48. Event Driven Services Command
  49. 49. Event Driven Services Command
  50. 50. Event Driven Services Command Event Event Stream
  51. 51. Event Driven Services Command Event EventEvent Event Stream
  52. 52. Event Driven Services Command Event EventEvent Event Stream
  53. 53. Event Driven Services Command Event EventEvent Event Stream
  54. 54. Event Driven Services Command Event EventEvent Event Stream
  55. 55. Event Driven Services Command Event EventEvent Event Stream
  56. 56. Event Driven Services Eventual Consistency Command Event EventEvent Event Stream
  57. 57. Event Stream Use The
  58. 58. Event Stream Use The as the communication fabric
  59. 59. Event Stream Use The
  60. 60. Event Stream Use The as the integration fabric
  61. 61. Event Stream Use The
  62. 62. Event Stream Use The as the replication fabric
  63. 63. Event Stream Use The
  64. 64. Event Stream Use The as the consensus fabric
  65. 65. Event Stream Use The
  66. 66. Event Stream Use The as the Persistence fabric
  67. 67. The Problem With CRUD Services
  68. 68. ✴ CRUD is fine for totally isolated data The Problem With CRUD Services
  69. 69. ✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency The Problem With CRUD Services
  70. 70. ✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency ➡ Is hard ⇨ No Joins The Problem With CRUD Services
  71. 71. ✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency ➡ Is hard ⇨ No Joins ➡ Has ad-hoc & weak guarantees The Problem With CRUD Services
  72. 72. ✴ CRUD is fine for totally isolated data ✴ But, cross CRUD services consistency ➡ Is hard ⇨ No Joins ➡ Has ad-hoc & weak guarantees The Problem With CRUD Services
  73. 73. “In general, application developers simply do not implement large scalable applications assuming distributed transactions” - Pat Helland Life Beyond Distributed Transactions - Pat Helland
  74. 74. “Two-phase commit is the anti-availability protocol.” - Pat Helland Standing on Distributed Shoulders of Giants - Pat Helland
  75. 75. STRONG Consistency Is the wrong default In distributed systems
  76. 76. STRONG Consistency Is the wrong default In distributed systems What can we do?
  77. 77. Eventual Consistency We have to rely on
  78. 78. Eventual Consistency We have to rely on But relax—it’s how the world works
  79. 79. Embrace Reality We Need to
  80. 80. Embrace Reality We Need to Not Fight it
  81. 81. Information Has Latency
  82. 82. Information Is Always From the Past
  83. 83. Welcome To The Wild Ocean Of Non Determinism Distributed Systems
  84. 84. “In a system which cannot count on distributed transactions, the management of uncertainty must be implemented in the business logic.” - Pat Helland Life Beyond Distributed Transactions, Pat Helland (2007) We Need To Model Uncertainty
  85. 85. Events Can Lead To Greater Certainty
  86. 86. “An autonomus component can only promise its own behavior.” “Autonomy makes information local, leading to greater certainty and stability.” - Mark Burgess In Search of Certainty, Thinking in Promises - Mark Burgess
  87. 87. Events Can Help Us Craft Autonomous Islands Of Determinism
  88. 88. Data on the inside vs Data on the outside - Pat Helland
  89. 89. Data on the inside vs Data on the outside - Pat Helland Inside Data Our current present ⇨ state
  90. 90. Data on the inside vs Data on the outside - Pat Helland Inside Data Our current present ⇨ state Outside Data Blast from the past ⇨ Events/facts
  91. 91. Data on the inside vs Data on the outside - Pat Helland Inside Data Our current present ⇨ state Outside Data Blast from the past ⇨ Events/facts Between Services Hope for the future ⇨ commands
  92. 92. A system of microservices is a never ending stream towards convergence
  93. 93. There Is No Now A system of microservices is a never ending stream towards convergence
  94. 94. Resilience is by Design Photo courtesy of FEMA/Joselyne Augustino
  95. 95. Events Can Help Us Manage Failure Instead Of Trying To Avoid It
  96. 96. Requirements for a Sane Failure Model 1. Contained—Avoid cascading failures 2. Reified—as Events 3. Signalled—Asynchronously 4. Observed—by 1-N 5. Managed—Outside failed Context Failures need to be
  97. 97. Event Based Persistence
  98. 98. You can use CRUD Together with Event Streams To get an internally consistent Materialized View
  99. 99. Service B Service A You can use CRUD Together with Event Streams To get an internally consistent Materialized View
  100. 100. Service B Service A CRUD You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD
  101. 101. Service B Service A TABLE A CRUD TABLE B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD
  102. 102. Service B Service A TABLE A CRUD TABLE B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD
  103. 103. Service B Service A TABLE A CRUD TABLE B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Atomic Update & event
  104. 104. Service B Service A TABLE A CRUD TABLE B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Atomic Update & event
  105. 105. Service C Service B Service A TABLE A CRUD TABLE B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Atomic Update & event
  106. 106. Service C Service B Service A TABLE A CRUD TABLE B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Atomic Update & event
  107. 107. Service C Service B Service A TABLE A CRUD TABLE B JOINS Table A Table B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Atomic Update & event
  108. 108. Service C Service B Service A TABLE A CRUD TABLE B JOINS Table A Table B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Read Only Atomic Update & event
  109. 109. Service C Service B Service A Eventual Consistency TABLE A CRUD TABLE B JOINS Table A Table B You can use CRUD Together with Event Streams To get an internally consistent Materialized View CRUD Read Only Atomic Update & event
  110. 110. “Update-in-place strikes systems designers as a cardinal sin: it violates traditional accounting practices that have been observed for hundreds of years.” - jim Gray The Transaction Concept, Jim Gray (1981)
  111. 111. “The truth is the log. The database is a cache of a subset of the log.” - Pat Helland Immutability Changes Everything, Pat Helland (2015)
  112. 112. Event Logging The Bedrock
  113. 113. Event Sourcing A Cure For the Cardinal Sin
  114. 114. Event Sourced Services
  115. 115. Happy Path Event Sourced Services
  116. 116. Happy Path Event Sourced Services 1) Receive and verify Command (“ApprovePayment”)
  117. 117. Happy Path Event Sourced Services 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”)
  118. 118. Happy Path Event Sourced Services 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log
  119. 119. Happy Path Event Sourced Services 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state
  120. 120. Happy Path Event Sourced Services 5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state
  121. 121. SAD Path - recover from failure Happy Path Event Sourced Services 5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state
  122. 122. SAD Path - recover from failure Happy Path Event Sourced Services 5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state 1) Rehydrate Events from Event Log
  123. 123. SAD Path - recover from failure Happy Path Event Sourced Services 5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state 1) Rehydrate Events from Event Log 2) Update internal component state
  124. 124. SAD Path - recover from failure Happy Path Event Sourced Services 5) Run side-effects (approve the payment) 2) Create new Event (“PaymentApproved”) 1) Receive and verify Command (“ApprovePayment”) 3) Append Event to Event Log 4) Update internal component state 1) Rehydrate Events from Event Log 2) Update internal component state Memory Image
  125. 125. Event Sourcing
  126. 126. Event Sourcing ✴ One single Source of Truth with All history
  127. 127. Event Sourcing ✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State)
  128. 128. Event Sourcing ✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State) ✴ Avoids the Object-relational mismatch
  129. 129. Event Sourcing ✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State) ✴ Avoids the Object-relational mismatch ✴ Allows others to Subscribe to state changes
  130. 130. Event Sourcing ✴ One single Source of Truth with All history ✴ Allows for Memory Image (Durable In-Memory State) ✴ Avoids the Object-relational mismatch ✴ Allows others to Subscribe to state changes ✴ Has good Mechanical sympathy
 (Single Writer Principle etc.)
  131. 131. ✴ Unfamiliar model ✴ Versioning of events ✴ Deletion of events (legal or moral reasons) Disadvantages Of Using Event Sourcing
  132. 132. Untangle Your Read and Write Models With CQRS
  133. 133. CQRS
  134. 134. CQRS
  135. 135. CQRS Write Side Model Commands
  136. 136. CQRS Write Side Model Write to Event Log Commands
  137. 137. CQRS Write Side Model Events Write to Event Log Commands
  138. 138. CQRS Write Side Model Events Events Write to Event Log Commands
  139. 139. CQRS Read Side Model Write Side Model Events Events Read Data Store Write to Event Log Commands
  140. 140. CQRS Read Side Model Write Side Model Events Queries Events Read Data Store Write to Event Log Commands
  141. 141. CQRS Read Side Model Write Side Model Events Queries Events Read Data Store Write to Event Log Eventual Consistency Commands
  142. 142. ✴ Temporal Decoupling ✴ Increased Resilience ✴ Increased Scalability ✴ Allows for Multiple Event subscribers ✴ Allows for Polyglot Persistence Advantages Of Using CQRS
  143. 143. ✴ More infrastructure to manage ✴ 2 data models (or more) to maintain ✴ Eventual Consistency Disadvantages Of Using CQRS
  144. 144. EventsAllow Us To Manage Time
  145. 145. “Modelling events forces you to have a 
 temporal focus on what’s going on in the system. Time becomes a crucial factor of the system.” - Greg Young A Decade of DDD, CQRS, Event Sourcing, Greg Young (Presentation from 2016)
  146. 146. ✴ Event is a snapshot in time ✴ Event ID is an index for time ✴ Event Log is our full history The database of Our past The path to Our present Event Sourcing Allows Us To Model Time
  147. 147. Event Sourcing Allows For Time Travel
  148. 148. Event Sourcing Allows For Time Travel
  149. 149. Event Sourcing Allows For Time Travel ✴Replay the log for historic debugging ✴Replay the log for auditing & traceability ✴Replay the log on failure ✴Replay the log for replication
  150. 150. We Can Even Fork the Past ...Or Join Two Distinct Pasts
  151. 151. Key Takeaways
  152. 152. Key Takeaways Events-First design helps you to: ✴ reduce risk when modernizing applications ✴ Move Faster towards a Resilient and Scalable architecture ✴ Design autonomous services ✴ Balance Certainty and Uncertainty
  153. 153. Key Takeaways Events-First design helps you to: ✴ reduce risk when modernizing applications ✴ Move Faster towards a Resilient and Scalable architecture ✴ Design autonomous services ✴ Balance Certainty and Uncertainty Event Logging allows you to: ✴ AVOID CRUD and ORM ✴ take control of your system’s history ✴ time-travel ✴Balance Strong and eventual consistency
  154. 154. http://akka.io
  155. 155. Learn More Download my latest book for free at: bit.ly/reactive-microsystems

×