Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Richer data-history-event-sourcing

1,368 views

Published on

With the rise of opinionated, full-featured frameworks and Object-Relational Mappers, we – as software developers – will generally build systems where our data is directly linked to the current state of our application; one row in the database equates to one entity’s state within the system. Only ever knowing the current state of the data is adequate for many systems, but imagine the possibilities if one had access to the states of all entities at any given time, and how that state was reached. It is just as important to understand the current state as it is to understand the steps that it took to reach the current state.

Enter Event Sourcing: instead of persisting the current state of our entities, we store historical events about our data. This pattern changes how we store and process our data, but is surprisingly lightweight and performant. In this talk I will present the basic concepts behind Event Sourcing and the positive implications it has on usability, visualization, and analytics within our applications. We’ll see how naturally it couples with the Event-oriented world of Reactive systems. Finally, we’ll examine some practical use cases and when one should and should not consider implementing the pattern. Event Sourcing will completely change how you think about your data.

Published in: Technology
  • Be the first to comment

Richer data-history-event-sourcing

  1. 1. THIRDCHANNEL Hi.
  2. 2. First, A Quick Scenario
  3. 3. THIRDCHANNEL Balance: $100
  4. 4. THIRDCHANNEL Balance: -$100
  5. 5. THIRDCHANNEL
  6. 6. THIRDCHANNEL
  7. 7. THIRDCHANNEL
  8. 8. THIRDCHANNEL
  9. 9. THIRDCHANNEL Richer Data History With Event Sourcing Steve Pember CTO, ThirdChannel
  10. 10. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It • Implementation (Theory) • Implementation (Reality) • Event Storage • Querying • Demo
  11. 11. What Is Event Sourcing?
  12. 12. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern
  13. 13. THIRDCHANNEL
  14. 14. Instead, Store Facts
  15. 15. THIRDCHANNEL
  16. 16. THIRDCHANNEL
  17. 17. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern • Series of Facts • Aggregates << Events
  18. 18. Event: Something Which Has Occurred In The System
  19. 19. Aggregate: The Object of Events’ Affections
  20. 20. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern • Series of Facts • Aggregates << Events • Purely Additive
  21. 21. There is No Delete
  22. 22. THIRDCHANNEL
  23. 23. Full History of the Aggregates
  24. 24. THIRDCHANNEL
  25. 25. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern • Series of Facts • Aggregates << Events • Purely Additive • Optimization: Snapshots
  26. 26. THIRDCHANNEL • snapshot photo
  27. 27. THIRDCHANNEL
  28. 28. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern • Series of Facts • Aggregates << Events • Purely Additive • Optimization: Snapshots • Example: Shopping Cart
  29. 29. THIRDCHANNEL
  30. 30. THIRDCHANNEL
  31. 31. THIRDCHANNEL
  32. 32. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern • Series of Facts • Aggregates << Events • Purely Additive • Optimization: Snapshots • Example: Shopping Cart • Transient Objects
  33. 33. Objects are Transient Derivatives of Your Events
  34. 34. THIRDCHANNEL Event Sourcing • Alternative Storage Pattern • Series of Facts • Aggregates << Events • Purely Additive • Optimization: Snapshots • Example: Shopping Cart • Transient Objects • Natural
  35. 35. THIRDCHANNEL
  36. 36. THIRDCHANNEL
  37. 37. THIRDCHANNEL Questions So Far? • Events & Aggregates • Snapshots • Additive, loss-less model • Transient Derivatives
  38. 38. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges
  39. 39. “I Think I Get What You’re Laying Down, But It Still Seems Bananas”
  40. 40. So much work!
  41. 41. “That’s gotta be a ton of CPU overhead”
  42. 42. “Where’d my Models go? I can’t live without my MVC!”
  43. 43. THIRDCHANNEL Event Sourcing Challenges • Additional Work To Apply • Additional CPU Processing • Non-Traditional Concept of Models
  44. 44. THIRDCHANNEL Event Sourcing Challenges • Additional Work To Apply • Additional CPU Processing • Non-Traditional Concept of Models • More Storage Required VS non-ES
  45. 45. THIRDCHANNEL
  46. 46. THIRDCHANNEL Event Sourcing Challenges • Additional Work To Apply • Additional CPU Processing • Non-Traditional Concept of Models • More Storage Required VS non-ES • Reduced Database Level Constraints
  47. 47. Make Use of Transactional Code
  48. 48. THIRDCHANNEL Event Sourcing Challenges • Additional Work To Apply • Additional CPU Processing • Non-Traditional Concept of Models • More Storage Required VS non-ES • Reduced Transactions / Database Level Constraints • May Be Confusing For Junior Engineers
  49. 49. THIRDCHANNEL
  50. 50. THIRDCHANNEL
  51. 51. So Why Use ES?
  52. 52. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It
  53. 53. THIRDCHANNEL Why Event Sourcing • More Than an Audit Log
  54. 54. Audit Logs Tell the History Events Tell the Intent of History
  55. 55. ES is also a perfect Audit Log
  56. 56. THIRDCHANNEL Why Event Sourcing • More Than an Audit Log • Data Storage is Inexpensive
  57. 57. THIRDCHANNEL Why Event Sourcing • More Than an Audit Log • Data Storage is Inexpensive • Used By All Long-Running Businesses
  58. 58. THIRDCHANNEL
  59. 59. All Business Problems Can Be Modeled With Event Sourcing
  60. 60. THIRDCHANNEL Why Event Sourcing • More Than an Audit Log • Data Storage is Inexpensive • Used By All Long-Running Businesses • Only Structural Model That Does Not Lose Information
  61. 61. Remember: There Is No Delete
  62. 62. THIRDCHANNEL Why Event Sourcing • More Than an Audit Log • Data Storage is Inexpensive • Used By All Long-Running Businesses • Only Structural Model That Does Not Lose Information • Simplified Testing and Debugging
  63. 63. Unit Test Events, Assert In Integration Tests
  64. 64. Time Travel Eases Debugging
  65. 65. THIRDCHANNEL
  66. 66. THIRDCHANNEL
  67. 67. THIRDCHANNEL
  68. 68. THIRDCHANNEL
  69. 69. THIRDCHANNEL
  70. 70. THIRDCHANNEL
  71. 71. THIRDCHANNEL Why Event Sourcing • More Than an Audit Log • Data Storage is Inexpensive • Used By All Long-Running Businesses • Only Structural Model That Does Not Lose Information • Simplified Testing and Debugging • Ideal for Business Analysis
  72. 72. Future-Proof your Data
  73. 73. THIRDCHANNEL
  74. 74. But With Event Sourcing…
  75. 75. –Johnny Appleseed “Type a quote here.”
  76. 76. THIRDCHANNEL
  77. 77. Current State is a Projection
  78. 78. A 3C Example
  79. 79. THIRDCHANNEL
  80. 80. Still…
  81. 81. THIRDCHANNEL
  82. 82. THIRDCHANNEL Have you ever built? • Versioning of objects • An ‘undo’ action • Audit Log • Tracking object value changes over time? • Time series?
  83. 83. THIRDCHANNEL Questions so far?
  84. 84. Next Up: Implementation
  85. 85. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It • Implementation (Theory)
  86. 86. THIRDCHANNEL Implementation (Theory) • Simple Base Objects
  87. 87. Three Base Objects
  88. 88. THIRDCHANNEL
  89. 89. THIRDCHANNEL
  90. 90. Past-Tense Event Names
  91. 91. THIRDCHANNEL
  92. 92. THIRDCHANNEL Implementation (Theory) • Simple Base Objects • Aggregate and Event SubClasses Have Transient Properties
  93. 93. THIRDCHANNEL
  94. 94. THIRDCHANNEL
  95. 95. THIRDCHANNEL Implementation (Theory) • Simple Base Objects • Aggregate and Event SubClasses Have Transient Properties • Aggregates Receive and Play Events
  96. 96. Events Modify Aggregates
  97. 97. THIRDCHANNEL Implementation (Theory) • Simple Base Objects • Aggregate and Event SubClasses Have Transient Properties • Aggregates Receive and Play Events • Aggregates Require Distinction Between New and Historical Changes
  98. 98. THIRDCHANNEL
  99. 99. THIRDCHANNEL Implementation (Theory) • Simple Base Objects • Aggregate and Event SubClasses Have Transient Properties • Aggregates Receive and Play Events • Aggregates Require Distinction Between New and Historical Changes • Event Service Layer Required
  100. 100. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It • Implementation (Theory) • Implementation (Reality)
  101. 101. THIRDCHANNEL Implementation (Reality) • Freely Add New Transient Properties
  102. 102. THIRDCHANNEL Implementation (Reality) • Freely Add New Transient Properties • Be Wary of Modifying Events
  103. 103. THIRDCHANNEL Implementation (Reality) • Freely Add New Transient Properties • Be Wary of Modifying Events • Be Wary of Removing Aggregate Transients
  104. 104. THIRDCHANNEL Implementation (Reality) • Freely Add New Transient Properties • Be Wary of Modifying Events • Be Wary of Removing Aggregate Transients • Snapshot Rarely
  105. 105. Plus so much more…
  106. 106. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It • Implementation (Theory) • Implementation (Reality) • Persistance
  107. 107. THIRDCHANNEL Persistence & Event Storage • No Real Need for an ORM
  108. 108. THIRDCHANNEL
  109. 109. THIRDCHANNEL Persistence & Event Storage • No Real Need for an ORM • Try a Library
  110. 110. THIRDCHANNEL Event Sourcing Libraries • NEventStore (.NET) • ProophEventStore (PHP) • Akka Persistence (JVM)
  111. 111. THIRDCHANNEL Persistence & Event Storage • No Real Need for an ORM • Try a Library • Database Agnostic
  112. 112. THIRDCHANNEL
  113. 113. THIRDCHANNEL
  114. 114. THIRDCHANNEL
  115. 115. THIRDCHANNEL Persistence & Event Storage • No Real Need for an ORM • Database Agnostic • Consider Using a Relational DB
  116. 116. THIRDCHANNEL Questions?
  117. 117. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It • Implementation (Theory) • Implementation (Reality) • Persistance • Querying
  118. 118. THIRDCHANNEL Querying Events • All Queries are Projections, including Current State
  119. 119. THIRDCHANNEL Querying Events • All Queries are Projections, including Current State • Returning Current State is Easy
  120. 120. THIRDCHANNEL
  121. 121. THIRDCHANNEL Querying Events • All Queries are Projections, including Current State • Returning Current State is Easy • Other Projections Can Be Tough
  122. 122. Query across Event Types
  123. 123. THIRDCHANNEL
  124. 124. THIRDCHANNEL Querying Events • All Queries are Projections, including Current State • Returning Current State is Easy • Other Projections Can Be Tough • Try Reactive Streams
  125. 125. THIRDCHANNEL
  126. 126. Use Streams to Filter Events in Code
  127. 127. THIRDCHANNEL
  128. 128. THIRDCHANNEL Querying Events • All Queries are Projections, including Current State • Returning Current State is Easy • Other Projections Can Be Tough • Try Reactive Streams • Initial Projection Work May Require Dev Time
  129. 129. Find Someone with a Statistics Background
  130. 130. THIRDCHANNEL Querying Events • All Queries are Projections, including Current State • Returning Current State is Easy • Other Projections Can Be Tough • Try Reactive Streams • Initial Projection Work May Require Dev Time • Consider Cacheable Query Layers
  131. 131. Separate Event Storage / Querying Services
  132. 132. THIRDCHANNEL
  133. 133. THIRDCHANNEL Agenda • Event Sourcing • Event Sourcing Challenges • Don’t Worry, ES Is Worth It • Implementation (Theory) • Implementation (Reality) • Persistance • Querying • Demo Time
  134. 134. Pray to the demo gods.
  135. 135. THIRDCHANNEL
  136. 136. In Summary
  137. 137. Event Sourcing -> Additive Only, Lossless. Can Be Tricky To Work With
  138. 138. Not for your Mom’s Blog
  139. 139. Thank You!
  140. 140. THIRDCHANNEL Any Questions? Steve Pember @svpember www.thirdchannel.com
  141. 141. THIRDCHANNEL Image Credits • BOA: http://marketmadhouse.com/wp-content/uploads/2014/11/BofA.jpg • Khaaaaan: http://d1oi7t5trwfj5d.cloudfront.net/38/0d9b80ae5311e1bcc4123138165f92/file/5-things-you-might-not-know-about-star-trek-ii-wrath-of-khan-30th-anniversary.jpg • tribbles: http://de.memory-alpha.org/wiki/Tribble • bank ledger: http://www.iowall.net/accounting-ledger-book.html • ORM: https://guides.codepath.com/android/ActiveAndroid-Guide • hoarders: http://www.aetv.com/hoarders • snapshot stream: https://geteventstore.com/ • office space bobs: http://galleryhip.com/office-space-bobs.html • Nye, Obama, Tyson photo: http://en.wikipedia.org/wiki/Selfie • Printing Press: http://www.directindustry.com/prod/koenig-bauer-ag/offset-printing-presses-high-volume-40462-602574.html

×