CQRS + Event Sourcing

  • 1,510 views
Uploaded on

CQRS + Event Sourcing @Frankfurt 2013

CQRS + Event Sourcing @Frankfurt 2013

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,510
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
46
Comments
0
Likes
8

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. CQRS + ES DDD
  • 2. • Intro! • CAP • CQRS • Event Driven • Event Sourcing / Event Store • Appendix
  • 3. performance trouble? scalability trouble? big large data? high frequency data? complex business domain?
  • 4. performance problems if your system is slow for a single user
  • 5. scalability problems if your system is fast for a single user but slow under heavy load
  • 6. system level performance maximal throughput acceptable latency
  • 7. system level requirements • fast response time = high availability • slow response time = low availability • instant (secure) consistency = ACID (Atomicity, Consistency, Isolation und Durability) • time window consistency = BASE (Basically Available, Soft state, Eventual consistency)
  • 8. common mistakes • The network is fail-safe • The latency time is equal to zero • The data throughput is infinite • The network is secure • The network topology will not change • There is always only one network administrator • The cost of data transport can be calculated as zero, • The network is homogeneous
  • 9. • Intro • CAP! • CQRS • Event Driven • Event Sourcing / Event Store • Appendix
  • 10. CAP You can only pick 2 Consistency Availability Partition tolerance At a given point in time!
  • 11. „…a distributed system cannot satisfy all three of these guarantees at the same time…“ – Brewer
  • 12. Centralized Systems In a centralized system (RDBMS etc.) we don’t have network partitions e.g. P in CAP you GET BOTH: Consistency Availability
  • 13. Distributed Systems In a distributed system we have network partitions e.g. P in CAP You can get to ONLY PICK ONE: Consistency Availability
  • 14. CAP in practice • … there are only 2 types of systems • • • CP! AP! … there is only one choice. In case of existing network partitions, what do you prefer? 1. Consistency 2. Availability
  • 15. BASE Eventual consistency Consistency of data is seen as a state which is eventually attained.
  • 16. Where are you? CA CP AP
  • 17. system level different needs easy & fast flexible modern task driven SOFTWARE SYSTEM valid correct solid data driven
  • 18. UI usability different needs Data-Driven Task-based
  • 19. Data Model different needs • • • • • • Reporting / Search? State Changes? Business behaviors and rules? Algorithms / Calculations? ORM + rich domain model is a anti pattern! The reading of an object ended very often in charge of the entire database.
  • 20. Storage different needs • • • When do you need ACID? When Eventually Consistent better fit? Should Reporting / Search strictly consistent? Scaling out READS to a RDBMS is hard (sharing/replication) Scaling out WRITES to a RDBMS is impossible!
  • 21. So… one fits all? „A single model cannot be appropriate for reporting, searching and transactional behavior.“ –Greg Young 2008
  • 22. needs single system RDBMS Data Analytics CA CP multiple systems Data Administration Bank Service / ATM NoSQL AP Web-/Mobile End-Users DNS Web Caching
  • 23. CA AP CP
  • 24. Do you really need a RDBMS? Sometimes! But many times we don’t!
  • 25. application building (today) ACID <vs.> EVENTUAL CONSISTENCY CUD <vs.> BEHAVIORAL DOMAIN READ <vs.> SEARCH / REPORTS DATA DRIVEN <vs.> TASK BASED
  • 26. Think about your data: Different data need different guarantees
  • 27. • Intro • CAP • CQRS! • Event Driven • Event Sourcing / Event Store • Appendix
  • 28. CQRS command query responsibility segregation
  • 29. conceptual strictly segregates the responsibility independent optimization and specialization asymmetrical execution of commands & queries
  • 30. „Do your domain experts speak with only four verbs (Create Read Update Delete)?“
  • 31. CRUD & OOP
  • 32. Bertrand Meyer's CQS split interfaces into state changes Commands side-effect free Queries
  • 33. simple pattern • side-effect-free query/read access • autonomous commands represent behavioral state changes
  • 34. That’s it! „A single model cannot be appropriate for reporting, searching and transactional behavior.“ –Greg Young 2008
  • 35. CRUD <—> CQRS
  • 36. CRUD <—> CQRS
  • 37. conceptual @Microsoft
  • 38. CQRS execution of commands represent changes in the state of the software system side-effect free processing of queries
  • 39. scaling benefits host the two services (read / write) differently we can host the read service on 25 servers write service on one server
  • 40. read side benefits • top-down driven and optimized view data model • denormalized schema - no joins • easy queries (select * from DB) • no mapping • no ORM mismatches
  • 41. write side benefits • behavioral driven • exposes only functionality • no read access • (event driven) state machine • forms a consistency boundary
  • 42. • Intro • CAP • CQRS! • Event Driven ! • Event Sourcing / Event Store • Appendix
  • 43. Event-Driven @Microsoft
  • 44. Events „It’s really become clear to me in the last couple of years that we need a new building block and that is the Domain Events.“ –Eric Evans, 2009 „State transitions are an important part of our problem space and should be modeled within our domain“ –Greg Young, 2008
  • 45. *syntax notes commands events domain events business events notifications Messages
  • 46. events something that has happened in the past CustomerHandled CargoShipped
  • 47. commands something that should do named in imperative ChangeCustomerLocale MakeCustomerPreferred
  • 48. storing structure
  • 49. events & commands • • • • • • • • • uniquely identifiable self contained pure data structure - no behavior Observable Time relevant non-blocking - async execution conceptual immutable e.g. new, deep copy, clone events represent serialized function calls conceptual duality between function calls and messages
  • 50. flow
  • 51. flow in a nutshell • Write side receive Commands and publish Events • All state changes are represented by Business Events • Read side is updated as a result of the published Events • All Queries go directly to the Reporting, the Business-Logic is not involved
  • 52. flow
  • 53. Common benefits • Focus to Business • allow to manage business and technology risks separately • Technology agnostic, platform neutral and easy externals integration • Fully encapsulated domain that only exposes behavior • Queries do not use the business model • No object-relational (ORM) impedance mismatch • Scalable and Flexible and Testable • Reduced complexity and simplify system architecture
  • 54. benefits Event-Driven • Explicit error/failure modeling • Easy integration with external systems • Location Transparency • Reactive Behavior • Bullet-proof auditing and historical tracing
  • 55. Event Aggregator • • • publish / subscribe pattern observe events dispatch / invoke functions sync or async
  • 56. Command-Handler • • execute functions by command (type) integration point for cross-cutting concerns (logging, transactions, validating, authorization)
  • 57. Event-Handler • • • execute functions by event (type) view or data model generation schema denormalizer
  • 58. trade-offs • initially feels complex • Consistency Boundaries (isn’t a top level architecture) • Monitoring / Heartbeats • Stale Data • Collaborative Domains • Data Integration • Modeling with time • Manage idempotency and event de-duplication
  • 59. • Intro • CAP • CQRS • Event Driven • Event Sourcing / Event Store! • Appendix
  • 60. Event Sourcing
  • 61. Event Sourcing Storing Deltas
  • 62. Event Sourcing replaying events
  • 63. Event Store Event Source record Produce Event Store #9 meta data time stamp payload identity version #5 #1 Consume replay append only log
  • 64. Event Sourcing Every state change is materialized in an Event All Events are sent to an EventProcessor EventProcessor stores all events in an Event Log System can be reset and Event Log replayed Many different EventListeners can be added to EventProcessor (or listen directly on the Event Log) No need for ORM, just persist the Events
  • 65. Event Sourcing
  • 66. recover state f(state, event) → state 1. EventProcessor record & replay all events in the context of a consistency boundary 2. projections - loop over a series of events and aggregate to a state
  • 67. Event Processor Restoring State from Deltas • Event Processor are tracking events as to what has changed • Current state is constructed by replaying events • Data is not persisted in a structure but as a series of transactions • No ORM is needed
  • 68. Projections Query language of Event Streams Filter, Map, Reduce over Event Streams Projection is about deriving current state from the stream of events Given the stream of events, we can transform them to any structural representation
  • 69. Event Sourcing benefits • • • • • • • Recording all state changes is a business transaction history Event History + Time Machine Replay summary of total records is the last state any regeneration of models projections as a useful event stream query language time related analysis and statistics (forecastings)
  • 70. Event Sourcing benefits • No object-relational impedance mismatch • time ordered series of business transactions • Support future ways of looking at data • Performance and scalability • Testability • Reconstruct production scenarios • Long-lived processes, tracking and auditability
  • 71. • Intro • CAP • CQRS • Event Driven • Event Sourcing / Event Store • Appendix
  • 72. Thanks Questions?
  • 73. appendix • messaging pattern • idempotency and de-duplication
  • 74. Messaging Pattern Publish / Subscribe
  • 75. Messaging Pattern Store and Forward
  • 76. Messaging Pattern Request-Reply
  • 77. Messaging Pattern Point-to-Point
  • 78. Idempotency f(x) = f(f(x)) request twice means the same outcome
  • 79. Idempotency • natural idempotent operations are mostly easy • • • nicely for state machines with alter state without side-effects turn the light ON - duplicate calls generate some result - the light is on not naturally idempotent operations are hard • only process ones that I see are new • Balance += Amount - increases balance again and again
  • 80. Idempotency • de-duplicated messages by query a log store • check incoming message by an identifier agains a log store and reject if a identifier match • check against a correlation identifier (e.g. ETags, TransactionID) • check against a computed HASH of the complete message (like GIT commits)
  • 81. Idempotency • using a buffer • time throttling