Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CQRS why, what, how

19,059 views

Published on

This talk covers WHY we might want to look at CQRS, by first looking at the kinds of problems we see with the current architectural approaches. We will look at WHAT CQRS is, how it works well with DDD, look at staleness of data in collaborative domains and how we can use this to our advantage. We will look at the building blocks for CQRS, how to test it and how to build a bigger solution the involves communication between different parts of the solution/bounded contexts. We will look at how we can use CQRS to scale the solution (by looking at the CAP theorem and the BASE properties of distributed computing). Finally we will look at how we can approach legacy system migration by looking at 4 different alternatives to rewriting them from scratch. Key words are for this is published language, Anti Corruption layer and an Event Driven Architecture (which might utilize CQRS internally).

Published in: Technology

CQRS why, what, how

  1. 1. CQRSWHY? WHAT? HOW? JEPPE CRAMON - 2012
  2. 2. WHAT PROBLEMS ARE WE TRYING TO SOLVE?
  3. 3. Complexity of code and solution Typical causes: • Too many layersComplexity • Big data models • Anemic domain model • Focus on frameworks instead of on the domain • Scalability not considered at the core of the design (scalability gets hacked in too late) Time
  4. 4. Typical One size fits all ”architecture” Client DTO DTO Remote Facade Application Service Domain Domain Object Object Data Access Layer Data Storage
  5. 5. Many perspectives on data Online Ordering System Product Pricing Unit Price Promotional Price Promotion End Date Stock Keeping Unit (SKU) Quantity On Hand (QOH) Location Code InventoryCustomer Price Quantity Ordered Name Sales What about the lifecycles for the data?
  6. 6. One big model to capture it ALL …. ….. …..….…..….. …. ….. ….. …. ….. …..
  7. 7. Result: Our queries get messy
  8. 8. ONE MODEL TO RULE THEM ALL ONE MODEL TO FIND THEM ONE MODEL TO BRING THEM ALL AND IN THE DARKNESS BIND THEM
  9. 9. The database determines our scalability
  10. 10. And databases often scale poorly To solve the performance problems for a few, you have to upgrade it all
  11. 11. Alternatively we can use Read replicas Slave Slave Master Slave Slave You’re now eventually consistent
  12. 12. Or introduce caching… ?What’s your synchronization mechanism?
  13. 13. SOA / Integration is often an after thought
  14. 14. THE Solution?
  15. 15. Same procedure as last year?Same procedure as every year!!!
  16. 16. Open for new ideas? Poul, you idiot, whats that!?You should only bring things we can eat!
  17. 17. IT exists to support the business by support various usecasesUsecases can be categorized as either• ”User” intent to manipulate information• ”User” intent to find and read informationWe already support this in OO at a Property level:• Setter (or mutator) methods Manipulate data• Getter (or accessor) methods Read data Let’s raise it to a higher Level
  18. 18. CQS Separation of functions that write & functions that readFunctions that write are called Command methodsand must not return a valueFunctions that read are called Query methods andmust have no side effects
  19. 19. CQSCommands – Change data UI Application Domain DataQueries – Ask for data (no side effects) UI Application Data
  20. 20. CQS from a code perspectivepublic class CustomerService{ // Commands void MakeCustomerPreferred(CustomerId) void ChangeCustomerLocale(CustomerId, NewLocale) void CreateCustomer(Customer) void EditCustomerDetails(CustomerDetails) // Queries Customer GetCustomer(CustomerId) CustomerSet GetCustomersWithName(Name) CustomerSet GetPreferredCustomers()} From: https://gist.github.com/1964094
  21. 21. SO WHAT IS CQRS?CQRS is simply the creation of two objects where there was previously only one Greg Young
  22. 22. CQRS In its simplest formpublic class CustomerWriteService{ // Commands void MakeCustomerPreferred(CustomerId) void ChangeCustomerLocale(CustomerId, NewLocale) void CreateCustomer(CreateCustomer) void EditCustomerDetails(CustomerDetails)}public class CustomerReadService{ // Queries Customer GetCustomer(CustomerId) CustomerSet GetCustomersWithName(Name) CustomerSet GetPreferredCustomers()} From: https://gist.github.com/1964094
  23. 23. Commands A Command’s primary goal is to capture USER INTENT A Command supports a single usecase and targets a single AggregateCommands are stated in the imperative: – DoStuff – CreateCustomer – ShipOrder – AddComment – DeleteComment
  24. 24. That’s all you need to know Thanks for your attention 
  25. 25. There’s always more than onemodel at play in any big project But when code based on multiple models is combined
  26. 26. … then the code becomes filled with errors, unreliable and difficult to understand Communication between team members becomes confusing. It is often unclear in what context a model should NOT be used!
  27. 27. Context is lacking
  28. 28. CONTEXTThe setting in which a word or a statement appears that determines its meaning
  29. 29. Smaller models & clear data ownership Online Ordering System Pricing Product DDD: SKU Bounded Unit Price Promotional Price Context … Sales Product SOA: SKU Name Entity identity Pricing Service Description Quantity Ordered … Inventory Product SKU QOH Location Code … Inventory Sales
  30. 30. Core DDD patternsBOUNDEDCONTEXT Repositories Factories Layered Architecture ENTITIES VALUE OBJECTS Modules Services AGGREGATES
  31. 31. Let’s first get the vocabulary right So we all speak the same UBIQUITOUS LANGUAGE
  32. 32. Entities Customer - ID/Social Security Number : NumberMotivators:Objects are defined by identity and NOT by their attributes.May have lifecycle, can change form by time and context (A Personthat becomes a father, who becomes a grand father, etc.) Focus is on Who and not on what!
  33. 33. Value Objects Customer - ID/Social Security Number : Number 1 AddressMotivators:Objects which describe things and are - street : Stringdefined by their attributes (not by their - zipCode : Stringidentity) and doesn’t have a lifecycle. Focus is on What and not on who!
  34. 34. Aggregates Account Invoice * RootWhat:• Cluster coherent Entities and Value Objects, with complex associations into Aggregates with well defined boundaries. *• Choose one entity to be root and control access to objects inside the boundary through the root. InvoiceLine• External objects hold references to the root• Aggregates only refer to other aggregates by identity (their id)Motivation:Control invariants and consistency through the aggregate root.Enables: Loading schemes, coarse grained locking, consistency/transactional boundariesfor DDDD
  35. 35. How have our ”architectures” evolved
  36. 36. Thin Layers of complexity Language of the database UI DataPro: Easy and simpleCon: Scales poorly for complex domains
  37. 37. Next step – more layers Language of Language of Language of getters and DTO’s ORM magic setters UI Application Domain Data ComplexityPro: Handles complex domains better by TimeSeparating usecase and domain logicCon: Increasing complexity. Risk of AnemicDomain model combined withtransaction-script.
  38. 38. Anemic Domain ModelThe Good parts The bad parts• All Order logic is called in ONE • Hard to test class • Hard to maintain• Lower entry level for junior • Code dupliction developers • An exposed domain model is like walking around without clothes – everyone can access your private parts and violate your invariants
  39. 39. We need a rich domain model Aggregate with Order as Aggregate Root• Order forms our Unit of consistency, which controls our invariants• Order and OrderLines are created, updated and deleted together
  40. 40. LET’S HAVE A LOOK AT A DIFFERENT TYPE OF ARCHITECTURE
  41. 41. Circular architecture Task based UI DomainRead Model Model
  42. 42. 1. Realization Commands and Queriessupport very different usecases
  43. 43. A single model cannot beappropriate for reporting, searching and transactional behavior Greg Young, 2008
  44. 44. 2. realizationQuery results contain only data – no logic• Give me the customer with his last 10 orders• Give me the customer with total sum of orders for the last year• Give me the complete order• Give me the shipping status for the order• Give me the customers last reviewSo why go through the domain layer? UI Application Data
  45. 45. Things to think about in relation to Queries• Why take the pain of performing JOINS and/or UNIONS of our nicely normalized Relational Model?• Why not having a completely denormalized data model (or View) to support our Query?• When we completely denormalize our data, do we really need a relational database or could we use a Key/Value- or Document Store?• Do we need our Read Data to up to the microsecond consistent with our Write data?
  46. 46. So how is this relevant in the real world? Let’s look at a real life scenario
  47. 47. Collaborative domains 1. Read 2. Read4. Update 3. Coffee time 5. Stale data
  48. 48. We’re working with stale data
  49. 49. We’re always working with stale data! + thinking time (1000-1500 ms) 20 ms 20 ms At this point our data is at least 120 ms stale 1 ms 1 ms + Thinking time 100 ms 100 ms 100 ms
  50. 50. LET’S USE STALE DATA TO OUR ADVANTAGE BY OFFLOADING THEDATABASE BY USING OUR READ MODELSCommands – Change data UI Application Domain Write modelQueries – Ask for data (no side effects) UI Application Read models
  51. 51. ALL WE NEED IS A GOOD WAY TO SYNCHRONIZE OUR WRITE AND READ MODELSCommands – Change data UI Application Domain Write modelQueries – Ask for data (no side effects) UI Application Read models
  52. 52. Let’s make the implicit explicit!The original DDD book left out a very important concept: Domain EventsWhich:• Signal that something happened• Closely aligned to the Domain Model• Are handled by a messaging system• They are in the past tense: – CustomerBilled – ParcelShipped – CustomerCreated – ReviewCreated – CommentAdded – CommentDeleted
  53. 53. Commands & Events• Commands mutate Aggregate state which results in one or more Events being issued/published. Command Event(s) AcceptOrder OrderAccepted ShipOrder OrderShipped AddComment CommentAdded QuarantineReview ReviewQuarantined UnquarantineReview ReviewUnquarantined
  54. 54. With Domain Events we now have amechanism to support our denormalized View/Query models
  55. 55. Commands, Events and Query Models UI ”Find all Accepted Orders” ”AcceptOrder” Query command Events Query model Domain model Read model Read model ”OrderAccepted” eventCommands are Imperative: DoStuff Events are Past tense: StuffDone
  56. 56. Messaging Architectures Most CQRS implementations see Commands and Events as (asynchronous) Messages public class CreateCustomer {Command public final Guid CustomerId; public final Name CustomerName; … } Difference: public class CustomerCreated INTENT { Event public final Guid CustomerId; public final DateTime CreationDateTime; public final Name CustomerName; }
  57. 57. CQRS Building blocks Command Command Modify Commands Domain Bus Handlers Command Services Read Write Sends Data store Repositories EventsClient Event Events Event Handlers Bus Queries Query Services Read store Query Results Query Handlers
  58. 58. LET’S TAKE IT A STEP FURTHER
  59. 59. Event Sourcing Aggregates track their own Domain Events and derive state from themOctober October October October October October 5 6 6 7 7 9 OrderCreated ProductAdded ProductAdded ProductRemoved ProductAdded OrderShipped
  60. 60. Full CQRS With EventSourcing Our singleCommands – Change data source of truth Commands Events UI Domain EventQueries – Ask for data Store Events UI Query Data Build SQL DB Document DB Graph DB
  61. 61. public class CustomerCommandHandler { private Repository<Customer> customerRepository; public CustomerCommandHandler(Repository<Customer> customerRepository) { this.customerRepository = customerRepository; } @CommandHandler public void handle(UnsignCustomer cmd) { Customer customer = repository.load(cmd.getCustomerId()); customer.unsign(); }}public class Customer { private boolean signedUp; public void unsign() { if (signedUp) { apply(new CustomerUnsignedEvent()); } } @EventHandler private void handle(CustomerUnsignedEvent event) { signedUp = false; }}
  62. 62. Testing CQRS BDD style@Testpublic void a_signedup_customer_can_unsign() { UUID customerId = UUID.randomUuid().toString(); FixtureConfiguration fixture = Fixtures.newGivenWhenThenFixture(); fixture.registerAnnotatedCommandHandler( new CustomerCommandHandler(fixture.createGenericRepository(Customer.class)) ); fixture.setAggregateIdentifier(customerId); fixture .given( TestFactory.customerCreatedEvent(customerId), TestFactory.customerSignedUpEvent(customerId) ) .when( TestFactory.unsignCustomerCommand(customerId) ) .expectEvents( TestFactory.customerUnsignedEvent(customerId) );}
  63. 63. CQRS / BDD Report generated based on the previous testScenario: A signedup customer can unsignGiven a Customer with id ”abc1234” thathas been created and a customer with id”abc1234” that is signedupWhen a request for Customer with id”abcd1234” to be unsigned is receivedThen the Customer with id ”abcd1234” isunsigned
  64. 64. SOA / Integration Commands, Queries and Events form natural integration interfacesSo integration is backed in from the beginning!
  65. 65. What about Scalability?CAP Theorem• Consistency: All nodes in the cluster see exactly the same data at any point in time• Availability: Failure of a node does not render the entire system inoperative• Partition tolerance: Nodes can still function when communication with other groups of nodes is lost You can’t have all three!
  66. 66. CAP theorem ACID systems are hard and expensive to scaleUnless you use Latency Pessimistic concerns Locking – all data is stale(and eventual consistentwhen delivered to the user) Strong  Weak  Eventual Consistency Source: http://bit.ly/QUurnY
  67. 67. BASE Basically Available Soft-state Eventually consistent Eventually Consistency levels: • Causal consistency: This involves a signal being sent from between application session indicating that a change has occurred. From that point on the receiving session will always see the updated value. • Read your own writes: In this mode of consistency, a session that performs a change to the database will immediately see that change, even if other sessions experience a delay. • Monotonic consistency: In this mode, A session will never see data revert to an earlier point in time. Once we read a value, we will never see an earlier value.See http://www.allthingsdistributed.com/2008/12/eventually_consistent.html for more
  68. 68. CQRS can give us BASE at the architectural level throughasynchronous Commands and Events This gives the business and IT control over where to spend money to scale our solution - instead of trying to buy a bigger database server.
  69. 69. CQRS requires a New UI paradigme Task-based/Inductive UI• A depart from the traditional WHAT UI (CRUD)• Task based UI’s focuses on HOW the user wants to use the application• Guides users through the work process• The UI becomes an intrinsic part of the design• The UI design directly affects our commands and thereby our transactional boundaries• Should be preferably use asynchronous Commands
  70. 70. What about Validation• Happens in the UI before Commands are sent• Typically validated using Read models• Helps to ensure that commands don’t often fail• Validates simple things (values ranges, unique keys/values)• Doesn’t enforce business logic rules
  71. 71. Circular architecture Task based UI DomainRead Model Event Event Event Model
  72. 72. Separation of Concerns• Different developer skills and roles for command and query parts• New Business Requirements ripples controllably between areas• No need for Getters & Setters in your domain model• Testability – Event based testing is very easy
  73. 73. CQRS is not top level architecture Somethings missing How do we collaborate?
  74. 74. Composite UI’s Online Ordering System Sales Pricing Inventory Composite UIWeb/Application UI tier Layered Data Access Layer Application Write model Read modelBackground server tier Storage tier
  75. 75. Composite UI - example
  76. 76. That covers data presentation What about transactions?
  77. 77. Next Step – Event Driven Architecture Online Ordering System Pricing Service Product SKU New SKU Unit Price Promotional Price Event … Web Shop Who (Composite UI) coordinates the sales process? PricingCustomers Sales Inventory Service (SAP) Product Product SKU SKU Name Order QOH Price Accepted Location Code Quantity Ordered … … Event Inventory Message Bus New SKU New SKU Event Event Sales
  78. 78. Introducing Sagas Online Ordering System Billing Service Order Accepted Customer BilledSales Service Order Shipping Accepted Order Process Manager Accepted (Saga) Customer Billed Ship Order Message Bus Shipping Service Ship Order
  79. 79. So what do we do? We have this dream
  80. 80. ”Nirvana” Sales Pricing Inventory Composite UIWeb/Application tier UI Layered Application Data Access Layer Write model Read modelBackground server tier Storage tier Billing Service Order Accepte d Customer Billed Sales Service Order Shipping Accepte Order Process Manager d Accepte d Customer Billed Ship Order Message Bus Shipping Service Ship Order
  81. 81. …and this realitySystem X System Y System Z SAP
  82. 82. Finding strategic value
  83. 83. Reimplementation Still relevant?
  84. 84. Perhaps a more viable solution? Big Ball Of Mud
  85. 85. What path to take?
  86. 86. 1. Bubble context
  87. 87. New Context Z Business Application Repository Repository Anti Corruption Layer Transformation Transformation X↔Z Coordinator Y↔ZSystem X Service Repository System Y Data
  88. 88. Bubbles burst• Functionality gets absorbed into the legacy systems• Maintenance becomes too expensive• Expansion becomes too cumbersome• People start bypassing the ACL (encapsulation breaks)• …• Performance becomes a problem
  89. 89. 2. ACL Synchronization New Context Z Business Application Repository Repository Data Anti Corruption Layer Asynchronous updates Transformation Transformation X↔Z Coordinator Y↔Z System X Service Repository System Y Data
  90. 90. 3. ACL backed Open Host service New Context B Service Anti Corruption Layer Transformation A↔B Coordinator System A
  91. 91. 4. Event Channels New Context ZRepository Repository System X Service Repository System Y Data Query Update Anti Corruption Layer Transformation Events X↔Z Events Coordinator Events Messagebus Event Channel Published Language
  92. 92. Questions?Email: jeppe@tigerteam.dk @TigerTeamDK on TwitterWeb: http://www.tigerteam.dk @jeppec on Twitter
  93. 93. Useful linksCQRS/DDD:• Bounded Contexts and Context Maps: http://www.infoq.com/articles/ddd-contextmapping• Rinat Abdullings Getting Started with CQRS: http://abdullin.com/cqrs/• CQRS journey: https://github.com/mspnp/cqrs-journey-doc/• Greg Youngs old blog: http://codebetter.com/gregyoung/• Greg Youngs new blog: http://goodenoughsoftware.net• Greg Youngs CQRS info site: http://cqrs.wordpress.com• Sagas: http://www.udidahan.com/2009/04/20/saga-persistence-and-event-driven-architectures/• Clarified CQRS: http://www.udidahan.com/2009/12/09/clarified-cqrs/• When to avoid CQRS: http://www.udidahan.com/2011/04/22/when-to-avoid-cqrs/• Why you should be using CQRS almost everywhere: http://www.udidahan.com/2011/10/02/why- you-should-be-using-cqrs-almost-everywhere…/• Videos from DDD exchange 2011: http://skillsmatter.com/event/design-architecture/ddd-exchange- 2011• Fohjin DDD example: https://github.com/MarkNijhof/Fohjin/tree/master/Fohjin.DDD.Example• Axon Framework: http://www.axonframework.org/• Implementing Domain Driven Design (IDDD): http://my.safaribooksonline.com/book/project- management/9780133039900• CQRS Info: www.cqrs.nu• Greg Youngs EventStore: http://geteventstore.com

×