Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Simplify Complex Query with CQRS


Published on

Simplify Complex Query with CQRS

Published in: Software
  • Be the first to comment

  • Be the first to like this

Simplify Complex Query with CQRS

  1. 1. Simplify Complex Query with CQRS - Jacky Lai
  2. 2. Optimization is about Resource Trade-off • The performance of an application is based on • Memory Resource • Computing Resource • Network Resource • Developer Resource • Disk space Resource • Disk space resource is relatively the cheapest resource compared to others.
  3. 3. Example Requirement: Find Shipping Methods • During checkout process, user will be presented a list of shipping methods to choose from, based on the product and shipping address. • shippingMethods = findShippingMethodsBy(product, shippingAddress);
  4. 4. Example Requirement: Request Payload Request Json Payload: { "productId": ”aabbcc", "address": ”123 Freedom Cir., Santa Clara, CA 95123" }
  5. 5. Common Strategy: Back-end Processing • Upon receiving request payload: // step 1: construct hierarchical object graph, an expensive operation. product = productRepository.findBy(productId); shippingAddress = new Address(address); // step 2: find shipping methods. shippingMethods = findShippingMethodsBy(product, shippingAddress);
  6. 6. Common Strategy: ER Diagram product warehouse product_type shipping_method n 1 nn n n • restricted? giftCard? • address • size – LARGE? SMALL? • delivery period
  7. 7. Common Strategy - Modeling • Model with Hierarchical Data object, e.g. • Product • size (LARGE, SMALL) • type (GIFT_CARD, RESTRICTED) • warehouses • address • shipping methods • Shipping Address
  8. 8. Issue #1: Network Traffic Increment. • For each request, application layer has to fetch huge amount of data across network from database, and process the data at Application layer.
  9. 9. Question: Which is the best layer to filter data?
  10. 10. Issue #2: Read Speed or Write Speed, Pick One. • We cannot optimize both Read and Write speed at the same time. • Without adding index, • time complexity for read = O(n) • After adding index, • time complexity for read = O(log n)
  11. 11. Issue #2: Read Speed or Write Speed, Pick One. – Cont. • Performance Summary from “The Performance Impact of Adding MySQL Indexes” • mc12/#.VmkYP51Viko • For a table with 553875 rows. Before Adding Indexes After Adding Indexes Insert Operation (sec) 7.14 24.77 (3x) Data (mb) 33.56 33.56 Index (mb) 13.52 95.70 (7x) Total = Data + Index (mb) 47.08 129.27
  12. 12. Issue #2: Read Speed or Write Speed, Pick One. – Cont. • What if we use Cache to reduce DB read? • Cache is a Key-Value DB. • Let’s say it takes 32 DB calls to build a complex object graph: • Best case: 32 cache hits. • Worst case: 32 cache misses + 32 DB calls. • Network IO delays is unavoidable. • There is another challenge: Cache Consistency. There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
  13. 13. Issue #2: Read Speed or Write Speed, Pick One. – Cont. We need to maintain consistency for both normalized DB and denormalized DB, and this is tricky. Overall Consistency = Consistency (Normalized) && Consistency (Denormalized)
  14. 14. Issue #3: “Join” logic has to be at both sides (W, R) product warehouse product_typemanufacture product warehouse product_typemanufacturer join join wrong join join WRITE Wrong READ Database Database
  15. 15. CQRS Comes to Rescue • Proposed by Greg Young. • Probably the best innovation from C# community to Java community. • Command-Query Responsibility Segregation. • Command -> Write • Query -> Read • Separate design for Write Operation and Read Operation. • For Write, we want consistency. • For Read, we want speed.
  16. 16. Common Strategy: product warehous e “write” join Write Operation: Write data Read Operation: Read data “read” join Read: O(log n) if it is indexed correctly, O(n) without index.
  17. 17. CQRS: product warehouse “write” join Write Operation: Write Data Read Operation: denormalized_warehouse_by_productRead Data NO READ JOIN REQUIRED. Broadcast Event WRITE WRITE READ
  18. 18. Benefit #1: Fast Read • Simple read. No join operation. • We can achieve O(1) time complexity by using appropriate database. • Minimized data transfer – reduced network IO delay. • Reduced memory requirement – reduced GC. Data filtering
  19. 19. Benefit #2: Fast Write • Less indexes created. • Tables (for write operation) are not polluted by Indexes (which are created for read operation).
  20. 20. Benefit #3: Simple Read Logic • Less convoluted Read-logic. • Simple logic reduces mistakes. • It promotes knowledge sharing among team members. • Shorten development time.
  21. 21. product warehouse “write” join Write Operation: Write data Read Operation: MySQL: denormalized_warehouse_by_product Read data NO READ JOIN REQUIRED. Broadcast: WAREHOUSE_CREATED_EVENT Read Operation: C*, Solr: denormalized_warehouse_by_product Read data Benefit #4: Flexibility - Different Databases
  22. 22. Benefit #5: Consistency • With CQRS, we only need to maintain consistency at 1 side (WRITE). Normalized Tables Write data Read data Consistency logic 1 Consistency logic 2 Normalized Tables Write data Read data Consistency logic Denormalized Tables No consistency logic Common Design CQRS Event
  23. 23. The End.