Successfully reported this slideshow.

Simplify Complex Query with CQRS

1

Share

Upcoming SlideShare
TrendForecasting Handbook
TrendForecasting Handbook
Loading in …3
×
1 of 23
1 of 23

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Simplify Complex Query with CQRS

  1. 1. Simplify Complex Query with CQRS - Jacky Lai
  2. 2. Optimization is about Resource Trade-off • The performance of an application is based on • Memory Resource • Computing Resource • Network Resource • Developer Resource • Disk space Resource • Disk space resource is relatively the cheapest resource compared to others.
  3. 3. Example Requirement: Find Shipping Methods • During checkout process, user will be presented a list of shipping methods to choose from, based on the product and shipping address. • shippingMethods = findShippingMethodsBy(product, shippingAddress);
  4. 4. Example Requirement: Request Payload Request Json Payload: { "productId": ”aabbcc", "address": ”123 Freedom Cir., Santa Clara, CA 95123" }
  5. 5. Common Strategy: Back-end Processing • Upon receiving request payload: // step 1: construct hierarchical object graph, an expensive operation. product = productRepository.findBy(productId); shippingAddress = new Address(address); // step 2: find shipping methods. shippingMethods = findShippingMethodsBy(product, shippingAddress);
  6. 6. Common Strategy: ER Diagram product warehouse product_type shipping_method n 1 nn n n • restricted? giftCard? • address • size – LARGE? SMALL? • delivery period
  7. 7. Common Strategy - Modeling • Model with Hierarchical Data object, e.g. • Product • size (LARGE, SMALL) • type (GIFT_CARD, RESTRICTED) • warehouses • address • shipping methods • Shipping Address
  8. 8. Issue #1: Network Traffic Increment. • For each request, application layer has to fetch huge amount of data across network from database, and process the data at Application layer.
  9. 9. Question: Which is the best layer to filter data?
  10. 10. Issue #2: Read Speed or Write Speed, Pick One. • We cannot optimize both Read and Write speed at the same time. • Without adding index, • time complexity for read = O(n) • After adding index, • time complexity for read = O(log n)
  11. 11. Issue #2: Read Speed or Write Speed, Pick One. – Cont. • Performance Summary from “The Performance Impact of Adding MySQL Indexes” • http://logicalread.solarwinds.com/impact-of-adding-mysql-indexes- mc12/#.VmkYP51Viko • For a table with 553875 rows. Before Adding Indexes After Adding Indexes Insert Operation (sec) 7.14 24.77 (3x) Data (mb) 33.56 33.56 Index (mb) 13.52 95.70 (7x) Total = Data + Index (mb) 47.08 129.27
  12. 12. Issue #2: Read Speed or Write Speed, Pick One. – Cont. • What if we use Cache to reduce DB read? • Cache is a Key-Value DB. • Let’s say it takes 32 DB calls to build a complex object graph: • Best case: 32 cache hits. • Worst case: 32 cache misses + 32 DB calls. • Network IO delays is unavoidable. • There is another challenge: Cache Consistency. There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
  13. 13. Issue #2: Read Speed or Write Speed, Pick One. – Cont. We need to maintain consistency for both normalized DB and denormalized DB, and this is tricky. Overall Consistency = Consistency (Normalized) && Consistency (Denormalized)
  14. 14. Issue #3: “Join” logic has to be at both sides (W, R) product warehouse product_typemanufacture product warehouse product_typemanufacturer join join wrong join join WRITE Wrong READ Database Database
  15. 15. CQRS Comes to Rescue • Proposed by Greg Young. • Probably the best innovation from C# community to Java community. • Command-Query Responsibility Segregation. • Command -> Write • Query -> Read • Separate design for Write Operation and Read Operation. • For Write, we want consistency. • For Read, we want speed.
  16. 16. Common Strategy: product warehous e “write” join Write Operation: Write data Read Operation: Read data “read” join Read: O(log n) if it is indexed correctly, O(n) without index.
  17. 17. CQRS: product warehouse “write” join Write Operation: Write Data Read Operation: denormalized_warehouse_by_productRead Data NO READ JOIN REQUIRED. Broadcast Event WRITE WRITE READ
  18. 18. Benefit #1: Fast Read • Simple read. No join operation. • We can achieve O(1) time complexity by using appropriate database. • Minimized data transfer – reduced network IO delay. • Reduced memory requirement – reduced GC. Data filtering
  19. 19. Benefit #2: Fast Write • Less indexes created. • Tables (for write operation) are not polluted by Indexes (which are created for read operation).
  20. 20. Benefit #3: Simple Read Logic • Less convoluted Read-logic. • Simple logic reduces mistakes. • It promotes knowledge sharing among team members. • Shorten development time.
  21. 21. product warehouse “write” join Write Operation: Write data Read Operation: MySQL: denormalized_warehouse_by_product Read data NO READ JOIN REQUIRED. Broadcast: WAREHOUSE_CREATED_EVENT Read Operation: C*, Solr: denormalized_warehouse_by_product Read data Benefit #4: Flexibility - Different Databases
  22. 22. Benefit #5: Consistency • With CQRS, we only need to maintain consistency at 1 side (WRITE). Normalized Tables Write data Read data Consistency logic 1 Consistency logic 2 Normalized Tables Write data Read data Consistency logic Denormalized Tables No consistency logic Common Design CQRS Event
  23. 23. The End.

×