Simplify Complex Query with CQRS

1. Simplify Complex Query with
CQRS
- Jacky Lai

2. Optimization is about Resource Trade-off
• The performance of an application is based on
• Memory Resource
• Computing Resource
• Network Resource
• Developer Resource
• Disk space Resource
• Disk space resource is relatively the cheapest resource compared to
others.

3. Example Requirement: Find Shipping
Methods
• During checkout process, user will be presented a list of shipping
methods to choose from, based on the product and shipping address.
• shippingMethods = findShippingMethodsBy(product,
shippingAddress);

4. Example Requirement: Request Payload
Request Json Payload:
{
"productId": ”aabbcc",
"address": ”123 Freedom Cir., Santa Clara, CA 95123"
}

5. Common Strategy: Back-end Processing
• Upon receiving request payload:
// step 1: construct hierarchical object graph, an expensive operation.
product = productRepository.findBy(productId);
shippingAddress = new Address(address);
// step 2: find shipping methods.
shippingMethods = findShippingMethodsBy(product, shippingAddress);

6. Common Strategy: ER Diagram
product
warehouse
product_type
shipping_method
n
1
nn
n
n
• restricted? giftCard?
• address
• size – LARGE? SMALL?
• delivery period

7. Common Strategy - Modeling
• Model with Hierarchical Data object, e.g.
• Product
• size (LARGE, SMALL)
• type (GIFT_CARD, RESTRICTED)
• warehouses
• address
• shipping methods
• Shipping Address

8. Issue #1: Network Traffic Increment.
• For each request, application layer has to fetch huge amount of data across network from
database, and process the data at Application layer.

9. Question:
Which is the best layer to filter data?

10. Issue #2:
Read Speed or Write Speed, Pick One.
• We cannot optimize both Read and Write speed at the same time.
• Without adding index,
• time complexity for read = O(n)
• After adding index,
• time complexity for read = O(log n)

11. Issue #2:
Read Speed or Write Speed, Pick One. – Cont.
• Performance Summary from “The Performance Impact of Adding
MySQL Indexes”
• http://logicalread.solarwinds.com/impact-of-adding-mysql-indexes-
mc12/#.VmkYP51Viko
• For a table with 553875 rows.
Before Adding Indexes After Adding Indexes
Insert Operation (sec) 7.14 24.77 (3x)
Data (mb) 33.56 33.56
Index (mb) 13.52 95.70 (7x)
Total = Data + Index (mb) 47.08 129.27

12. Issue #2:
Read Speed or Write Speed, Pick One. – Cont.
• What if we use Cache to reduce DB read?
• Cache is a Key-Value DB.
• Let’s say it takes 32 DB calls to build a complex object graph:
• Best case: 32 cache hits.
• Worst case: 32 cache misses + 32 DB calls.
• Network IO delays is unavoidable.
• There is another challenge: Cache Consistency.
There are only two hard
things in Computer Science:
cache invalidation and
naming things.
-- Phil Karlton

13. Issue #2:
Read Speed or Write Speed, Pick One. – Cont.
We need to maintain consistency for both normalized DB and denormalized DB, and this is tricky.
Overall Consistency = Consistency (Normalized) && Consistency (Denormalized)

14. Issue #3:
“Join” logic has to be at both sides (W, R)
product
warehouse
product_typemanufacture
product
warehouse
product_typemanufacturer
join join
wrong join
join
WRITE
Wrong READ
Database
Database

15. CQRS Comes to Rescue
• Proposed by Greg Young.
• Probably the best innovation from C# community to Java community.
• Command-Query Responsibility Segregation.
• Command -> Write
• Query -> Read
• Separate design for Write Operation and Read Operation.
• For Write, we want consistency.
• For Read, we want speed.

16. Common Strategy:
product
warehous
e
“write” join
Write Operation:
Write
data
Read Operation: Read
data
“read” join
Read:
O(log n) if it is indexed correctly,
O(n) without index.

17. CQRS:
product warehouse
“write”
join
Write Operation: Write Data
Read Operation: denormalized_warehouse_by_productRead Data
NO READ JOIN REQUIRED.
Broadcast Event
WRITE
WRITE
READ

18. Benefit #1: Fast Read
• Simple read. No join operation.
• We can achieve O(1) time complexity by using appropriate database.
• Minimized data transfer – reduced network IO delay.
• Reduced memory requirement – reduced GC.
Data filtering

19. Benefit #2: Fast Write
• Less indexes created.
• Tables (for write operation) are not polluted by Indexes (which are
created for read operation).

20. Benefit #3: Simple Read Logic
• Less convoluted Read-logic.
• Simple logic reduces mistakes.
• It promotes knowledge sharing among team members.
• Shorten development time.

21. product warehouse
“write”
join
Write Operation:
Write
data
Read Operation: MySQL:
denormalized_warehouse_by_product
Read
data
NO READ JOIN REQUIRED.
Broadcast: WAREHOUSE_CREATED_EVENT
Read Operation: C*, Solr:
denormalized_warehouse_by_product
Read
data
Benefit #4: Flexibility - Different Databases

22. Benefit #5: Consistency
• With CQRS, we only need to maintain consistency at 1 side (WRITE).
Normalized Tables
Write
data
Read
data
Consistency logic 1
Consistency logic 2
Normalized Tables
Write
data
Read
data
Consistency logic
Denormalized Tables
No consistency logic
Common Design CQRS
Event

23. The End.