We are
CQRS AND EVENT SOURCING
What, Why and How by PHU HOANG (Jimmy)
2
Agenda
● What is CQRS
● Why CQRS
● How to CQRS
● Example
● Q&A
● What is Event Sourcing
● Why Event Sourcing
● How to Event Sourcing
● Example
● Q&A
● The Good and Bad of CQRS & ES
● CQRS - ES Framework
3
A Important Note
RAISE YOUR HAND
IF YOU HAVE
QUESTIONS
4
What is CQRS
Command
Query
Responsibility
Segregation
5
COMMAND
● It is a method that performs an action. These would be the Create, Update and Delete parts of a
CRUD system.
● Commands are the only way to update data in your system
● Should be task based
● Can be placed on a queue for asynchronous processing, rather than being processed
synchronously.
What is CQRS
6
What is CQRS
QUERY
● A Query is a method that returns Data to the caller without modifying the records stored. This is
the Read part of a CRUD system.
● Queries never modify the database. A query returns a DTO that does not encapsulate any
domain knowledge.
● It is simply a read-only view into your database.
7
Why CQRS
● Independent scaling. CQRS allows the read and write workloads to scale independently, and
may result in fewer lock contentions.
● Optimized data schemas. The read side can use a schema that is optimized for queries, while
the write side uses a schema that is optimized for updates.
● Security. It's easier to ensure that only the right domain entities are performing writes on the
data.
● Separation of concerns. Segregating the read and write sides can result in models that are
more maintainable and flexible. Most of the complex business logic goes into the write model.
The read model can be relatively simple.
● Simpler queries. By storing a materialized view in the read database, the application can avoid
complex joins when querying.
● Team scalability. CQRS let you scale your team and let you best devs focus on the core stuff.
8
How to CQRS
CRUD / MVC
Controller
Model
DB
Make changes
via UI
Present new UI
or pass data to user
Model validate
change and
passes to DB
Database reads
model data
9
How to CQRS
CQRS
Command Controller
Command Model
DB
Make changes
via UI
Present new UI
or pass data to user
Command Model
validate change
and passes to DB
Query Model
reads from DB
Query Controller
Query Model
10
How to CQRS
11
How to CQRS
CQRS
Command Controller
Command Model
Write
DB
Make changes
via UI
Present new UI
or pass data to user
Command Model
validate change
and passes to DB
Query Model
reads from DB
Query Controller
Query Model
Read
DB
Projection
12
CQRS
DEMO
13
CQRS
Q & A
14
What is Event Sourcing
“Event Sourcing ensures that all changes to application state are stored as a sequence of events.
Not just can we query these events, we can also use the event log to reconstruct past states, and as
a foundation to automatically adjust the state to cope with retroactive changes.”
__ Martin Fowler, 2005
● Capture all changes to an application state as a sequence of events.
● Sequence of events:
○ Events are first-class citizens
○ Events track a change in state
○ The order of events is essential
○ Track the version of an event, track when an event occurred (VCS)
○ Events are append-only
○ History must not be re-written
○ Use compensating events to correct issues (AccountBilled > AccountRefunded)
15
What is Event Sourcing
DB
Order {
items=[itemA, itemB]
}
DB
Order {
items=[itemA, itemB, itemC]
}
DB
Order {
items=[itemA, itemC]
}
without Event Sourcing
16
What is Event Sourcing
DB
ItemAdded(ItemA)
ItemAdded(ItemB)
DB
ItemAdded(ItemA)
ItemAdded(ItemB)
ItemAdded(ItemC)
DB
ItemAdded(ItemA)
ItemAdded(ItemB)
ItemAdded(ItemC)
ItemRemoved(ItemB)
with Event Sourcing
17
Why Event Sourcing
● Storage Ignorance
You will never know what data you might need in the future or what questions will come up, so
keeping the history of events is something that can be extremely valuable to your future
business
● Flexibility
By replaying an event we could get a state of an object for any moment in time for free
● Improve performance
Append-only model storing events is a far easier model to scale.
And by having a read model we could have best of both worlds. Read side optimised for fast
queries and write side highly optimised for writes (and since there is no delete here, it could
really be fast writes)
● Simplify Developer's Life
Simplifies deployment and maintenance (less SQL, upgrade scripts and versioning)
Reduces expenses on both the hardware and software
Integration between systems is more straightforward
Since no data is ever lost, we will get a far better understanding of what the system is doing, we
gain full audit and excellent debugging capabilities.
18
HAVE YOU EVER BEEN ASKED?
● BOSS: Please give me a report of what are the products often be removed from shopping cart
● YOU: ...
● CUSTOMER: There was a bug while I make payment and my cart has been cleared, it has 100
items. What happened and how do I restore this?
● YOU: ….
● ADMINS: Hi there! I did a wrong mass update, all products are now have wrong quantity :( Can
you please revert it back?
● YOU: ...
Why Event Sourcing
19
version = 0
version = 0
version = 0
version = 0
How to Event Sourcing
SEQUENCE OF THE EVENTS
CartCreated
items = {}
total = $0
ItemAdded
item = ItemA
price = $100
ItemAdded
item = ItemB
price = $30
ItemAdded
item = ItemC
price = $20
ItemRemoved
item = ItemB
Event
Store
ItemRemoved
item=ItemB
ItemAdded
item=ItemC
ItemAdded
item=ItemB
ItemAdded
item=ItemA
CartCreated
items = {}
total = $0
Shopping Cart
items = {ItemA, ItemC}
total = $120
Aggregates
version = 1version = 0 version = 2 version = 3 version = 4
version = 0
20
How to Event Sourcing
EVENTS
● Events track a change in state
● The order of events is essential
● Events are append-only
● History must not be re-written
● Use compensating events to correct issues (eg: ItemAdded > ItemRemoved)
21
How to Event Sourcing
EVENTS
● Include the name of the affected aggregate/entity
e.g. Cart
● If necessary, include the name of the sub-entity/value object
e.g. CartItem
● Describe what exactly has happened, use past tense
e.g. CartItemAdded, CartItemRemoved
● Include all the properties modifying the state of the aggregate/entity in the payload of the
event
22
How to Event Sourcing
EVENT STORE
● A database where events are stored
● Optimized for events recording
● You can only write the events and the data connected to them
● Publishes events to the consumers
23
EVENT STORE
● A basic Event Storage can be represented in a Relational Database utilizing only two tables.
How to Event Sourcing
Column Name Column Type
AggregatedId GuiID
Data Blob
Version Int
Column Name Column Type
AggregatedId GuiID
Type Varchar
Version Int
EventsTableAggregatesTable
24
EVENT STREAMING
● Get events for an aggregate
SELECT * FROM Events WHERE AggregateId=`` ORDER BY Version
● Writing of a set of events to an Events Table
BEGIN
version = SELECT VERSION FROM Aggregates WHERE AggregateId = "@ID"
IF version IS null
INSERT INTO Aggregates
version = 0
END
IF your_version != version
raise concurrency_problem
FOREACH event
insert into Events with incremented version number
UPDATE aggregate WITH last version number
END TRANSACTION
How to Event Sourcing
25
SNAPSHOTS
● Prevent the need to load all of the events when issuing a query to rebuild an Aggregate.
● Create them at regular intervals
● Can be created on-the-fly using a trigger condition
● Can be created in the background using a scheduled job
● Start without it and see how far you can get
● Once necessary, start with larger intervals
How to Event Sourcing
Column Name Column Type
AggregatedId GuiID
SerializedData Blob
Version Int
SnapshotsTable
26
HOW IS EVENT SOURCING CAN HELP CQRS?
How to Event Sourcing
27
How to Event Sourcing
BACK TO CQRS
Command Controller
Command Model
Write
DB
Make changes
via UI
Present new UI
or pass data to user
Command Model
validate change
and passes to DB
Query Model
reads from DB
Query Controller
Query Model
Read
DB
Projection
28
How to Event Sourcing
BACK TO CQRS
Command Controller
Command Model
EVENT
STORE
Make changes
via UI
Present new UI
or pass data to user
Command Model
validate change
and passes to DB
Query Model
reads from DB
Query Controller
Query Model
Read
DB
Projection
(Aggregator )
Publish Event
29
How to Event Sourcing
BACK TO CQRS
Command Controller
Command Model
EVENT
STORE
Make changes
via UI
Present new UI
or pass data to user
Command Model
validate change
and passes to DB
Query Model
reads from DB
Query Controller
Query Model
Read
DB
Projection
(Aggregator )
Publish Event
Another
Read
DB
External
Systems
30
EVENT SOURCING
DEMO
31
EVENT SOURCING
Q & A
32
When to use CQRS
● It is perfect in case of a large difference between the number of reads and writes from a system
● When performance is critical
● When you have complex business logic
● When your UI is based on workflows and utilizes the Interface pattern
● When you want to parallelize the development and use two teams
● When you already use the Event Sourcing
The Good and Bad of CQRS & ES
33
The Good and Bad of CQRS & ES
When not to use CQRS
● In case of simple user interface e.g. CRUD style
● When you have simple business logic, then implementation of CQRS would be too overwhelming
● Do not use it for a whole solution. You do not need CQRS for every part of a system. Focus only
on this Bounded Context, which really needs CQRS
34
CQRS - ES Framework
● Broadway (PHP) https://github.com/broadway/broadway
● Prooph (PHP) https://github.com/prooph, http://getprooph.org
● Axon (Java + Scala) http://www.axonframework.org
● Akka Persistence (Java + Scala) http://doc.akka.io/docs/akka/current/scala/persistence.html
● Eventuate (Java + Scala) http://rbmhtechnology.github.io/eventuate/
● Nest (Javascript/NodeJS) https://nestjs.com/
● Wolkenkit (Javascript) https://www.wolkenkit.io/
● Kant (Python) https://github.com/patrickporto/kant
● Sequent (Ruby) https://www.sequent.io/
35
Whoooo Hooo It’s END

fram^ TechTalk #1 - CQRS and Event Sourcing (ES)

  • 1.
    We are CQRS ANDEVENT SOURCING What, Why and How by PHU HOANG (Jimmy)
  • 2.
    2 Agenda ● What isCQRS ● Why CQRS ● How to CQRS ● Example ● Q&A ● What is Event Sourcing ● Why Event Sourcing ● How to Event Sourcing ● Example ● Q&A ● The Good and Bad of CQRS & ES ● CQRS - ES Framework
  • 3.
    3 A Important Note RAISEYOUR HAND IF YOU HAVE QUESTIONS
  • 4.
  • 5.
    5 COMMAND ● It isa method that performs an action. These would be the Create, Update and Delete parts of a CRUD system. ● Commands are the only way to update data in your system ● Should be task based ● Can be placed on a queue for asynchronous processing, rather than being processed synchronously. What is CQRS
  • 6.
    6 What is CQRS QUERY ●A Query is a method that returns Data to the caller without modifying the records stored. This is the Read part of a CRUD system. ● Queries never modify the database. A query returns a DTO that does not encapsulate any domain knowledge. ● It is simply a read-only view into your database.
  • 7.
    7 Why CQRS ● Independentscaling. CQRS allows the read and write workloads to scale independently, and may result in fewer lock contentions. ● Optimized data schemas. The read side can use a schema that is optimized for queries, while the write side uses a schema that is optimized for updates. ● Security. It's easier to ensure that only the right domain entities are performing writes on the data. ● Separation of concerns. Segregating the read and write sides can result in models that are more maintainable and flexible. Most of the complex business logic goes into the write model. The read model can be relatively simple. ● Simpler queries. By storing a materialized view in the read database, the application can avoid complex joins when querying. ● Team scalability. CQRS let you scale your team and let you best devs focus on the core stuff.
  • 8.
    8 How to CQRS CRUD/ MVC Controller Model DB Make changes via UI Present new UI or pass data to user Model validate change and passes to DB Database reads model data
  • 9.
    9 How to CQRS CQRS CommandController Command Model DB Make changes via UI Present new UI or pass data to user Command Model validate change and passes to DB Query Model reads from DB Query Controller Query Model
  • 10.
  • 11.
    11 How to CQRS CQRS CommandController Command Model Write DB Make changes via UI Present new UI or pass data to user Command Model validate change and passes to DB Query Model reads from DB Query Controller Query Model Read DB Projection
  • 12.
  • 13.
  • 14.
    14 What is EventSourcing “Event Sourcing ensures that all changes to application state are stored as a sequence of events. Not just can we query these events, we can also use the event log to reconstruct past states, and as a foundation to automatically adjust the state to cope with retroactive changes.” __ Martin Fowler, 2005 ● Capture all changes to an application state as a sequence of events. ● Sequence of events: ○ Events are first-class citizens ○ Events track a change in state ○ The order of events is essential ○ Track the version of an event, track when an event occurred (VCS) ○ Events are append-only ○ History must not be re-written ○ Use compensating events to correct issues (AccountBilled > AccountRefunded)
  • 15.
    15 What is EventSourcing DB Order { items=[itemA, itemB] } DB Order { items=[itemA, itemB, itemC] } DB Order { items=[itemA, itemC] } without Event Sourcing
  • 16.
    16 What is EventSourcing DB ItemAdded(ItemA) ItemAdded(ItemB) DB ItemAdded(ItemA) ItemAdded(ItemB) ItemAdded(ItemC) DB ItemAdded(ItemA) ItemAdded(ItemB) ItemAdded(ItemC) ItemRemoved(ItemB) with Event Sourcing
  • 17.
    17 Why Event Sourcing ●Storage Ignorance You will never know what data you might need in the future or what questions will come up, so keeping the history of events is something that can be extremely valuable to your future business ● Flexibility By replaying an event we could get a state of an object for any moment in time for free ● Improve performance Append-only model storing events is a far easier model to scale. And by having a read model we could have best of both worlds. Read side optimised for fast queries and write side highly optimised for writes (and since there is no delete here, it could really be fast writes) ● Simplify Developer's Life Simplifies deployment and maintenance (less SQL, upgrade scripts and versioning) Reduces expenses on both the hardware and software Integration between systems is more straightforward Since no data is ever lost, we will get a far better understanding of what the system is doing, we gain full audit and excellent debugging capabilities.
  • 18.
    18 HAVE YOU EVERBEEN ASKED? ● BOSS: Please give me a report of what are the products often be removed from shopping cart ● YOU: ... ● CUSTOMER: There was a bug while I make payment and my cart has been cleared, it has 100 items. What happened and how do I restore this? ● YOU: …. ● ADMINS: Hi there! I did a wrong mass update, all products are now have wrong quantity :( Can you please revert it back? ● YOU: ... Why Event Sourcing
  • 19.
    19 version = 0 version= 0 version = 0 version = 0 How to Event Sourcing SEQUENCE OF THE EVENTS CartCreated items = {} total = $0 ItemAdded item = ItemA price = $100 ItemAdded item = ItemB price = $30 ItemAdded item = ItemC price = $20 ItemRemoved item = ItemB Event Store ItemRemoved item=ItemB ItemAdded item=ItemC ItemAdded item=ItemB ItemAdded item=ItemA CartCreated items = {} total = $0 Shopping Cart items = {ItemA, ItemC} total = $120 Aggregates version = 1version = 0 version = 2 version = 3 version = 4 version = 0
  • 20.
    20 How to EventSourcing EVENTS ● Events track a change in state ● The order of events is essential ● Events are append-only ● History must not be re-written ● Use compensating events to correct issues (eg: ItemAdded > ItemRemoved)
  • 21.
    21 How to EventSourcing EVENTS ● Include the name of the affected aggregate/entity e.g. Cart ● If necessary, include the name of the sub-entity/value object e.g. CartItem ● Describe what exactly has happened, use past tense e.g. CartItemAdded, CartItemRemoved ● Include all the properties modifying the state of the aggregate/entity in the payload of the event
  • 22.
    22 How to EventSourcing EVENT STORE ● A database where events are stored ● Optimized for events recording ● You can only write the events and the data connected to them ● Publishes events to the consumers
  • 23.
    23 EVENT STORE ● Abasic Event Storage can be represented in a Relational Database utilizing only two tables. How to Event Sourcing Column Name Column Type AggregatedId GuiID Data Blob Version Int Column Name Column Type AggregatedId GuiID Type Varchar Version Int EventsTableAggregatesTable
  • 24.
    24 EVENT STREAMING ● Getevents for an aggregate SELECT * FROM Events WHERE AggregateId=`` ORDER BY Version ● Writing of a set of events to an Events Table BEGIN version = SELECT VERSION FROM Aggregates WHERE AggregateId = "@ID" IF version IS null INSERT INTO Aggregates version = 0 END IF your_version != version raise concurrency_problem FOREACH event insert into Events with incremented version number UPDATE aggregate WITH last version number END TRANSACTION How to Event Sourcing
  • 25.
    25 SNAPSHOTS ● Prevent theneed to load all of the events when issuing a query to rebuild an Aggregate. ● Create them at regular intervals ● Can be created on-the-fly using a trigger condition ● Can be created in the background using a scheduled job ● Start without it and see how far you can get ● Once necessary, start with larger intervals How to Event Sourcing Column Name Column Type AggregatedId GuiID SerializedData Blob Version Int SnapshotsTable
  • 26.
    26 HOW IS EVENTSOURCING CAN HELP CQRS? How to Event Sourcing
  • 27.
    27 How to EventSourcing BACK TO CQRS Command Controller Command Model Write DB Make changes via UI Present new UI or pass data to user Command Model validate change and passes to DB Query Model reads from DB Query Controller Query Model Read DB Projection
  • 28.
    28 How to EventSourcing BACK TO CQRS Command Controller Command Model EVENT STORE Make changes via UI Present new UI or pass data to user Command Model validate change and passes to DB Query Model reads from DB Query Controller Query Model Read DB Projection (Aggregator ) Publish Event
  • 29.
    29 How to EventSourcing BACK TO CQRS Command Controller Command Model EVENT STORE Make changes via UI Present new UI or pass data to user Command Model validate change and passes to DB Query Model reads from DB Query Controller Query Model Read DB Projection (Aggregator ) Publish Event Another Read DB External Systems
  • 30.
  • 31.
  • 32.
    32 When to useCQRS ● It is perfect in case of a large difference between the number of reads and writes from a system ● When performance is critical ● When you have complex business logic ● When your UI is based on workflows and utilizes the Interface pattern ● When you want to parallelize the development and use two teams ● When you already use the Event Sourcing The Good and Bad of CQRS & ES
  • 33.
    33 The Good andBad of CQRS & ES When not to use CQRS ● In case of simple user interface e.g. CRUD style ● When you have simple business logic, then implementation of CQRS would be too overwhelming ● Do not use it for a whole solution. You do not need CQRS for every part of a system. Focus only on this Bounded Context, which really needs CQRS
  • 34.
    34 CQRS - ESFramework ● Broadway (PHP) https://github.com/broadway/broadway ● Prooph (PHP) https://github.com/prooph, http://getprooph.org ● Axon (Java + Scala) http://www.axonframework.org ● Akka Persistence (Java + Scala) http://doc.akka.io/docs/akka/current/scala/persistence.html ● Eventuate (Java + Scala) http://rbmhtechnology.github.io/eventuate/ ● Nest (Javascript/NodeJS) https://nestjs.com/ ● Wolkenkit (Javascript) https://www.wolkenkit.io/ ● Kant (Python) https://github.com/patrickporto/kant ● Sequent (Ruby) https://www.sequent.io/
  • 35.