When CQRS meets Event Sourcing
A warehouse management system done in PHP
When CQRS meets Event Sourcing / Ulabox
ULABOX
About me
● @manelselles
● Backend at Ulabox
● Symfony Expert
Certified by Sensiolabs
● DDD-TDD fan
When CQRS meets Event Sourcing / Warehouse
Warehouse management system
● PHP and framework agnostic
○ (almost) all of us love Symfony
● Independent of other systems
○ Ulabox ecosystem is complex -> Microservices
● Extensible and maintainable
○ Testing
● The system must log every action
○ Event driven architecture
When CQRS meets Event Sourcing / Warehouse
Please test!
Good practices
When CQRS meets Event Sourcing / Good practices
Outside-in TDD
● Behat features
● Describe behaviour with PhpSpec
● Testing integration with database of repository methods with Phpunit
When CQRS meets Event Sourcing / Good practices
Continuous integration
When CQRS meets Event Sourcing / Good practices
Other good practices
● SOLID
● Coding Style
● Pair programming
● Refactor
DDD-Hexagonal architecture
When CQRS meets Event Sourcing / DDD-Hexagonal
DDD Basics
● Strategic
○ Ubiquitous language
○ Bounded contexts
● Tactical
○ Value objects
○ Aggregates and entities
○ Repositories
○ Domain events
○ Domain and application services
When CQRS meets Event Sourcing / DDD-Hexagonal
Aggregate
When CQRS meets Event Sourcing / DDD-Hexagonal
Hexagonal architecture
namespace UlaboxChangoInfrastructureUiHttpController;
class ReceptionController
{
public function addContainerAction(JsonApiRequest $request, $receptionId)
{
$containerPayload = $this->jsonApiTransformer->fromPayload($request->jsonData(), 'container');
$this->receptionService->addContainer(ReceptionId::fromString($receptionId), $containerPayload);
return JsonApiResponse::createJsonApiData(200, null, []);
}
}
namespace UlaboxChangoInfrastructureUiAmqpConsumer;
class ContainerAddedToReceptionConsumer extends Consumer
{
public function execute(AMQPMessage $rabbitMessage)
{
$message = $this->messageBody($rabbitMessage);
$containerPayload = $this->amqpTransformer->fromPayload($message, 'container');
$this->receptionService->addContainer(ReceptionId::fromString($message['reception_id']), $containerPayload);
return ConsumerInterface::MSG_ACK;
}
}
namespace UlaboxChangoApplicationService;
class ReceptionService
{
public function addContainer(ReceptionId $receptionId, ContainerPayload $payload)
{
$reception = $this->receptionRepository->get($receptionId);
$reception->addContainer($payload->temperature(), $payload->lines());
$this->receptionRepository->save($reception);
$this->eventBus->dispatch($reception->recordedEvents());
}
}
When CQRS meets Event Sourcing / DDD-Hexagonal
Why application service?
● Same entry point
● Coordinate tasks on model
● Early checks
● User authentication
namespace UlaboxChangoDomainModelReception;
class Reception extends Aggregate
{
public function addContainer(Temperature $temperature, array $containerLines)
{
Assertion::allIsInstanceOf($containerLines, ContainerLinePayload::class);
$containerId = ContainerId::create($this->id(), $temperature, count($this->containers));
$this->containers->set((string) $containerId, new Container($containerId, $temperature));
$this->recordThat(new ContainerWasAdded($this->id, $containerId, $temperature));
foreach ($containerLines as $line) {
$this->addLine($containerId, $line->label(), $line->quantity(), $line->type());
}
}
public function addLine(ContainerId $containerId, Label $label, LineQuantity $quantity, ItemType $type)
{
if (!$container = $this->containers->get((string) $containerId)) {
throw new EntityNotFoundException("Container not found");
}
$container->addLine(ContainerLine::create($label, $quantity, $type));
$this->recordThat(new ContainerLineWasAdded($this->id, $containerId, $label, $quantity, $type));
}
}
namespace UlaboxChangoDomainModelReceptionContainer;
class Container
{
public function __construct(ContainerId $id, Temperature $temperature)
{
$this->id = $id;
$this->temperature = $temperature;
$this->lines = new ArrayCollection();
$this->status = ContainerStatus::PENDING();
}
public function addLine(ContainerLine $line)
{
if ($this->containsLine($line->label())) {
throw new AlreadyRegisteredException("Line already exists");
}
$this->lines->set((string) $line->label(), $line);
}
}
namespace UlaboxChangoInfrastructurePersistenceDoctrineReception;
class DoctrineReceptionRepository implements ReceptionRepository
{
public function get(ReceptionId $id)
{
return $this->find($id);
}
public function save(Reception $reception)
{
$this->_em->persist($reception);
}
}
Let’s apply Command and Query
Responsibility Segregation
When CQRS meets Event Sourcing / CQRS
CQRS
Separate:
● Command: do something
● Query: ask for something
Different source of data for read and write:
● Write model with DDD tactical patterns
● Read model with listeners to events
When CQRS meets Event Sourcing / CQRS
Command bus
● Finds handler for each action
● Decoupled command creator and handler
● Middlewares
○ Transactional
○ Logging
● Asynchronous actions
● Separation of concerns
When CQRS meets Event Sourcing / CQRS
Event bus
● Posted events are delivered to matching event handlers
● Decouples event producers and reactors
● Middlewares
○ Rabbit
○ Add correlation id
● Asynchronous actions
● Separation of concerns
When CQRS meets Event Sourcing / CQRS
namespace UlaboxChangoApplicationService;
class ReceptionService
{
public function addContainer(ReceptionId $receptionId, ContainerPayload $payload)
{
$command = new AddContainer($receptionId, $payload->temperature(), $payload->containerLines());
$this->commandBus->handle($command);
}
}
namespace UlaboxChangoDomainCommandReception;
class ReceptionCommandHandler extends CommandHandler
{
public function handleAddContainer(AddContainer $command)
{
$reception = $this->receptionRepository->get($command->aggregateId());
$reception->addContainer($command->temperature(), $command->lines());
$this->receptionRepository->save($reception);
$this->eventBus->dispatch($reception->recordedEvents());
}
}
namespace UlaboxChangoDomainReadModelReception;
class ReceptionProjector extends ReadModelProcessor
{
public function applyContainerWasAdded(ContainerWasAdded $event)
{
$reception = $this->receptionInfoView->receptionOfId($event->aggregateId());
$container = new ContainerProjection($event->containerId(), $event->temperature());
$this->receptionInfoView->save($reception->addContainer($container));
}
public function applyContainerLineWasAdded(ContainerLineWasAdded $event)
{
$reception = $this->receptionInfoView->receptionOfId($event->aggregateId());
$line = ContainerLineProjection($event->label(), $event->quantity(), $event->itemType());
$this->receptionInfoView->save($reception->addContainerLine($event->containerId(), $line));
}
}
namespace UlaboxChangoDomainReadModelReception;
interface ReceptionView
{
public function save(ReceptionProjection $reception);
public function receptionOfId(ReceptionId $receptionId);
public function find(Query $query);
}
namespace UlaboxChangoApplicationService;
class ReceptionQueryService
{
public function byId(ReceptionId $receptionId)
{
return $this->receptionView->receptionOfId($receptionId);
}
public function byContainer(ContainerId $containerId)
{
return $this->receptionView->find(new byContainer($containerId));
}
public function search($filters, Paging $paging = null, Sorting $sorting = null)
{
return $this->receptionView->find(new ByFilters($filters, $sorting, $paging));
}
}
Let’s get crazy: event sourcing
When CQRS meets Event Sourcing / Event sourcing
Event sourcing
● Entities are reconstructed with events
● No state
● No database to update manually
● No joins
When CQRS meets Event Sourcing / Event sourcing
Why event sourcing?
● Get state of an aggregate at any moment in time
● Append-only model storing events is easier to scale
● Forces to log because everything is an event
● No coupling between current state in the domain and in storage
● Simulate business suppositions
○ Change picking algorithm
When CQRS meets Event Sourcing / Event sourcing
Event Store
● PostgreSQL
● jsonb
● DBAL
namespace UlaboxChangoInfrastructurePersistenceEventStore;
class PDOEventStore implements EventStore
{
public function append(AggregateId $id, EventStream $eventStream)
{
$stmt = $this->connection->prepare("INSERT INTO event_store (data) VALUES (:message)");
$this->connection->beginTransaction();
foreach ($eventStream as $event) {
if (!$stmt->execute(['message' => $this->eventSerializer->serialize($event)])) {
$this->connection->rollBack();
}
}
$this->connection->commit();
}
public function load(AggregateId $id)
{
$stmt = $this->connection->prepare("SELECT data FROM event_store WHERE data->'payload'->>'aggregate_id' = :id");
$stmt->execute(['id' => (string) $id]);
$rows = $stmt->fetchAll(PDO::FETCH_ASSOC);
$events = [];
foreach ($rows as $row) {
$events[] = $this->eventSerializer->deserialize($row['data']);
}
return new EventStream($events);
}
}
When CQRS meets Event Sourcing / Event sourcing
namespace UlaboxChangoInfrastructurePersistenceModelReception;
class EventSourcingReceptionRepository implements ReceptionRepository
{
public function save(Reception $reception)
{
$events = $reception->recordedEvents();
$this->eventStore->append($reception->id(), $events);
foreach ($events as $event) {
$this->eventBus->dispatch($event);
}
}
public function load(ReceptionId $id)
{
$eventStream = $this->eventStore->load($id);
return Reception::reconstituteFromEvents(
new AggregateHistory($id, $eventStream)
);
}
}
namespace UlaboxChangoDomainModelReception;
class Reception extends EventSourcedAggregate
{
public static function create(
ReceptionId $id, DateTime $receptionDate, SupplierId $supplierId
) {
$instance = new self($id);
$instance->recordThat(
new ReceptionWasScheduled($id, $receptionDate, $supplierId)
);
return $instance;
}
protected function applyReceptionWasScheduled(ReceptionWasScheduled $event)
{
$this->receptionDate = $event->receptionDate();
$this->supplierId = $event->supplierId();
$this->status = ReceptionStatus::PENDING();
$this->containers = new ArrayCollection();
}
}
namespace UlaboxChangoDomainModel;
abstract class EventSourcedAggregate implements AggregateRoot, EventRecorder
{
protected function __construct()
{
$this->version = 0;
$this->eventStream = new EventStream();
}
protected function recordThat(Event $event)
{
$this->apply($event);
$this->eventStream->append($event);
}
protected function apply(Event $event)
{
$classParts = explode('', get_class($event));
$methodName = 'apply'.end($classParts);
if (method_exists($this, $methodName)) {
$this->$methodName($event);
}
$this->version++;
}
}
namespace UlaboxChangoDomainModelReception;
class Reception extends EventSourcedAggregate
{
public static function reconstituteFromEvents(AggregateHistory $history)
{
$instance = new self($history->aggregateId());
foreach ($history->events() as $event) {
$instance->apply($event);
}
return $instance;
}
public function addContainer(Temperature $temperature, array $containerLines)
{
$containerId = ContainerId::create(
$this->id(), $temperature, count($this->containers)
);
$this->recordThat(
new ContainerWasAdded($this->id, $containerId, $temperature)
);
foreach ($containerLines as $line) {
$this->addLine(
$containerId, $line->label(), $line->quantity(), $line->type()
);
}
}
protected function applyContainerWasAdded(ContainerWasAdded $event)
{
$container = new Container($event->containerId(), $event->temperature());
$this->containers->set((string) $event->containerId(), $container);
}
}
Conclusions
When CQRS meets Event Sourcing / Conclusions
Benefits
● Decoupling
● Performance in Read Model
● Scalability
● No joins
● Async with internal events and consumers
● Communicate other bounded contexts with events
When CQRS meets Event Sourcing / Conclusions
Problems found
● With DDD
○ Decide aggregates => talk a LOT with the domain experts
○ Boilerplate => generate as much boilerplate as possible
● With CQRS
○ Forgetting listeners in read model
○ Repeated code structure
● With event sourcing
○ Adapting your mindset
○ Forgetting applying the event to the entity
○ Retro compatibility with old events
● Concurrency/eventual consistency
Work with us!
When CQRS meets Event Sourcing / Work with us
Work with us
Thanks to...
When CQRS meets Event Sourcing / Conclusions
Thank you!
Questions?
@manelselles
manelselles@gmail.com

When cqrs meets event sourcing

  • 1.
    When CQRS meetsEvent Sourcing A warehouse management system done in PHP
  • 2.
    When CQRS meetsEvent Sourcing / Ulabox ULABOX
  • 3.
    About me ● @manelselles ●Backend at Ulabox ● Symfony Expert Certified by Sensiolabs ● DDD-TDD fan
  • 4.
    When CQRS meetsEvent Sourcing / Warehouse Warehouse management system ● PHP and framework agnostic ○ (almost) all of us love Symfony ● Independent of other systems ○ Ulabox ecosystem is complex -> Microservices ● Extensible and maintainable ○ Testing ● The system must log every action ○ Event driven architecture
  • 5.
    When CQRS meetsEvent Sourcing / Warehouse
  • 6.
  • 7.
    When CQRS meetsEvent Sourcing / Good practices Outside-in TDD ● Behat features ● Describe behaviour with PhpSpec ● Testing integration with database of repository methods with Phpunit
  • 8.
    When CQRS meetsEvent Sourcing / Good practices Continuous integration
  • 9.
    When CQRS meetsEvent Sourcing / Good practices Other good practices ● SOLID ● Coding Style ● Pair programming ● Refactor
  • 10.
  • 11.
    When CQRS meetsEvent Sourcing / DDD-Hexagonal DDD Basics ● Strategic ○ Ubiquitous language ○ Bounded contexts ● Tactical ○ Value objects ○ Aggregates and entities ○ Repositories ○ Domain events ○ Domain and application services
  • 12.
    When CQRS meetsEvent Sourcing / DDD-Hexagonal Aggregate
  • 13.
    When CQRS meetsEvent Sourcing / DDD-Hexagonal Hexagonal architecture
  • 14.
    namespace UlaboxChangoInfrastructureUiHttpController; class ReceptionController { publicfunction addContainerAction(JsonApiRequest $request, $receptionId) { $containerPayload = $this->jsonApiTransformer->fromPayload($request->jsonData(), 'container'); $this->receptionService->addContainer(ReceptionId::fromString($receptionId), $containerPayload); return JsonApiResponse::createJsonApiData(200, null, []); } } namespace UlaboxChangoInfrastructureUiAmqpConsumer; class ContainerAddedToReceptionConsumer extends Consumer { public function execute(AMQPMessage $rabbitMessage) { $message = $this->messageBody($rabbitMessage); $containerPayload = $this->amqpTransformer->fromPayload($message, 'container'); $this->receptionService->addContainer(ReceptionId::fromString($message['reception_id']), $containerPayload); return ConsumerInterface::MSG_ACK; } }
  • 15.
    namespace UlaboxChangoApplicationService; class ReceptionService { publicfunction addContainer(ReceptionId $receptionId, ContainerPayload $payload) { $reception = $this->receptionRepository->get($receptionId); $reception->addContainer($payload->temperature(), $payload->lines()); $this->receptionRepository->save($reception); $this->eventBus->dispatch($reception->recordedEvents()); } }
  • 16.
    When CQRS meetsEvent Sourcing / DDD-Hexagonal Why application service? ● Same entry point ● Coordinate tasks on model ● Early checks ● User authentication
  • 17.
    namespace UlaboxChangoDomainModelReception; class Receptionextends Aggregate { public function addContainer(Temperature $temperature, array $containerLines) { Assertion::allIsInstanceOf($containerLines, ContainerLinePayload::class); $containerId = ContainerId::create($this->id(), $temperature, count($this->containers)); $this->containers->set((string) $containerId, new Container($containerId, $temperature)); $this->recordThat(new ContainerWasAdded($this->id, $containerId, $temperature)); foreach ($containerLines as $line) { $this->addLine($containerId, $line->label(), $line->quantity(), $line->type()); } } public function addLine(ContainerId $containerId, Label $label, LineQuantity $quantity, ItemType $type) { if (!$container = $this->containers->get((string) $containerId)) { throw new EntityNotFoundException("Container not found"); } $container->addLine(ContainerLine::create($label, $quantity, $type)); $this->recordThat(new ContainerLineWasAdded($this->id, $containerId, $label, $quantity, $type)); } }
  • 18.
    namespace UlaboxChangoDomainModelReceptionContainer; class Container { publicfunction __construct(ContainerId $id, Temperature $temperature) { $this->id = $id; $this->temperature = $temperature; $this->lines = new ArrayCollection(); $this->status = ContainerStatus::PENDING(); } public function addLine(ContainerLine $line) { if ($this->containsLine($line->label())) { throw new AlreadyRegisteredException("Line already exists"); } $this->lines->set((string) $line->label(), $line); } }
  • 19.
    namespace UlaboxChangoInfrastructurePersistenceDoctrineReception; class DoctrineReceptionRepositoryimplements ReceptionRepository { public function get(ReceptionId $id) { return $this->find($id); } public function save(Reception $reception) { $this->_em->persist($reception); } }
  • 20.
    Let’s apply Commandand Query Responsibility Segregation
  • 21.
    When CQRS meetsEvent Sourcing / CQRS CQRS Separate: ● Command: do something ● Query: ask for something Different source of data for read and write: ● Write model with DDD tactical patterns ● Read model with listeners to events
  • 22.
    When CQRS meetsEvent Sourcing / CQRS Command bus ● Finds handler for each action ● Decoupled command creator and handler ● Middlewares ○ Transactional ○ Logging ● Asynchronous actions ● Separation of concerns
  • 23.
    When CQRS meetsEvent Sourcing / CQRS Event bus ● Posted events are delivered to matching event handlers ● Decouples event producers and reactors ● Middlewares ○ Rabbit ○ Add correlation id ● Asynchronous actions ● Separation of concerns
  • 24.
    When CQRS meetsEvent Sourcing / CQRS
  • 25.
    namespace UlaboxChangoApplicationService; class ReceptionService { publicfunction addContainer(ReceptionId $receptionId, ContainerPayload $payload) { $command = new AddContainer($receptionId, $payload->temperature(), $payload->containerLines()); $this->commandBus->handle($command); } } namespace UlaboxChangoDomainCommandReception; class ReceptionCommandHandler extends CommandHandler { public function handleAddContainer(AddContainer $command) { $reception = $this->receptionRepository->get($command->aggregateId()); $reception->addContainer($command->temperature(), $command->lines()); $this->receptionRepository->save($reception); $this->eventBus->dispatch($reception->recordedEvents()); } }
  • 26.
    namespace UlaboxChangoDomainReadModelReception; class ReceptionProjectorextends ReadModelProcessor { public function applyContainerWasAdded(ContainerWasAdded $event) { $reception = $this->receptionInfoView->receptionOfId($event->aggregateId()); $container = new ContainerProjection($event->containerId(), $event->temperature()); $this->receptionInfoView->save($reception->addContainer($container)); } public function applyContainerLineWasAdded(ContainerLineWasAdded $event) { $reception = $this->receptionInfoView->receptionOfId($event->aggregateId()); $line = ContainerLineProjection($event->label(), $event->quantity(), $event->itemType()); $this->receptionInfoView->save($reception->addContainerLine($event->containerId(), $line)); } } namespace UlaboxChangoDomainReadModelReception; interface ReceptionView { public function save(ReceptionProjection $reception); public function receptionOfId(ReceptionId $receptionId); public function find(Query $query); }
  • 27.
    namespace UlaboxChangoApplicationService; class ReceptionQueryService { publicfunction byId(ReceptionId $receptionId) { return $this->receptionView->receptionOfId($receptionId); } public function byContainer(ContainerId $containerId) { return $this->receptionView->find(new byContainer($containerId)); } public function search($filters, Paging $paging = null, Sorting $sorting = null) { return $this->receptionView->find(new ByFilters($filters, $sorting, $paging)); } }
  • 28.
    Let’s get crazy:event sourcing
  • 29.
    When CQRS meetsEvent Sourcing / Event sourcing Event sourcing ● Entities are reconstructed with events ● No state ● No database to update manually ● No joins
  • 30.
    When CQRS meetsEvent Sourcing / Event sourcing Why event sourcing? ● Get state of an aggregate at any moment in time ● Append-only model storing events is easier to scale ● Forces to log because everything is an event ● No coupling between current state in the domain and in storage ● Simulate business suppositions ○ Change picking algorithm
  • 31.
    When CQRS meetsEvent Sourcing / Event sourcing Event Store ● PostgreSQL ● jsonb ● DBAL
  • 32.
    namespace UlaboxChangoInfrastructurePersistenceEventStore; class PDOEventStoreimplements EventStore { public function append(AggregateId $id, EventStream $eventStream) { $stmt = $this->connection->prepare("INSERT INTO event_store (data) VALUES (:message)"); $this->connection->beginTransaction(); foreach ($eventStream as $event) { if (!$stmt->execute(['message' => $this->eventSerializer->serialize($event)])) { $this->connection->rollBack(); } } $this->connection->commit(); } public function load(AggregateId $id) { $stmt = $this->connection->prepare("SELECT data FROM event_store WHERE data->'payload'->>'aggregate_id' = :id"); $stmt->execute(['id' => (string) $id]); $rows = $stmt->fetchAll(PDO::FETCH_ASSOC); $events = []; foreach ($rows as $row) { $events[] = $this->eventSerializer->deserialize($row['data']); } return new EventStream($events); } }
  • 33.
    When CQRS meetsEvent Sourcing / Event sourcing
  • 34.
    namespace UlaboxChangoInfrastructurePersistenceModelReception; class EventSourcingReceptionRepositoryimplements ReceptionRepository { public function save(Reception $reception) { $events = $reception->recordedEvents(); $this->eventStore->append($reception->id(), $events); foreach ($events as $event) { $this->eventBus->dispatch($event); } } public function load(ReceptionId $id) { $eventStream = $this->eventStore->load($id); return Reception::reconstituteFromEvents( new AggregateHistory($id, $eventStream) ); } }
  • 35.
    namespace UlaboxChangoDomainModelReception; class Receptionextends EventSourcedAggregate { public static function create( ReceptionId $id, DateTime $receptionDate, SupplierId $supplierId ) { $instance = new self($id); $instance->recordThat( new ReceptionWasScheduled($id, $receptionDate, $supplierId) ); return $instance; } protected function applyReceptionWasScheduled(ReceptionWasScheduled $event) { $this->receptionDate = $event->receptionDate(); $this->supplierId = $event->supplierId(); $this->status = ReceptionStatus::PENDING(); $this->containers = new ArrayCollection(); } }
  • 36.
    namespace UlaboxChangoDomainModel; abstract classEventSourcedAggregate implements AggregateRoot, EventRecorder { protected function __construct() { $this->version = 0; $this->eventStream = new EventStream(); } protected function recordThat(Event $event) { $this->apply($event); $this->eventStream->append($event); } protected function apply(Event $event) { $classParts = explode('', get_class($event)); $methodName = 'apply'.end($classParts); if (method_exists($this, $methodName)) { $this->$methodName($event); } $this->version++; } }
  • 37.
    namespace UlaboxChangoDomainModelReception; class Receptionextends EventSourcedAggregate { public static function reconstituteFromEvents(AggregateHistory $history) { $instance = new self($history->aggregateId()); foreach ($history->events() as $event) { $instance->apply($event); } return $instance; } public function addContainer(Temperature $temperature, array $containerLines) { $containerId = ContainerId::create( $this->id(), $temperature, count($this->containers) ); $this->recordThat( new ContainerWasAdded($this->id, $containerId, $temperature) ); foreach ($containerLines as $line) { $this->addLine( $containerId, $line->label(), $line->quantity(), $line->type() ); } } protected function applyContainerWasAdded(ContainerWasAdded $event) { $container = new Container($event->containerId(), $event->temperature()); $this->containers->set((string) $event->containerId(), $container); } }
  • 38.
  • 39.
    When CQRS meetsEvent Sourcing / Conclusions Benefits ● Decoupling ● Performance in Read Model ● Scalability ● No joins ● Async with internal events and consumers ● Communicate other bounded contexts with events
  • 40.
    When CQRS meetsEvent Sourcing / Conclusions Problems found ● With DDD ○ Decide aggregates => talk a LOT with the domain experts ○ Boilerplate => generate as much boilerplate as possible ● With CQRS ○ Forgetting listeners in read model ○ Repeated code structure ● With event sourcing ○ Adapting your mindset ○ Forgetting applying the event to the entity ○ Retro compatibility with old events ● Concurrency/eventual consistency
  • 41.
  • 42.
    When CQRS meetsEvent Sourcing / Work with us Work with us
  • 43.
  • 44.
    When CQRS meetsEvent Sourcing / Conclusions
  • 45.