Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Async Messaging in CQRS: Part 1 - Masstransit + DDD Intro
Similar to Async Messaging in CQRS: Part 1 - Masstransit + DDD Intro (20)
Recently uploaded
Recently uploaded (20)
Async Messaging in CQRS: Part 1 - Masstransit + DDD Intro
- 1. ASYNC MESSAGING in CQRS PART 1: MASSTRANSIT DDD GREECE / dotNETZone.gr
- 2. DDD
- 3. DDD TYPICAL DDD SYSTEM LAYERS
- 4. DDD DOMAIN MODEL ▸ exists in domain layer ▸ reflects structured knowledge about the domain ▸ ubiquitous language ▸ agreed/shared with domain experts ▸ typically implemented as a SOLID - OO model ▸ “technology” free !
- 5. DDD DOMAIN STRUCTURE 1/2 ▸ Entities ▸ id ▸ constant throughout its lifecycle ▸ mutable ▸ Value Objects ▸ no id ▸ immutable ▸ diff state = diff object ▸ offloading logic from entities
- 6. DDD DOMAIN STRUCTURE 2/2 ▸ Domain Services ▸ stateless ▸ behaviour that cannot be modelled with Entities, Value Objects ▸ Domain Events ▸ immutable ▸ timestamped ▸ ref to entities
- 7. DDD AGGREGATE ▸ object graph of entities and value objects ▸ single entity is its root ▸ global identity ▸ external entities ref ONLY to the aggregate root ▸ root enforces invariants ▸ consistency boundary
- 9. CARGO TRACKER ABOUT ▸ Eric Evans “Big Blue Book” ▸ Published 2003 ▸ Imaginary Shipping Company
- 10. CARGO TRACKER KEY FUNCTIONALITY 1/2 ▸ Book a Cargo ▸ Delivery Spec (Origin, Destination + Arrival DeadLine) ▸ (Re-)Assign an Itinerary ▸ matches Delivery Spec ▸ derives from Legs of 1+ Vessel Voyages
- 11. CARGO TRACKER KEY FUNCTIONALITY 2/2 ▸ Register a Handling Event ▸ Load, Unload etc ▸ Report the: ▸ Transport Status (In Port, On Board Vessel …) ▸ Routing Status (Not Routed, Routed, …) ▸ Next Expected Handling Event ▸ Delivery History
- 12. CARGO TRACKER DOMAIN MODEL - CARGO AGGREGATE 1/3 ▸ Cargo (Root) ▸ HandlingEvent ▸ Root’s Methods: ▸ New(DeliverySpec) ▸ AssignItinerary(Itinerary) ▸ ChangeDelivery(DeliverySpec) ▸ RegisterHandlingEvent(HandlingEvent)
- 13. CARGO TRACKER DOMAIN MODEL - CARGO AGGREGATE 1/3 ▸ Cargo (Root) ▸ HandlingEvent ▸ Root’s Methods: ▸ New(DeliverySpec) ▸ AssignItinerary(Itinerary) ▸ ChangeDelivery(DeliverySpec) ▸ RegisterHandlingEvent(HandlingEvent)
- 14. CARGO TRACKER DOMAIN MODEL - CARGO AGGREGATE 1/3 ▸ Cargo (Root) ▸ HandlingEvent ▸ Root’s Methods: ▸ New(DeliverySpec) ▸ AssignItinerary(Itinerary) ▸ ChangeDelivery(DeliverySpec) ▸ RegisterHandlingEvent(HandlingEvent)
- 15. CARGO TRACKER DOMAIN MODEL - CARGO AGGREGATE 1/3 ▸ Cargo (Root) ▸ HandlingEvent ▸ Root’s Methods: ▸ New(DeliverySpec) ▸ AssignItinerary(Itinerary) ▸ ChangeDelivery(DeliverySpec) ▸ RegisterHandlingEvent(HandlingEvent) WRITE / COMMANDS
- 16. CARGO TRACKER DOMAIN MODEL - CARGO AGGREGATE 2/3 ▸ Root’s State: ▸ TransportStatus ▸ RoutingStatus ▸ NextExpected: HandlingEvent ▸ DeliveryHistory ▸ Collection of HandlingEvents
- 17. CARGO TRACKER DOMAIN MODEL - CARGO AGGREGATE 2/3 ▸ Root’s State: ▸ TransportStatus ▸ RoutingStatus ▸ NextExpected: HandlingEvent ▸ DeliveryHistory ▸ Collection of HandlingEvents READ / QUERIES
- 18. CARGO TRACKER REFACTORING - HANDLING EVENT
- 19. CARGO TRACKER REFACTORING - HANDLING EVENT
- 20. CARGO TRACKER REFACTORING - DELIVERY HISTORY ▸ “leaves Delivery History with no persistent state” ▸ “no real need to keep it around” ▸ “we could derive Delivery History itself whenever it is needed to answer some question”
- 21. CARGO TRACKER REFACTORING - HANDLING EVENT AGGREGATE ▸ HandlingEvent Aggregate: ▸ no domain logic (anemic) ▸ only repository + event class ▸ repository servicing query needs of other aggregates
- 22. CQRS RECAP
- 23. CQRS RECAP ABOUT ▸ Command-Query Responsibility Segregation ▸ keep separate read (query) and write (command) models ▸ DeliveryHistory: Read/Query ▸ Cargo: Write/Command ▸ practically SRP
- 24. CQRS RECAP COMMANDS ▸ BookNewCargo, AssignItinerary etc. ▸ App Layer ▸ CommandHandler ▸ BookNewCargoHandler, AssignItineraryHandler etc. ▸ practically an App Service ▸ SRP
- 25. CQRS RECAP EVENTS 1/2 ▸ NewCargoBooked, DeliveryChanged etc. ▸ Domain Layer ▸ Emitted by A/R after Command Applying ▸ Dispatched by Command Handler
- 26. CQRS RECAP EVENTS 2/2 ▸ EventHandler ▸ e.g. NewCargoBookedHandler, DeliveryChangedHandler ▸ App Layer ▸ Typical Handlers: ▸ Aggregate / Bounded Context Integration ▸ wired to another Command Handler ▸ Read Model Update
- 27. CQRS RECAP READ MODEL UPDATE 1/3 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
- 28. CQRS RECAP READ MODEL UPDATE 2/3 1 M 1 M 1 M 1M
- 29. CQRS RECAP READ MODEL UPDATE 3/3 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
- 30. CQRS RECAP MULTIPLE EVENT HANDLERS * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
- 31. CQRS RECAP MISCONCEPTIONS 1/2 ▸ needs two datastores/databases ▸ must be NoSQL ▸ requires async processing ▸ needs async messaging/queueing framework ▸ consistency only eventual
- 32. CQRS RECAP MISCONCEPTIONS 1/2 ▸ needs two datastores/databases ▸ must be NoSQL ▸ requires async processing ▸ needs async messaging/queueing framework ▸ consistency only eventual TRADE OFFS
- 33. CQRS RECAP MISCONCEPTIONS 2/2 ▸ needs a CQRS framework ▸ works only with event sourcing ▸ applied end-to-end ▸ e.g. CRUD
- 34. CQRS RECAP SYNC PROCESSING ▸ single process (e.g. Web App) ▸ decreases availability ▸ if process is UI affects UX ▸ in-process event dispatcher for decoupling ▸ rely on built-in framework facilities ▸ immediate consistency
- 35. CQRS RECAP ASYNC PROCESSING ▸ multiple processes (e.g. WebApp and WorkerApp) ▸ higher availability, scaling out ▸ consistency eventual ▸ typically implemented with async messaging ▸ message bus ▸ in-process event dispatcher allows for applying selectively
- 36. CQRS RECAP TYPICAL SYSTEM with CQRS + ASYNC PROCESSING / * -BBB 3 B BB # / A A * 3 BB # 3 A A BC A B A C A C B BC C B 2 3 AB BCB BC C CB B B A B B CB C B BB # B 3B C 2 3 A B C A 3 3 . 3 - 3 C 2 B BC C# / A A * ( 3 A A A 2 BB C 3 AB# C A 2 BB# C B B2A A# / A A * ) 2 ACB CB C A 3 3 2CB 3 AB BCB /A C 3
- 37. ASYNC MESSAGING
- 38. ASYNC MESSAGING MESSAGE BUS 1/2 ▸ frameworks: NServiceBus, MassTransit, Rebus etc. ▸ transports: RabbitMQ, Azure Service Bus etc. ▸ at least-once delivery ▸ FIFO ▸ reliable messaging ▸ automatic transport configuration setup ▸ pub/sub, send/receive
- 39. ASYNC MESSAGING MESSAGE BUS 1/2 ▸ frameworks: NServiceBus, MassTransit, Rebus etc. ▸ transports: RabbitMQ, Azure Service Bus etc. ▸ at least-once delivery ▸ FIFO ▸ reliable messaging ▸ automatic transport configuration setup ▸ pub/sub, send/receive
- 40. ASYNC MESSAGING ASYNC & RELIABLE MESSAGING ALL=CA .KI AK N ELDAK A AK AM N L KE AK ( A AE AK AM N L KE AK ) 1P D= CA - NANA ( NANA ) 1P D= CA . R ) -/3IK1KKIK ( A -/3IK = LC A E AKQ EL 142-.41 ( # , 2B LC EL IM - A Q MDA .KI AK = 1KKIK EL K=ELA E MDA A AK LC A E AKQ EL 142-.41 ) # , 2B LC EL IM - A IK = A Q MDA A AE AK MDA LC LM=QL E MDA NANA LC A E AKQ A MI A # EL - / , A AK EL I A =BMAK CAMME C MDA - LC A E AKQ EL 142-.41 # , NANAL = A AKLELMA M = KILL KI AK KALM=KML - = EM # 1 - 41 AAEA
- 41. ASYNC MESSAGING MESSAGE BUS 2/2 ▸ connection management ▸ persistent sagas/process managers ▸ retrying policy on subscriber/receiver errors ▸ alarm service ▸ (de)serialization
- 42. ASYNC MESSAGING CQRS & MESSAGING PATTERNS Command Handler Receiver Command Issuer Sender Event Handler Subscriber Event Emitter Publisher
- 43. ASYNC MESSAGING COMMAND FLOW 1. Process A A. Issue B. Send 2. Process B C. Receive D. Handle - Create Events
- 44. ASYNC MESSAGING EVENT FLOW 1. Process B A. Emit B. Publish 2. Process C C. Subscribe D. Handle
- 46. IMPLEMENTATION
- 47. IMPLEMENTATION WARNING ASYNC MESSAGING != ASYNC/AWAIT
- 49. IMPLEMENTATION COMMAND ISSUING - (WEB)APP LAYER ▸ IssueAsync implemented with Bus.Send<T>
- 50. IMPLEMENTATION COMMAND SENDING / BOOTSTRAPPING MASSTRANSIT ▸ bus is a Single Instance / AppDomain
- 51. IMPLEMENTATION MASSTRANSIT / BUS.SEND<T> ▸ Send(er)/Receive(r) Pattern
- 52. IMPLEMENTATION MASSTRANSIT / BUS.SEND<T> ▸ Send(er)/Receive(r) Pattern
- 57. IMPLEMENTATION COMMAND ISSUING/SENDING SUMMARY ▸ Use Domain Structures in Command ▸ Command Integrity ▸ ICommandIssuer abstraction ▸ MassTransit implementation ▸ Converts Commands to Messages (TbC) ▸ Encapsulate in an App Service ▸ Validation
- 59. IMPLEMENTATION COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT
- 60. IMPLEMENTATION COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT
- 61. IMPLEMENTATION COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT
- 62. IMPLEMENTATION COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT
- 63. IMPLEMENTATION COMMAND RECEIVING / BOOTSTRAPPING MASSTRANSIT
- 66. IMPLEMENTATION MASSTRANSIT RETRING FLOW (SIMPLIFIED) 1. message deserialised 2. identity consumer from message type A. if consumer identified ‣ run handler in try/catch ‣ on exception check retry policy ‣ either retry on ignore B. if consumer is NOT identified (TbC)
- 67. IMPLEMENTATION COMMAND HANDLING / EVENT DISPATCHING - APP LAYER
- 68. IMPLEMENTATION COMMAND HANDLING / EVENT DISPATCHING - APP LAYER
- 69. IMPLEMENTATION AGGREGATE EVENTS ▸ A/Rs should inherit from BaseAggregateRoot ▸ exposes Events collection
- 70. IMPLEMENTATION AGGREGATE EVENTS ▸ A/Rs should inherit from BaseAggregateRoot ▸ exposes Events collection
- 71. IMPLEMENTATION AGGREGATE EVENTS ▸ A/Rs should inherit from BaseAggregateRoot ▸ exposes Events collection
- 72. IMPLEMENTATION EVENT DISPATCHER 1/2 ▸ wires SYNChronously (in-process) command with event handlers ▸ is the “Emitter” ▸ wired handlers can be ▸ an actual, in-process one (e.g. read model update) ▸ a forwarder (event->message + publish) to the bus ▸ both ▸ even for the same event ▸ infra free implementation
- 73. IMPLEMENTATION EVENT DISPATCHER 2/2 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * IN-PROCESS FORWARD FORWARD
- 75. IMPLEMENTATION EVENT EMITTING/PUBLISHING ▸ as with Commands, Events should use domain structures ▸ IEventEmitter abstraction similar to ICommandIssuer ▸ MassTransit implementation with Bus.Publish<T> ▸ Converts Events to Messages (TbC)
- 76. IMPLEMENTATION MASSTRANSIT / BUS.PUBLISH<T> ▸ EndPoint agnostic ▸ Publisher(er)/Subscribe(r) Pattern
- 78. IMPLEMENTATION EVENT SUBSCRIBING / BOOTSTRAPPING MASSTRANSIT ▸ bootstrapping identical to command receiving ▸ need a Consumer<T> ▸ translates the message to event ▸ instantiates and invokes event handler
- 79. IMPLEMENTATION MASSTRANSIT / CONSUMER<T> (AGAIN)
- 80. IMPLEMENTATION MASSTRANSIT / CONSUMER<T> (AGAIN)
- 84. BEST PRACTICES
- 85. BEST PRACTICES MESSAGE DESIGN ▸ messages are POCOS. (period) ▸ do NOT reuse domain structures ▸ SERIOUS danger of message loss ▸ type repetition and mapping are unavoidable ▸ Automapper FTW ! ▸ messages might change (aka message versioning) ▸ subscribe to interfaces ▸ ensure correct version or serialiser assembly deployed ▸ JSON .NET
- 86. BEST PRACTICES SYSTEM ARCHITECTURE ▸ apply selectively not globally ▸ always forward (i.e. publish) the events ▸ bounded context interface ▸ rely on the retrying facilities ▸ idempotency ▸ fence “expensive” services ▸ perhaps even with sync send/receive (e.g. Routing Service)
- 87. BEST PRACTICES SCALING OPTIONS ▸ scale up: can invoke Message Handlers in parallel using a ThreadPool = Round Robin ▸ bus.SetConcurrencyLimit ▸ scale out: “competing consumers” ▸ same EndPointID ▸ probable concurrency issues ▸ aggregate versioning !
- 88. BEST PRACTICES HOUSE-KEEPING ▸ descriptive End Point IDs mostly for subscribers/receivers ▸ web_SERVER1_sender ▸ cmd_rec_worker ▸ evt_sub_booking_reporting ▸ evt_sub_shipping ▸ monitor transport
- 89. BEST PRACTICES COMPETING CONSUMERS 1 1 1 1 # 1 #
- 90. INDICATIONS
- 91. INDICATIONS ▸ command/event handlers that are: ▸ resource intensive/blocking ▸ involve multiple aggregate instances ▸ e.g. reroute all cargos unloading @ Hong Kong ▸ error prone ▸ involve external service calls ▸ e.g. email multiple recipients ▸ multiple bounded contexts
- 92. INDICATIONS ▸ separate physical read datastores ▸ high load of commands/events ▸ non-human/UI command issuers
- 93. CHALLENGES
- 94. CHALLENGES EVENTUAL CONSISTENCY ▸ heavily affects UI/UX ▸ contradicts user mentality ▸ need to notify on command handling error ▸ makes UI implementation more complex
- 95. CHALLENGES IMPLEMENTATION-RELATED ▸ a LOT of type repetition ▸ more physical processes ▸ changes MIGHT need orchestration ▸ concurrency issues ▸ transport = additional infra
- 96. RESOURCES
- 97. RESOURCES DDD ▸ Domain Driven Design by Eric Evans http://amzn.to/2uhbOSc ▸ Patterns, Principles and Practices of Domain-Driven Design by Scott Millett http://amzn.to/2hsNbLg ▸ Effective Aggregate Design (3 Part Series) by Vaughn Vernon https://vaughnvernon.co/?p=838 ▸ Awesome DDD by Nick Chamberlain https://github.com/heynickc/awesome-ddd
- 98. RESOURCES CQRS ▸ CQRS Journey by Microsoft Patterns & Practises cqrsjourney.github.io ▸ CQRS by Edument cqrs.nu ▸ Clarified CQRS by Udi Dahan udidahan.com/2009/12/09/clarified-cqrs
- 99. RESOURCES SAMPLES ▸ Cargo Tracker ▸ github.com/citerus/dddsample-core (Java) by Citerus AB ▸ github.com/SzymonPobiega/DDDSample.Net (.ΝΕΤ) by Szymon Pobiega ▸ CQRS, The example - Mark Nijhof leanpub.com/cqrs
- 100. RESOURCES SANDBOX SERVICES ▸ RabbitMQ: CloudAMQP www.cloudamqp.com ▸ MongoDB: Atlas www.mongodb.com/cloud ▸ PostgreSQL, SQL Server etc: AWS Free Tier aws.amazon.com/free/
- 101. Q&A