Greenfield Development with CQRSDavid HoersterCo-Founder and CTO, BrainCredits
About MeC# MVP (April 2011)Co-Founder of BrainCredits (braincredits.com)Senior Technical Director for Resources Global ProfessionalsPresident of Pittsburgh .NET Users GroupOrganizer of recent Pittsburgh Code CampsTwitter - @DavidHoersterBlog – http://geekswithblogs.net/DavidHoersterEmail – david@braincredits.com
GoalsUnderstanding the CQRS Architecture & Benefits (& Drawbacks)CommandsDomain ObjectsEventsHandlersIncorporating into AzureQueues and BlobsFronting it with MVC
Pre-RequisitesSome knowledge and familiarity with Domain Driven DesignSome knowledge of Azure objectsSome knowledge of ASP.NET MVCLeaving the notion that your data store must be normalized to the Nth degree at the door
BackstoryGoals for BrainCredits.comFast readsScalableHandle, potentially, large volumes of requestsBe able to change quicklyAbility to track user in system and events that occurEasy to deploy, manageInexpensive (FREE????)
BackstoryDecided to jump into cloud with AzureBizSparkcompanyHey, it’s the cloud!ScalabilityDeployment via VS2010Pretty inexpensive – no HW to manageBut how to store data and design system?
BackstoryI would recommend to start looking along the CQRS style of architectures- RinatAbdullinArticles by Greg Young, UdiDahan, J. Oliver, R. Abdullin
Traditional ArchitectureIssuesGet all of the orders for user “David” in last 30 days
Traditional ArchitectureIssuesGet all the orders for user ‘David’ in last 30 daysSELECT c.FirstName, c.MiddleName, c.LastName, soh.SalesOrderID, soh.OrderDate,sod.UnitPrice, sod.OrderQty, sod.LineTotal,p.Name as 'ProductName', p.Color, p.ProductNumber,pm.Name as 'ProductModel',pc.Name as 'ProductCategory',pcParent.Name as 'ProductParentCategory'FROM SalesLT.Customer c INNER JOIN SalesLT.SalesOrderHeadersoh		ON c.CustomerID = soh.CustomerID		INNER JOIN SalesLT.SalesOrderDetail sod ON soh.SalesOrderID = sod.SalesOrderID		INNER JOIN SalesLT.Product p ON sod.ProductID = p.ProductID		INNER JOIN SalesLT.ProductModel pm ON p.ProductModelID = pm.ProductModelID		INNER JOIN SalesLT.ProductCategory pc ON p.ProductCategoryID = pc.ProductCategoryID		INNER JOIN SalesLT.ProductCategorypcParent ON pc.ParentProductCategoryID = pcParent.ProductCategoryIDWHERE c.FirstName = 'David'		AND soh.OrderDate > (GETDATE()-30)
Traditional ArchitectureIssuesWouldn’t it be great if it was something like?SELECT FirstName, MiddleName, LastName, SalesOrderID, OrderDate,UnitPrice, OrderQty, LineTotal, ProductName, Color, ProductNumber,ProductModel, ProductCategory, ProductParentCategoryFROM CustomerSalesWHERE FirstName= 'David'	AND OrderDate> (GETDATE()-30)
Traditional ArchitecureIssuesIt seems that many applications cater to fast     C-U-D operations, but the Reads sufferHighly normalized databasesUpdating a Product Name is easyWhat would make querying faster?Reducing JOINSFlattening out our modelHeresy!Business logic and domain objects bleed through layers as a result, too.
What Alternatives Are There?That’s where CQRS may be an optionProvides for fast reads at the ‘expense’ of more expensive writesOnce the data is read by the client, it’s already oldA system that is eventually consistent is usually acceptableEVENTUAL CONSISTENCY!The read model for the system eventually is consistent with the system eventsAllows for scalability very easilyEven in Azure!Encapsulates business logic in the domainAdheres strictly to CQS
What is CQRSCommand Query Responsibility SegregationBased upon Bertrand Meyer’s Command Query Separation principle:“every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer” (Meyer)In essence, commands return acknowledgement; queries return data.
What is CQRSRecipe of CQRSHealthy serving of CQSEqual parts DDDSome denormalization for seasoningAdd Event Sourcing to TasteOptionally Add Dash of NoSQLOptionally A Pinch of Cloud
What is CQRSTaking this further, CQRS describes an architecture where commands are messages (possibly asynchronous)queries go against a read modelSince commands are returning void, it allows for the message to be transformed into the read modelIf we’re transforming the message, why not make the read model as fast as possible?Since we’re transforming the message, perhaps we can take a few extra ticks for additional processingThis flips the conventional wisdom of fast writes and slow readsNow we have fast reads and “slower” writes (since we’re writing to a denormalized store)
What is CQRSCQRS essentially splits your application into two partsThe Read SideThe Command SideThe Read Side requests data and receives itThe Command Side issues commands (inserts, updates, deletes) and receives just an acknowledgment that it was received.
What is CQRS (Simplified)ClientCommand SideQuery SideData Requested/ReturnedCommand Issued – ack/nackRepositoryCommand ServiceDomain (ARs)Event HandlerRead Model StoreEvent StoreDenormalizer
Getting StartedSteps on the Command SideCommand is issuedRouted to proper Domain object methodDomain raises zero, one or more EventsEvents handled by Denormalizer to persist data to read modelEach step is really one or more messages being sent
Getting StartedYou can roll your own, or use a frameworkSeveral out there, including:NCQRSLokad.CQRSAgr.CQRS on CodePlex (not much activity)Axon (Java)By using a framework, there are “extras”
CQRS FrameworksWith NCQRS (ncqrs.org), you get:CQRS supportEvent SourcingSupport for SQL or NoSQL event sourcingSnapshottingNot always good thingsIf you’re doing ‘pure’ or ‘true’ CQRS, then this is goodOtherwise, may be overkillDon’t depend on the frameworkLet the framework help you, not constrict you
Demo – In Which David Turns To A Life-Long Desire – Buzzword BingoDemo – BuzzyGo
Command Side FlowEvent 1Handler 1ReadModelCommandEvent 2Handler 2DomainEvent NHandler N
CommandsSimple message from the client to the back-end requesting something to be doneImperative toneCreateUserDeleteCardMarkSquareContains enough information for the back-end to process the requestNo return valueMaps to Aggregate Root’s constructor or method
DomainCommand maps to a AR method/ctorBusiness logic happens hereMay assume validation done on client, but can be done hereBusiness rules applied heree.g. FREE space is in the middle and is automatically marked.Once business rules applied, data is added to events that are applied.One or more events can be appliedOnce event is applied, domain updates itself and is added to the event source repositoryNot exposed to client, generally
EventsSimple messages from domainSimilar to commandsBut not the sameSemantically differentCardCreated; SquareMarked; CardWonHas enough information for event handler to write information to the read model
Denormalizer (Handler)Handles the events applied by the domainShould NOT perform any validation or business rulesIts job is to perform minimal projections and transformations of data and then send that data to the repository/datastoreRepository/DataStore being updated is the read model
Read Side FlowRequest for DataReadModelQuery Provider(Repo/WCFDS/etc)ClientQueryResultsDTO
Where Can I Use CQRS?Windows appsServicesWeb appsCloud-based appsReally doesn’t matterJust depends on the problem being solved
CQRS in the CloudHosting CQRS in AzureReally no different on the web side than a CQRS web appTypically, you’d want 2 rolesWeb Role for the UI, which also performs queryingWorker Role for the Command handlingBoth roles can scale out as much as neededMultiple web roles with a single worker role, or vice versa or both!
Communication in theCloudHow do I issue commands from my web role?2 typical solutionsWCF service on the worker role sideUse Azure storage features (blobs, queues, tables)Both have advantages and disadvantagesWCF is by default synchronous, so you’ll need to design for fire-and-forgetBut can be  easier to get up and running quicklyAzure Storage has a lot of moving piecesWatch out for transaction costs!
Using Azure StorageGeneral flow is thisWeb role issues a command by dropping it in a blob container (name is GUID).Command blob name is then dropped in a queueWorker role monitors queue for workWhen it finds a message in the queue, it grabs blob from storage based on id in queue messageDeserializes command and then sends it through normal CQRS pipeline
CQRS in the CloudBlob StorageQueueControllerCommand HandlerDomainAzure Table StorageEvent(s)RepositoryDenormalizerRead Model
CQRS in the CloudASP.NET MVC ControllerReading data is no different, going through a RepositoryWriting data (issuing commands) involves dropping command into Azure QueueCloudHelper class facilitates thisAlso provides LogMessage facility
CQRS in the CloudUsing Azure Table Storage for logging user actions…Different than events.  Want to know user behavior.Storage is huge – just watch transaction costsConsider batching log readsBut you can create replays of user interactionSimilar to event sourcing’s replay of eventsEvent sourcing doesn’t capture “Cancels” or “Reads”, which may be useful from a marketing perspective
Claim Credit for thisSessionBrainCreditshttp://www.braincredits.com/Lesson/16852

Greenfield Development with CQRS

  • 1.
    Greenfield Development withCQRSDavid HoersterCo-Founder and CTO, BrainCredits
  • 2.
    About MeC# MVP(April 2011)Co-Founder of BrainCredits (braincredits.com)Senior Technical Director for Resources Global ProfessionalsPresident of Pittsburgh .NET Users GroupOrganizer of recent Pittsburgh Code CampsTwitter - @DavidHoersterBlog – http://geekswithblogs.net/DavidHoersterEmail – david@braincredits.com
  • 3.
    GoalsUnderstanding the CQRSArchitecture & Benefits (& Drawbacks)CommandsDomain ObjectsEventsHandlersIncorporating into AzureQueues and BlobsFronting it with MVC
  • 4.
    Pre-RequisitesSome knowledge andfamiliarity with Domain Driven DesignSome knowledge of Azure objectsSome knowledge of ASP.NET MVCLeaving the notion that your data store must be normalized to the Nth degree at the door
  • 5.
    BackstoryGoals for BrainCredits.comFastreadsScalableHandle, potentially, large volumes of requestsBe able to change quicklyAbility to track user in system and events that occurEasy to deploy, manageInexpensive (FREE????)
  • 6.
    BackstoryDecided to jumpinto cloud with AzureBizSparkcompanyHey, it’s the cloud!ScalabilityDeployment via VS2010Pretty inexpensive – no HW to manageBut how to store data and design system?
  • 7.
    BackstoryI would recommendto start looking along the CQRS style of architectures- RinatAbdullinArticles by Greg Young, UdiDahan, J. Oliver, R. Abdullin
  • 8.
    Traditional ArchitectureIssuesGet allof the orders for user “David” in last 30 days
  • 9.
    Traditional ArchitectureIssuesGet allthe orders for user ‘David’ in last 30 daysSELECT c.FirstName, c.MiddleName, c.LastName, soh.SalesOrderID, soh.OrderDate,sod.UnitPrice, sod.OrderQty, sod.LineTotal,p.Name as 'ProductName', p.Color, p.ProductNumber,pm.Name as 'ProductModel',pc.Name as 'ProductCategory',pcParent.Name as 'ProductParentCategory'FROM SalesLT.Customer c INNER JOIN SalesLT.SalesOrderHeadersoh ON c.CustomerID = soh.CustomerID INNER JOIN SalesLT.SalesOrderDetail sod ON soh.SalesOrderID = sod.SalesOrderID INNER JOIN SalesLT.Product p ON sod.ProductID = p.ProductID INNER JOIN SalesLT.ProductModel pm ON p.ProductModelID = pm.ProductModelID INNER JOIN SalesLT.ProductCategory pc ON p.ProductCategoryID = pc.ProductCategoryID INNER JOIN SalesLT.ProductCategorypcParent ON pc.ParentProductCategoryID = pcParent.ProductCategoryIDWHERE c.FirstName = 'David' AND soh.OrderDate > (GETDATE()-30)
  • 10.
    Traditional ArchitectureIssuesWouldn’t itbe great if it was something like?SELECT FirstName, MiddleName, LastName, SalesOrderID, OrderDate,UnitPrice, OrderQty, LineTotal, ProductName, Color, ProductNumber,ProductModel, ProductCategory, ProductParentCategoryFROM CustomerSalesWHERE FirstName= 'David' AND OrderDate> (GETDATE()-30)
  • 11.
    Traditional ArchitecureIssuesIt seemsthat many applications cater to fast C-U-D operations, but the Reads sufferHighly normalized databasesUpdating a Product Name is easyWhat would make querying faster?Reducing JOINSFlattening out our modelHeresy!Business logic and domain objects bleed through layers as a result, too.
  • 12.
    What Alternatives AreThere?That’s where CQRS may be an optionProvides for fast reads at the ‘expense’ of more expensive writesOnce the data is read by the client, it’s already oldA system that is eventually consistent is usually acceptableEVENTUAL CONSISTENCY!The read model for the system eventually is consistent with the system eventsAllows for scalability very easilyEven in Azure!Encapsulates business logic in the domainAdheres strictly to CQS
  • 13.
    What is CQRSCommandQuery Responsibility SegregationBased upon Bertrand Meyer’s Command Query Separation principle:“every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer” (Meyer)In essence, commands return acknowledgement; queries return data.
  • 14.
    What is CQRSRecipeof CQRSHealthy serving of CQSEqual parts DDDSome denormalization for seasoningAdd Event Sourcing to TasteOptionally Add Dash of NoSQLOptionally A Pinch of Cloud
  • 15.
    What is CQRSTakingthis further, CQRS describes an architecture where commands are messages (possibly asynchronous)queries go against a read modelSince commands are returning void, it allows for the message to be transformed into the read modelIf we’re transforming the message, why not make the read model as fast as possible?Since we’re transforming the message, perhaps we can take a few extra ticks for additional processingThis flips the conventional wisdom of fast writes and slow readsNow we have fast reads and “slower” writes (since we’re writing to a denormalized store)
  • 16.
    What is CQRSCQRSessentially splits your application into two partsThe Read SideThe Command SideThe Read Side requests data and receives itThe Command Side issues commands (inserts, updates, deletes) and receives just an acknowledgment that it was received.
  • 17.
    What is CQRS(Simplified)ClientCommand SideQuery SideData Requested/ReturnedCommand Issued – ack/nackRepositoryCommand ServiceDomain (ARs)Event HandlerRead Model StoreEvent StoreDenormalizer
  • 18.
    Getting StartedSteps onthe Command SideCommand is issuedRouted to proper Domain object methodDomain raises zero, one or more EventsEvents handled by Denormalizer to persist data to read modelEach step is really one or more messages being sent
  • 19.
    Getting StartedYou canroll your own, or use a frameworkSeveral out there, including:NCQRSLokad.CQRSAgr.CQRS on CodePlex (not much activity)Axon (Java)By using a framework, there are “extras”
  • 20.
    CQRS FrameworksWith NCQRS(ncqrs.org), you get:CQRS supportEvent SourcingSupport for SQL or NoSQL event sourcingSnapshottingNot always good thingsIf you’re doing ‘pure’ or ‘true’ CQRS, then this is goodOtherwise, may be overkillDon’t depend on the frameworkLet the framework help you, not constrict you
  • 21.
    Demo – InWhich David Turns To A Life-Long Desire – Buzzword BingoDemo – BuzzyGo
  • 22.
    Command Side FlowEvent1Handler 1ReadModelCommandEvent 2Handler 2DomainEvent NHandler N
  • 23.
    CommandsSimple message fromthe client to the back-end requesting something to be doneImperative toneCreateUserDeleteCardMarkSquareContains enough information for the back-end to process the requestNo return valueMaps to Aggregate Root’s constructor or method
  • 24.
    DomainCommand maps toa AR method/ctorBusiness logic happens hereMay assume validation done on client, but can be done hereBusiness rules applied heree.g. FREE space is in the middle and is automatically marked.Once business rules applied, data is added to events that are applied.One or more events can be appliedOnce event is applied, domain updates itself and is added to the event source repositoryNot exposed to client, generally
  • 25.
    EventsSimple messages fromdomainSimilar to commandsBut not the sameSemantically differentCardCreated; SquareMarked; CardWonHas enough information for event handler to write information to the read model
  • 26.
    Denormalizer (Handler)Handles theevents applied by the domainShould NOT perform any validation or business rulesIts job is to perform minimal projections and transformations of data and then send that data to the repository/datastoreRepository/DataStore being updated is the read model
  • 27.
    Read Side FlowRequestfor DataReadModelQuery Provider(Repo/WCFDS/etc)ClientQueryResultsDTO
  • 28.
    Where Can IUse CQRS?Windows appsServicesWeb appsCloud-based appsReally doesn’t matterJust depends on the problem being solved
  • 29.
    CQRS in theCloudHosting CQRS in AzureReally no different on the web side than a CQRS web appTypically, you’d want 2 rolesWeb Role for the UI, which also performs queryingWorker Role for the Command handlingBoth roles can scale out as much as neededMultiple web roles with a single worker role, or vice versa or both!
  • 30.
    Communication in theCloudHowdo I issue commands from my web role?2 typical solutionsWCF service on the worker role sideUse Azure storage features (blobs, queues, tables)Both have advantages and disadvantagesWCF is by default synchronous, so you’ll need to design for fire-and-forgetBut can be easier to get up and running quicklyAzure Storage has a lot of moving piecesWatch out for transaction costs!
  • 31.
    Using Azure StorageGeneralflow is thisWeb role issues a command by dropping it in a blob container (name is GUID).Command blob name is then dropped in a queueWorker role monitors queue for workWhen it finds a message in the queue, it grabs blob from storage based on id in queue messageDeserializes command and then sends it through normal CQRS pipeline
  • 32.
    CQRS in theCloudBlob StorageQueueControllerCommand HandlerDomainAzure Table StorageEvent(s)RepositoryDenormalizerRead Model
  • 33.
    CQRS in theCloudASP.NET MVC ControllerReading data is no different, going through a RepositoryWriting data (issuing commands) involves dropping command into Azure QueueCloudHelper class facilitates thisAlso provides LogMessage facility
  • 34.
    CQRS in theCloudUsing Azure Table Storage for logging user actions…Different than events. Want to know user behavior.Storage is huge – just watch transaction costsConsider batching log readsBut you can create replays of user interactionSimilar to event sourcing’s replay of eventsEvent sourcing doesn’t capture “Cancels” or “Reads”, which may be useful from a marketing perspective
  • 35.
    Claim Credit forthisSessionBrainCreditshttp://www.braincredits.com/Lesson/16852