Message Oriented Architecture
using NServiceBus
Lars-Erik Kindblad
Senior Consultant
Twitter: @kindblad
E-mail: le@kindbla...
What is NServiceBus?
 Lightweight messaging framework for designing distributed systems in
.NET
 An Enterprise Service B...
 Remote Procedure Call (Request/Response)
Request: Order

Website

Response: Order ID

Order Service
PlaceOrder(order)

S...
NServiceBus Terminology & Components
Sender
/Send-Only
Endpoint

Message,
Command
or Event
Message: PlaceOrder

Website

M...
Messages in NServiceBus
 Just simple .NET classes with properties

 2 types of messages
 Command
• Do something: PlaceO...
 Command
Strongly coupled to
the receiving
endpoint
Command: PlaceOrder

Website

Command: PlaceOrder

Message Queue

End...
Queues
 First in-first out datastructure for storing and retrieving messages
PlaceOrder message 4
First-In
Queue
PlaceOrd...
How to Send & Handle Messages
Command: PlaceOrder

Website

Send

Command: PlaceOrder

Message Queue

Endpoint

Handle
(Co...
Summary
 Sender endpoint
 Sends a message
 Can be a website, desktop application or even a worker endpoint

 Command
...
NServiceBus Requirements
 MSMQ (default) or any other supported queue






ActiveMQ
RabbitMQ
SQL Server
Windows Azu...
NuGet packages
 NServiceBus Interfaces
 Interfaces that we need to define messages

 NServiceBus
 Most of the code tha...
The Case – The eBook Shop
1. Create the order in the
database
2. Auto-pay the order using a
previously stored credit
card
...
The Usual Way - Request/Response
Place order
1. Add order

Browser

OrderController
Confirmation page
with order ID

Datab...
Problem #1 – The order is lost during an error
Place order
1. Add order

Browser

OrderController

Database

Error page
Ex...
Problem #2 – Poor Transaction Management
Place order
1. Add order

Browser

OrderController

Database

Error page
2. Charg...
Let’s Fix This With NServiceBus
 Change to a message oriented architecture
Add PlaceOrder
message

Place order

Browser

...
The Message Must Also Be Processed
MSMQ

1. Connect

2. Receive message PlaceOrder

Worker
(Windows Service)

PlaceOrder

...
When an Error Occur

4. Rollback. Put the message
back on the queue
and retry

MSMQ

1. Connect

2. Receive message
PlaceO...
More about MSMQ
 Max 4MB message size
 Persists messages to files. Default location C:WINDOWSSystem32msmqstorage
 Guran...
How NServiceBus Handles Failure
 Transaction management and rollback are automatically handled by
NServiceBus in message ...
A challenge – The Order ID
Old

New

 The order ID has not been generated yet – it’s generated in the message
handler at ...
Possible Solutions
 Change the UI to not show the order ID
 Use a GUID (Guid.NewGuid()) or find another way to generate ...
We Still Have Poor Transaction Management
1. Add order

Handle PlaceOrder

Database
2. Charge credit card

PaymentService
...
Solution
 Split into man small messages - One message per transactional boundary
Place order

OrderController

Browser

1...
Code - The Messages
Code – Message Handlers #1
Code – Message Handlers #2
Yet another transactional issue
MSMQ

Processed
ok

Handle PayOrder message

Charge credit card

Request
Handle PayOrder

...
Solution
 Operations should be idempotent
 Idempotent operations can be run many times without changing the result

MSMQ...
Another Challenge – Eventual Consistency

Users clicks on View
Order History

 The PlaceOrder message is processed async ...
Publish/Subscribe
Place order

OrderController

Browser

1. Add PlaceOrder message
PlaceOrder command

2. Handle PlaceOrde...
Summary
Request/Response

Command

Event

X

X

Automatically retries

X

X

Full transactional
support

X

X

Reliable
Co...
Scaling
 Scale up
 Easy to scale up if business processes are split into multiple messages
 Increase the number of thre...
How the Distributor Works
1. Worker:
Send I’m ready
message

I’m ready message

Distributor:
Control Queue

I’m ready mess...
Other Features
 Sagas
 Long running workflow like processes
 State is shared between multiple message handlers

 Sched...
NServiceBus License & Pricing
 Open source
 Used to be free, but not anymore
 Indefinitely license with 1 year maintena...
Demo
About Capgemini
With more than 120,000 people in 40 countries, Capgemini is one
of the world's foremost providers of consu...
Upcoming SlideShare
Loading in...5
×

Message Oriented Architecture using NServiceBus

8,394

Published on

These slides are from my presentation on the benefits and disadvantages of moving from a remote procedure call type of architecture and to a message oriented architecture by using NServiceBus.

Published in: Technology
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
8,394
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
244
Comments
0
Likes
16
Embeds 0
No embeds

No notes for slide

Transcript of "Message Oriented Architecture using NServiceBus"

  1. 1. Message Oriented Architecture using NServiceBus Lars-Erik Kindblad Senior Consultant Twitter: @kindblad E-mail: le@kindblad.com
  2. 2. What is NServiceBus?  Lightweight messaging framework for designing distributed systems in .NET  An Enterprise Service Bus  Helps to make distributed systems more  Reliable  Scalable  Extensible  Founded in 2006 by UDI Dahan, an international renowned expert on software architecture and design  Services and support provided by Particular Software  Open Source but not free  Source code available at https://github.com/Particular/NServiceBus  Binaries available as NuGet packages
  3. 3.  Remote Procedure Call (Request/Response) Request: Order Website Response: Order ID Order Service PlaceOrder(order) Synchronous – The website is blocked while calling the Order Service  Messaged Oriented Architecture (One-way messaging) Message: PlaceOrder Website Message Queue Connect Receive PlaceOrder message Windows Service Process the PlaceOrder message Asynchronous – The website is NOT blocked while the place order is processed
  4. 4. NServiceBus Terminology & Components Sender /Send-Only Endpoint Message, Command or Event Message: PlaceOrder Website Message Queue Connect Receive PlaceOrder message Windows Service Process the PlaceOrder message Message Handler Worker Endpoint, Host or Receiver
  5. 5. Messages in NServiceBus  Just simple .NET classes with properties  2 types of messages  Command • Do something: PlaceOrder • Sent from one or more senders, processed by one endpoint • Implement ICommand  Event • Something has happened: OrderWasPlaced • Published from one sender, processed by several endpoints/subscribers • Implement IEvent
  6. 6.  Command Strongly coupled to the receiving endpoint Command: PlaceOrder Website Command: PlaceOrder Message Queue Endpoint  Event (Publish/Subscribe) Subscribing Endpoint Loosely coupled to the receiving endpoints Event: OrderWasPlaced Worker NServiceBus will add 3 messages to the queue, one for each subscriber Event: OrderWasPlaced Message Queue Subscribing Endpoint Subscribing Endpoint
  7. 7. Queues  First in-first out datastructure for storing and retrieving messages PlaceOrder message 4 First-In Queue PlaceOrder message 3 PlaceOrder message 2 PlaceOrder message 1 First-Out  Default queue in NServiceBus is Microsoft Message Queuing (MSMQ)  Available on all Windows editions since NT 4 and Windows 95  Also support for      ActiveMQ RabbitMQ SQL Server Windows Azure Queues Windows Azure ServiceBus  One endpoint has one queue, Send-Only endspoints do not have queues
  8. 8. How to Send & Handle Messages Command: PlaceOrder Website Send Command: PlaceOrder Message Queue Endpoint Handle (Command or Event)
  9. 9. Summary  Sender endpoint  Sends a message  Can be a website, desktop application or even a worker endpoint  Command  Message where we want to do something concrete  ICommand interface  Event  Message where we notify about something that has happened  IEvent interface  Message handler  Processes a given message  IHandleMessages<Message> interface  Worker endpoint  Finds and executes the handler for the message  Can be a Windows Service, console application, WCF, website
  10. 10. NServiceBus Requirements  MSMQ (default) or any other supported queue      ActiveMQ RabbitMQ SQL Server Windows Azure Queues Windows Azure ServiceBus  Distributed Transaction Cordinator (DTC)  RavenDB (default) or a relational database  Download the installer from NServiceBus from http://particular.net/ to setup the required components
  11. 11. NuGet packages  NServiceBus Interfaces  Interfaces that we need to define messages  NServiceBus  Most of the code that drives NServiceBus except hosting  Use for send-only endpoints  NServiceBus Host  Console application that can be installed as a Windows Service  Used to process/handle messages  NServiceBus Testing  Framework for testing NServiceBus endpoints
  12. 12. The Case – The eBook Shop 1. Create the order in the database 2. Auto-pay the order using a previously stored credit card 3. Send an order confirmation by e-mail 4. Deliver the book to the Windows Phone device
  13. 13. The Usual Way - Request/Response Place order 1. Add order Browser OrderController Confirmation page with order ID Database 2. Charge credit card PaymentService 3. Send mail SMTP-Server 4. Deliver ebook DeviceService
  14. 14. Problem #1 – The order is lost during an error Place order 1. Add order Browser OrderController Database Error page Exception 2. Charge credit card Error PaymentService 3. Send mail SMTP-Server 4. Deliver ebook DeviceService  Conseqence  The database is rolled back  User receives an error  Order must be sent again
  15. 15. Problem #2 – Poor Transaction Management Place order 1. Add order Browser OrderController Database Error page 2. Charge credit card PaymentService 3. Send mail SMTP-Server Exception 4. Deliver ebook Error DeviceService  Conseqence:  The credit card is charged (no transaction support)  A mail saying the order was successful was sent (no transaction support)  The order do not exist in the database (supports transaction – rolled back)  The user will receive an error
  16. 16. Let’s Fix This With NServiceBus  Change to a message oriented architecture Add PlaceOrder message Place order Browser Queue OrderController Confirmation page Old New
  17. 17. The Message Must Also Be Processed MSMQ 1. Connect 2. Receive message PlaceOrder Worker (Windows Service) PlaceOrder 3. PlaceOrder Message Handler 4. Add order Database 5. Charge credit card PaymentService 6. Send mail SMTP-Server 7. Deliver ebook DeviceService
  18. 18. When an Error Occur 4. Rollback. Put the message back on the queue and retry MSMQ 1. Connect 2. Receive message PlaceOrder Worker (Windows Service) PlaceOrder 3. PlaceOrder Error Message Handler
  19. 19. More about MSMQ  Max 4MB message size  Persists messages to files. Default location C:WINDOWSSystem32msmqstorage  Guranteed once-and-only-once delivery  The message will only be handled once  Supports transactional queues and distributed transactions (using DTC)  If an error occur both changes to the database and to the queue can be rolled back  Supports store-and-forward  MSMQ will store messages locally before forwarding it to the queue on another server  MSMQ will confirm when the message has been written to the disk in the outgoing queue, not when it has been delivered to the server  Important feature for distributed systems with unrealiable networks Server A Sender MSMQ Outgoing queue Server B Message MSMQ Input queue Worker
  20. 20. How NServiceBus Handles Failure  Transaction management and rollback are automatically handled by NServiceBus in message handlers  When an error occur 1. The message is put back on the queue 2. Attempt to process the message 5 times, wait for 10 seconds 3. Attempt to process the message 5 times, wait for 20 seconds 4. Attempt to process the message 5 times, wait for 30 seconds 5. Attempt to process the message 5 times, send the message to an error queue The time interval is configurable  Messages can be moved back to the original queue for replay using ReturnToSourceQueue.exe tool.
  21. 21. A challenge – The Order ID Old New  The order ID has not been generated yet – it’s generated in the message handler at the worker endpoint
  22. 22. Possible Solutions  Change the UI to not show the order ID  Use a GUID (Guid.NewGuid()) or find another way to generate the ID in the controller instead of in the database • message.OrderId = Guid.NewGuid(); Bus.Send(message); ViewBag.OrderId = message.OrderId; • 5773DD0E-0AB0-446B-8649-B2D43D7DA4AA is not a very user friendly ID  Move OrderRepository.Add(order) to the Controller: • var orderId = OrderRepository.Add(order); Bus.Send(message); • Less reliable and more error prone solution  Use Send/Reply to simulate request/response • The controller will wait for a PlaceOrderCompleted message that the PlaceOrder handler will send • Too much overhead • An Anti-pattern
  23. 23. We Still Have Poor Transaction Management 1. Add order Handle PlaceOrder Database 2. Charge credit card PaymentService 3. Send mail SMTP-Server 4. Deliver ebook Exception Error DeviceService  Conseqence:  The credit card is charged (no transaction support)  A mail saying the order was successful was sent (no transaction support)  The order do not exist in the database (supports transaction – rolledback)  The message is put back on the queue and will be retried = credit card will be charged twice
  24. 24. Solution  Split into man small messages - One message per transactional boundary Place order OrderController Browser 1. Add PlaceOrder message PlaceOrder message 2. Handle PlaceOrder Add PayOrder message If an error occur only DeliverEbook is affected. The message will be put back on the queue and retried SendMail message 4. Handle SendMail DeliverEbook message 5. Handle DeliverEbook PayOrder message Queue Add SendMail & DeliverEbook message 3. Handle PayOrder
  25. 25. Code - The Messages
  26. 26. Code – Message Handlers #1
  27. 27. Code – Message Handlers #2
  28. 28. Yet another transactional issue MSMQ Processed ok Handle PayOrder message Charge credit card Request Handle PayOrder Request: Error PaymentService Payment OK Network error External Payment Gateway  The message will be put back on the queue and processed again  Consequence: The credit card will be charged more than once
  29. 29. Solution  Operations should be idempotent  Idempotent operations can be run many times without changing the result MSMQ Handle PayOrder message Charge credit card Request Handle PayOrder Payment OK PaymentService Has the order been paid? Use OrderId Database Payment OK External Payment Gateway Has the message already been processed? Use order ID or a GUID Database
  30. 30. Another Challenge – Eventual Consistency Users clicks on View Order History  The PlaceOrder message is processed async by another process  If it has not gotten processed yet then the new order will not be in the list  Solution  Some UI trick?  Move the OrderRepository.Add to the controller  Make sure the messages are handled fast enough • Define a SLA on the message to help monitoring
  31. 31. Publish/Subscribe Place order OrderController Browser 1. Add PlaceOrder message PlaceOrder command 2. Handle PlaceOrder Add PayOrder command With Publish/Subscribe the Subscribers are loosely coupled to the publisher Subscribe to OrderWasPaid 4. Handle SendMail Subscribes to OrderWasPaid Queue 3. Handle PayOrder Add OrderWasPaid event OrderWasPaid Subscribes to OrderWasPaid 4. Handle DeliverEbook PayOrder command OrderWasPaid NServiceBus will add 2 messages to the queue, one for each subscriber
  32. 32. Summary Request/Response Command Event X X Automatically retries X X Full transactional support X X Reliable Consistency X Can return ID generated by the database to the sender X Loosely coupled X  Reads is synchronous and should be request/response  Using a queue for synchronous operations gives little value, only overhead  Writes can be asynchronous one-way messaging or events  There are no silver bullet – We are always trading one set of problems for another set of problems
  33. 33. Scaling  Scale up  Easy to scale up if business processes are split into multiple messages  Increase the number of threads in the config – NServiceBus will concurrently process multiple messages  Scale out  Use the Distributor or the Master (Both a distributor and a worker) Worker #1 Message MSMQ Distributor Worker #2 Worker #3  The distributor will forward the message to a worker that is ready to process the message
  34. 34. How the Distributor Works 1. Worker: Send I’m ready message I’m ready message Distributor: Control Queue I’m ready message Distributor Any messages in the input queue? PlaceOrder message 1. Website: Send PlaceOrder message Send PlaceOrder message from input queue PlaceOrder message Distributor Input Queue Yes No PlaceOrder message Distributor Find worker through message in Storage queue Store I’m ready message Storage Queue
  35. 35. Other Features  Sagas  Long running workflow like processes  State is shared between multiple message handlers  Scheduling  Send a message every x minute. A handler will handle the message  Unobtrusive mode  Use your own ICommand, IEvent etc. to reduce coupling to NServiceBus  Supports  Various dependency injection containers  Logging frameworks  Etc.  ++
  36. 36. NServiceBus License & Pricing  Open source  Used to be free, but not anymore  Indefinitely license with 1 year maintenance updates • $500 USD per processing core • $250 USD per sending core  Monthly subscription • $35 USD per processing core • $17 USD per sending core  Development and testing environments + disaster recovery and passive backups are free  Developers need a license (free) that must be renewed every 3rd month through particular.net
  37. 37. Demo
  38. 38. About Capgemini With more than 120,000 people in 40 countries, Capgemini is one of the world's foremost providers of consulting, technology and outsourcing services. The Group reported 2011 global revenues of EUR 9.7 billion. Together with its clients, Capgemini creates and delivers business and technology solutions that fit their needs and drive the results they want. A deeply multicultural organization, Capgemini has developed its own way of working, the Collaborative Business ExperienceTM, and draws on Rightshore ®, its worldwide delivery model. Rightshore® is a trademark belonging to Capgemini www.capgemini.com The information contained in this presentation is proprietary. © 2012 Capgemini. All rights reserved.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×