Reliable integrations with NServiceBusAndreas ÖhlundEnterprise Development Expert
Agenda• War story• Dragons and how to manage them• A “saga” with a happy ending• Q &A
Fact:A trip to dragon territory cantake a while...
The usual waypublic void Handle(OrderPlaced message){    var order = new Order{ Id = message.OrderID };    order.TrackingC...
The usual waypublic void Handle(OrderPlaced message){    var order = new Order{ Id = message.OrderID };    2: order.Tracki...
The usual waypublic void Handle(OrderPlaced message){    var order = new Order{ Id = message.OrderID };    2: order.Tracki...
Fact:Dragons doesn’t supporttransactions.
Database rollbackOrder             Ship        TX             Store                     DB
Database rollbackOrder             Ship        TX             Store                     DB
Database rollbackOrder             Ship        TX             Store                     DB
Database rollbackOrder            No TX                Support             Ship        TX             Store               ...
Database rollbackDid we just loose an order?          Order                   No TX                                 Suppor...
Database rollbackOrder            No TX                Support             Ship        TX             Store               ...
Database rollback        Order            No TX                        Support                     ShipOrderHttp          ...
Design guideline“Interact with non transactional resourcesusing separate endpoints”
R# Extract endpoint public void Handle(OrderPlaced message){         repository.Save(new Order               {            ...
R# Extract endpoint public void Handle(OrderPlaced message){         repository.Save(new Order               {            ...
R# Extract endpoint public void Handle(OrderPlaced message){         repository.Save(new Order               {            ...
R# Extract endpointpublic void Handle(ShipmentBooked message){  var order = repository.Get<Order>(message.OrderID);    ord...
Consistency across    rollbacks Order         TX
Consistency across    rollbacks Order              Store                      DB         TX
Consistency across    rollbacks Order                Store                        DB         TX              Book shipment...
Consistency across    rollbacks Order                Store                        DB         TX              Book shipment...
Consistency across    rollbacks Order                Store                        DB         TX              Book shipment...
Consistency across    rollbacks Order                Store                        DB         TX              Book shipment...
Fact:Not everyone will make it back.  Not all return from a trip to the dragons
What happens if the       response is lost?BookShipment                    Fedex.Ship               TX
What happens if the       response is lost?BookShipment                      Fedex.Ship                    TimeoutExceptio...
What happens if the       response is lost?BookShipment                      Fedex.Ship                    TimeoutExceptio...
What happens if the       response is lost?BookShipment                      Fedex.Ship   Commit                    Timeou...
IdempotencyBookShipment          Integration           Endpoint
IdempotencyBookShipment                        http://fedex.com/ship ?id=xyz123          Integration           Endpoint
IdempotencyBookShipment                        http://fedex.com/ship ?id=xyz123                                           ...
IdempotencyBookShipment                        http://fedex.com/ship ?id=xyz123                                           ...
IdempotencyBookShipment                        http://fedex.com/ship ?id=xyz123                                           ...
IdempotencyBookShipment                        http://fedex.com/ship ?id=xyz123                                           ...
IdempotencyBookShipment                        http://fedex.com/ship ?id=xyz123                                           ...
Idempotency BookShipment                          http://fedex.com/ship ?id=xyz123                                        ...
Fact:A trip to the dragons may be costly.
Timeouts vNext• Thread.Sleep is not a good solution• Need a way to have durable timeouts• The NServiceBus timeout manager ...
Fact:Dragons have home turf advantage.
Scalable integrations  Use throttling to:          • Manage traffic peaks          • Allow you to control the pace
A more scalable designClient                 Server
A more scalable design            RequestClient                 Server
A more scalable design            Request             SendClient                 Server
A more scalable design                 Request                                Send         Ticket - come back in T        ...
A more scalable design                 Request                                Send         Ticket - come back in T        ...
A more scalable design                         Request                                Send                 Ticket - come b...
A more scalable design                         Request                                Send                 Ticket - come b...
A more scalable design                         Request                                Send                 Ticket - come b...
A more scalable design                         Request                                Send                 Ticket - come b...
Using sagas to controlBookShipment             message flow     Integration              Fedex                   Timeout   ...
Using sagas to controlBookShipment             message flow                   Get ticket     Integration                   ...
Using sagas to controlBookShipment             message flow                   Get ticket     Integration                   ...
Using sagas to controlBookShipment             message flow                   Get ticket     Integration                   ...
Using sagas to controlBookShipment             message flow                        Get ticket                   Wake me up ...
Using sagas to controlBookShipment             message flow                        Get ticket                   Wake me up ...
Using sagas to controlBookShipment             message flow                        Get ticket                   Wake me up ...
Using sagas to controlBookShipment             message flow                        Get ticket                   Wake me up ...
Using sagas to controlBookShipment             message flow                        Get ticket                   Wake me up ...
Code...
Thanks for listening!
Thanks for listening! Andreas Öhlund Enterprise Development Expert
Reliable integrations with NServiceBus
Reliable integrations with NServiceBus
Reliable integrations with NServiceBus
Reliable integrations with NServiceBus
Reliable integrations with NServiceBus
Reliable integrations with NServiceBus
Reliable integrations with NServiceBus
Upcoming SlideShare
Loading in...5
×

Reliable integrations with NServiceBus

2,290

Published on

Developers are dealing with more integrations today than ever before, and handling consistency and reliability across those connections isn't getting any easier either. Many developers are already using NServiceBus in the core parts of their systems for the reliability it brings but aren't aware that it can also help with integration as well. Come see how the saga capabilities in NServiceBus make integration code simpler, more robust, and testable.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,290
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
77
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Fallacies of distributed computing\n
  • Good DDD but still dangerous\nWorks well in test, but takes a long time in production\nKeeps the transactions open longer (other handlers can have opened the db connection\nFallacy: Latency is zero\n
  • Good DDD but still dangerous\nWorks well in test, but takes a long time in production\nKeeps the transactions open longer (other handlers can have opened the db connection\nFallacy: Latency is zero\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Find a way to break up the process\nIntroducing a new status\n
  • Find a way to break up the process\nIntroducing a new status\n
  • \n
  • Consistent processing of orders\n
  • Consistent processing of orders\n
  • Consistent processing of orders\n
  • Consistent processing of orders\n
  • Consistent processing of orders\n
  • \n
  • Fallacy: the network is reliable\nRetries might cause problems\nNot idempotent\n \n\n
  • Fallacy: the network is reliable\nRetries might cause problems\nNot idempotent\n \n\n
  • Fallacy: the network is reliable\nRetries might cause problems\nNot idempotent\n \n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Requires collaboration with the other party\nNServiceBus preserves message ids even across the error queue\n\n
  • Partner might charge per call\nOn a device with limited battery or where traffic cost might be high\n
  • \n
  • Sleep locks up threads, bad for throughput\nSleep doesn&amp;#x2019;t survive restarts\nMessage based alarm clock\n
  • You play according to their rules\nDragons need to control the pace of the raiding parties\nMust do throttling to be scalable\nImpossible to maintain SLA&amp;#x2019;s otherwise\n
  • \n
  • The \n
  • The \n
  • The \n
  • \n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Msmq.Send is very quick, 7000msg/s given no DTC\nAdjust T to tune the load\nResponses can be served of different servers\nCQRS\nCallback protocol can be used if the number of clients are small (REST / HTTP)\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • Picture == the essence of sagas\n * Only control logic\n\n
  • \n
  • \n
  • \n
  • If we learn to manage our dragons they can be of great use\nTell the story on how we built our own queuing system\n\n
  • If we learn to manage our dragons they can be of great use\nTell the story on how we built our own queuing system\n\n
  • Reliable integrations with NServiceBus

    1. 1. Reliable integrations with NServiceBusAndreas ÖhlundEnterprise Development Expert
    2. 2. Agenda• War story• Dragons and how to manage them• A “saga” with a happy ending• Q &A
    3. 3. Fact:A trip to dragon territory cantake a while...
    4. 4. The usual waypublic void Handle(OrderPlaced message){ var order = new Order{ Id = message.OrderID }; order.TrackingCode = fedex.BookPickup(order); order.Status = ShippingStatus.PickupBooked; repository.Save(order);}
    5. 5. The usual waypublic void Handle(OrderPlaced message){ var order = new Order{ Id = message.OrderID }; 2: order.TrackingCode = fedex.BookPickup(order); 3: order.Status = ShippingStatus.PickupBooked; repository.Save(order);}
    6. 6. The usual waypublic void Handle(OrderPlaced message){ var order = new Order{ Id = message.OrderID }; 2: order.TrackingCode = fedex.BookPickup(order); This might take a while! 3: order.Status = ShippingStatus.PickupBooked; repository.Save(order);}
    7. 7. Fact:Dragons doesn’t supporttransactions.
    8. 8. Database rollbackOrder Ship TX Store DB
    9. 9. Database rollbackOrder Ship TX Store DB
    10. 10. Database rollbackOrder Ship TX Store DB
    11. 11. Database rollbackOrder No TX Support Ship TX Store DB
    12. 12. Database rollbackDid we just loose an order? Order No TX Support Ship TX Store DB
    13. 13. Database rollbackOrder No TX Support Ship TX Store DB
    14. 14. Database rollback Order No TX Support ShipOrderHttp TX Store DB
    15. 15. Design guideline“Interact with non transactional resourcesusing separate endpoints”
    16. 16. R# Extract endpoint public void Handle(OrderPlaced message){ repository.Save(new Order { Id = message.OrderID, Status = ShippingStatus.AwaitingShipment }); bus.Send<BookShipment>(m => { m.OrderID = message.OrderID; });}
    17. 17. R# Extract endpoint public void Handle(OrderPlaced message){ repository.Save(new Order { Id = message.OrderID, Status = ShippingStatus.AwaitingShipment }); bus.Send<BookShipment>(m => { m.OrderID = message.OrderID; });}
    18. 18. R# Extract endpoint public void Handle(OrderPlaced message){ repository.Save(new Order { Id = message.OrderID, Status = ShippingStatus.AwaitingShipment }); bus.Send<BookShipment>(m => { m.OrderID = message.OrderID; });}
    19. 19. R# Extract endpointpublic void Handle(ShipmentBooked message){ var order = repository.Get<Order>(message.OrderID); order.TrackingCode = message.TrackingCode; repository.Save(order);}
    20. 20. Consistency across rollbacks Order TX
    21. 21. Consistency across rollbacks Order Store DB TX
    22. 22. Consistency across rollbacks Order Store DB TX Book shipment MQ
    23. 23. Consistency across rollbacks Order Store DB TX Book shipment MQ
    24. 24. Consistency across rollbacks Order Store DB TX Book shipment MQ
    25. 25. Consistency across rollbacks Order Store DB TX Book shipment MQ
    26. 26. Fact:Not everyone will make it back. Not all return from a trip to the dragons
    27. 27. What happens if the response is lost?BookShipment Fedex.Ship TX
    28. 28. What happens if the response is lost?BookShipment Fedex.Ship TimeoutException TX
    29. 29. What happens if the response is lost?BookShipment Fedex.Ship TimeoutException TX
    30. 30. What happens if the response is lost?BookShipment Fedex.Ship Commit TimeoutException TX
    31. 31. IdempotencyBookShipment Integration Endpoint
    32. 32. IdempotencyBookShipment http://fedex.com/ship ?id=xyz123 Integration Endpoint
    33. 33. IdempotencyBookShipment http://fedex.com/ship ?id=xyz123 Commit Integration Endpoint
    34. 34. IdempotencyBookShipment http://fedex.com/ship ?id=xyz123 Commit TimeoutException X Integration Endpoint
    35. 35. IdempotencyBookShipment http://fedex.com/ship ?id=xyz123 Commit TimeoutException X Integration Endpoint http://fedex.com/ship ?id=xyz123
    36. 36. IdempotencyBookShipment http://fedex.com/ship ?id=xyz123 Commit TimeoutException X Integration Endpoint http://fedex.com/ship ?id=xyz123 Discard
    37. 37. IdempotencyBookShipment http://fedex.com/ship ?id=xyz123 Commit TimeoutException X Integration Endpoint http://fedex.com/ship ?id=xyz123 Discard
    38. 38. Idempotency BookShipment http://fedex.com/ship ?id=xyz123 Commit TimeoutException X Integration Endpoint http://fedex.com/ship ?id=xyz123ShipmentBooked Discard
    39. 39. Fact:A trip to the dragons may be costly.
    40. 40. Timeouts vNext• Thread.Sleep is not a good solution• Need a way to have durable timeouts• The NServiceBus timeout manager solves this for us
    41. 41. Fact:Dragons have home turf advantage.
    42. 42. Scalable integrations Use throttling to: • Manage traffic peaks • Allow you to control the pace
    43. 43. A more scalable designClient Server
    44. 44. A more scalable design RequestClient Server
    45. 45. A more scalable design Request SendClient Server
    46. 46. A more scalable design Request Send Ticket - come back in T http://fedex.com/responses/xyz123Client Server
    47. 47. A more scalable design Request Send Ticket - come back in T http://fedex.com/responses/xyz123Client Server Recv Cache
    48. 48. A more scalable design Request Send Ticket - come back in T } http://fedex.com/responses/xyz123Client T Server http://fedex.com/responses/xyz123 Recv Cache
    49. 49. A more scalable design Request Send Ticket - come back in T } http://fedex.com/responses/xyz123Client T Server http://fedex.com/responses/xyz123 Recv Response / come back in T2 Cache
    50. 50. A more scalable design Request Send Ticket - come back in T } http://fedex.com/responses/xyz123Client T Server http://fedex.com/responses/xyz123 Recv Response / come back in T2 Cache
    51. 51. A more scalable design Request Send Ticket - come back in T } http://fedex.com/responses/xyz123Client T Server http://fedex.com/responses/xyz123 Recv Response / come back in T2 Cache
    52. 52. Using sagas to controlBookShipment message flow Integration Fedex Timeout Fedex Saga Endpoint Manager
    53. 53. Using sagas to controlBookShipment message flow Get ticket Integration Fedex Timeout Fedex Saga Endpoint Manager
    54. 54. Using sagas to controlBookShipment message flow Get ticket Integration Fedex Timeout Fedex Saga Endpoint Manager
    55. 55. Using sagas to controlBookShipment message flow Get ticket Integration Fedex Timeout Fedex Saga Endpoint Manager
    56. 56. Using sagas to controlBookShipment message flow Get ticket Wake me up in T Integration Fedex Timeout Fedex Saga Endpoint Manager
    57. 57. Using sagas to controlBookShipment message flow Get ticket Wake me up in T Integration Fedex Timeout Fedex Saga Endpoint Manager
    58. 58. Using sagas to controlBookShipment message flow Get ticket Wake me up in T Integration Fedex Timeout Fedex Saga Endpoint Manager Get Data
    59. 59. Using sagas to controlBookShipment message flow Get ticket Wake me up in T Integration Fedex Timeout Fedex Saga Endpoint Manager Get Data
    60. 60. Using sagas to controlBookShipment message flow Get ticket Wake me up in T Integration Fedex Timeout Fedex Saga Endpoint Manager Get Data
    61. 61. Code...
    62. 62. Thanks for listening!
    63. 63. Thanks for listening! Andreas Öhlund Enterprise Development Expert
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×