Presentation for ITCamp 2013. www.itcamp.ro Talking about different patterns that can be used when we need to deal with messages. Sample code is based on Windows Azure Service Bus

1. itcampro@ itcamp13# Premium conference on Microsoft technologies
Messaging Patterns in the
Cloud
Radu Vunvulea
iQuest Group
@RaduVunvulea
http://vunvulearadu.blogspot.com

2. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudHuge thanks to our sponsors!

3. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• World of Messages
• On-Premise and Cloud solutions
• Messaging Patterns
• Costs, Benefits and Limitations
Agenda

4. itcampro@ itcamp13# Premium conference on Microsoft technologies
WORLD OF MESSAGES

5. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudWorld of Messages

6. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudWorld of Messages

7. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudWorld of Messages

8. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudWorld of Messages

9. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudWorld of Messages
Message
Message
Message

10. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus

11. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Interaction and communication between
software applications

12. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Topic
Subscriptions

13. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Topic
Subscriptions

14. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus
Topic
Subscriptions

15. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudEnterprise Service Bus

16. itcampro@ itcamp13# Premium conference on Microsoft technologies
ON-PREMISE AND CLOUD
SOLUTIONS

17. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Pro
– Open Source
– Flexible
– Good price
– Transactional queue read/write
– Flexible load distribution
• Cons
– No queue management
– Document not up to date
– Only one queue technology supported (MSMQ)
NService Bus

18. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Pro
– Free for commercial use
– Supports MSMQ and RabittMQ
– Open Source
– Good administration console
• Cons
– No commercial support
– Not as fast as NServiceBus
MassTransit

19. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Pro
– Scalable
– Lots of features
– Good administration console
– Support
– Cross platform and multiple environments
• Cons
– Is not free
– Latency
Azure Service Bus

20. itcampro@ itcamp13# Premium conference on Microsoft technologies
MESSAGING PATTERNS

21. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Filter

22. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Filter
Topic
Subscriptions

23. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Custom rules based on message attributes
• Add attributes to message
Message Filter
BrokeredMessage msg = new BrokeredMessage(messageContent);
msg.Properties["colorCode"] = 1;
• Create subscription with custom rule
RuleDescription rule = new RuleDescription()
{
// 1 - green, 2 - orange
Filter = new SqlFilter("colorCode = 1");
}
namespaceManager.CreateSubscription(
"itcamp", "greensubscription", rule);

24. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Splitter

25. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Splitter
Topic
Subscriptions

26. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Group messages based on session
Message Splitter
MessageSession msgSession =
msgReceiver.AcceptMessageSession();
BrokeredMessage message;
while ((message = msgSession.Receive()) != null)
{
// Process message
}
msgSession.Complete()
BrokeredMessage message = new BrokereMessage(content);
message.SessionId = "sessionId";
• Consume messages based on session id

27. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Group messages based on session
Message Splitter
MessageSession msgSession =
msgReceiver.AcceptMessageSession();
BrokeredMessage message;
while ((message = msgSession.Receive()) != null)
{
// Process message
}
msgSession.Complete()
BrokeredMessage message = new BrokereMessage(content);
message.SessionId = "sessionId";
• Consume messages based on session id

28. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Aggregator

29. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Aggregator
Topic
Subscriptions

30. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Group messages based on session
Message Aggregator
BrokeredMessage message = new BrokereMessage(content);
message.SessionId = "sessionId";
• Use correlation id and filters
BrokereMessage message = new BrokereMessage(content);
message.CorrelationId = “CJ"
topic.Send(message);
namespaceManager.CreateSubscription(
“itcamp”,
“iquestsubscriptioncar”,
new CorrelationFilterExpression(“CJ"));

31. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer

32. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer

33. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer
Topic
Subscriptions
1
2
3
4
1
2
3
4
1234

34. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer
Topic
Subscriptions
1
2
3
4
1
2
3
4
3412

35. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Session, message index and message count
Message Resequencer
• Mark message as dead letter when in
incorrect order
• Iterate dead letter queue to access a
message with a lower index
BrokeredMessage message = new BrokereMessage(content);
message.Properties["index"] = 2;
message.Properties["count"] = 10
message.SessionId = “CJ-15-IQU";

36. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Resequencer
MessageSession messageSession =
queueClient.AcceptMessageSession();
int currentIndex = 1;
while(true)
{
BrokeredMessage message = messageSession.Receive();
if(int.Parse(message.Properties[“index”]) != currentIndex)
{
message.DeadLetter();
continue;
}
…
message.Complete();
if(int.Parse(messsage[“count”]) == currentIndex)
{
break;
}
currentIndex++;
}

37. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudMessage Recipient
Topic
Subscriptions

38. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Use a property to specific the list of groups
Message Recipient
• Define custom filters using LIKE operator
BrokeredMessage message = new BrokeredMessage(content);
message.Properties[“groups”] = “orange magenta”;
SqlFilter filter = new SqlFilter(
“groups LIKE „%orange%‟”);
topic.AddSubscription(“subscription3”, filter);
• OR different properties for each group type
SqlFilter filter = new SqlFilter(
“EXISTS orange”);
topic.AddSubscription(“subscription3”, filter);

39. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Use a property to specific the list of groups
Message Recipient
• Define custom filters using LIKE operator
BrokeredMessage message = new BrokeredMessage(content);
message.Properties[“groups”] = “orange magenta”;
SqlFilter filter = new SqlFilter(
“groups LIKE „%orange%‟”);
topic.AddSubscription(“subscription3”, filter);
• OR different properties for each group type
• Use only one property and a prime number
for each group

40. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudContent-Based Router
Topic
Subscriptions

41. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudContent-Based Router
Topic
Subscriptions

42. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudScatter-Gather
Topic
Subscriptions
Queue

43. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudScatter-Gather
Topic
Subscriptions
Queue

44. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudDynamic Router
Topic
Subscriptions

45. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudDynamic Router
Topic
Subscriptions

46. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public CloudDynamic Router
Topic
Subscriptions

47. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• [key, values] properties
• Store properties
• Decorate properties
Dynamic Router

48. itcampro@ itcamp13# Premium conference on Microsoft technologies
COSTS, BENEFITS AND
LIMITATIONS

49. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• What are the costs of processing:
– 24kB message size
– 10M messages
– 1 Topic
- 8 hours
Costs

50. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
Costs

51. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
• 10 $ Receiving
• 0 $ Bandwidth (receiving)
Costs

52. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
• 10 $ Receiving
• 0 $ Bandwidth (receiving)
• 47.46$ Costs related to Service Bus
Costs

53. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• 10$ Sending
• 27.46$ Bandwidth (sending)
• 10 $ Receiving
• 0 $ Bandwidth (receiving)
• 47.46$ Costs related to Service Bus
• 8.64$ 4 Medium Worker Roles used to
consume messages
• 56.1$ Total cost
Costs

54. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Cheap
• 99.9% Uptime
• Extremely scalable and flexible
• No message is lost
• Filters and actions support
• REST API
• Death-letter and transaction support
• Same API for on-premise and cloud
Benefits

55. itcampro@ itcamp13# Premium conference on Microsoft technologies
Private &
Public Cloud
• Not for real-time application
• When processing more than 1M messages
on the same topic in a 30 minutes time
interval latency increases
• You need to pay for each send/receive
command
• Batch maximum size – 100 messages
Limitations

56. itcampro@ itcamp13# Premium conference on Microsoft technologies
Q & A

57. itcampro@ itcamp13# Premium conference on Microsoft technologies
THANK YOU