Messaging patterns

  • 11,299 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
11,299
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
288
Comments
0
Likes
35

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Messaging Patterns Álvaro Videla - Liip AGMonday, June 6, 2011
  • 2. About Me • Developer at Liip AG • Blog: http://videlalvaro.github.com/ • Twitter: @old_soundMonday, June 6, 2011
  • 3. About Me Co-authoring RabbitMQ in Action http://bit.ly/rabbitmqMonday, June 6, 2011
  • 4. Why Do I need Messaging?Monday, June 6, 2011
  • 5. An ExampleMonday, June 6, 2011
  • 6. Implement a Photo GalleryMonday, June 6, 2011
  • 7. Two Parts:Monday, June 6, 2011
  • 8. Pretty SimpleMonday, June 6, 2011
  • 9. ‘Till new requirements arriveMonday, June 6, 2011
  • 10. The Product OwnerMonday, June 6, 2011
  • 11. Can we also notify the user friends when she uploads a new image?Monday, June 6, 2011
  • 12. Can we also notify the user friends when she uploads a new image? I forgot to mention we need it for tomorrow…Monday, June 6, 2011
  • 13. The Social Media GuruMonday, June 6, 2011
  • 14. We need to give badges to users for each picture uploadMonday, June 6, 2011
  • 15. We need to give badges to users for each picture upload and post uploads to TwitterMonday, June 6, 2011
  • 16. The SysadminMonday, June 6, 2011
  • 17. Dumb! You’re delivering full size images! The bandwidth bill has tripled!Monday, June 6, 2011
  • 18. Dumb! You’re delivering full size images! The bandwidth bill has tripled! We need this fixed for yesterday!Monday, June 6, 2011
  • 19. The Developer in the other teamMonday, June 6, 2011
  • 20. I need to call your PHP stuff but from PythonMonday, June 6, 2011
  • 21. I need to call your PHP stuff but from Python And also Java starting next weekMonday, June 6, 2011
  • 22. The UserMonday, June 6, 2011
  • 23. I don’t want to wait till your app resizes my image!Monday, June 6, 2011
  • 24. YouMonday, June 6, 2011
  • 25. FML!Monday, June 6, 2011
  • 26. Let’s see the code evolutionMonday, June 6, 2011
  • 27. First Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok.Monday, June 6, 2011
  • 28. Second Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), ok.Monday, June 6, 2011
  • 29. Third Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), ok.Monday, June 6, 2011
  • 30. Fourth Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), ok.Monday, June 6, 2011
  • 31. Final Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), tweet_new_image(User, Image), ok.Monday, June 6, 2011
  • 32. Can our code scale to new requirements?Monday, June 6, 2011
  • 33. What ifMonday, June 6, 2011
  • 34. What if • We need to speed up image conversionMonday, June 6, 2011
  • 35. What if • We need to speed up image conversion • User notification has to be sent by emailMonday, June 6, 2011
  • 36. What if • We need to speed up image conversion • User notification has to be sent by email • Stop tweeting about new imagesMonday, June 6, 2011
  • 37. What if • We need to speed up image conversion • User notification has to be sent by email • Stop tweeting about new images • Resize in different formatsMonday, June 6, 2011
  • 38. Can we do better?Monday, June 6, 2011
  • 39. Sure. Using messagingMonday, June 6, 2011
  • 40. Design Publish / Subscribe PatternMonday, June 6, 2011
  • 41. First Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message(new_image, Msg).Monday, June 6, 2011
  • 42. First Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message(new_image, Msg). %% friends notifier on(new_image, Msg) -> notify_friends(Msg.user, Msg.image).Monday, June 6, 2011
  • 43. First Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message(new_image, Msg). %% friends notifier on(new_image, Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on(new_image, Msg) -> add_points(Msg.user, new_image).Monday, June 6, 2011
  • 44. First Implementation: %% image_controller handle(PUT, "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message(new_image, Msg). %% friends notifier on(new_image, Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on(new_image, Msg) -> add_points(Msg.user, new_image). %% resizer on(new_image, Msg) -> resize_image(Msg.image).Monday, June 6, 2011
  • 45. Second Implementation:Monday, June 6, 2011
  • 46. Second Implementation: %% there’s none.Monday, June 6, 2011
  • 47. MessagingMonday, June 6, 2011
  • 48. Messaging • Share data across processesMonday, June 6, 2011
  • 49. Messaging • Share data across processes • Processes can be part of different appsMonday, June 6, 2011
  • 50. Messaging • Share data across processes • Processes can be part of different apps • Apps can live in different machinesMonday, June 6, 2011
  • 51. Messaging • Share data across processes • Processes can be part of different apps • Apps can live in different machines • Communication is AsynchronousMonday, June 6, 2011
  • 52. Main ConceptsMonday, June 6, 2011
  • 53. Main Concepts • Messages are sent by ProducersMonday, June 6, 2011
  • 54. Main Concepts • Messages are sent by Producers • Messages are delivered to ConsumersMonday, June 6, 2011
  • 55. Main Concepts • Messages are sent by Producers • Messages are delivered to Consumers • Messages goes through a ChannelMonday, June 6, 2011
  • 56. Messaging and RabbitMQMonday, June 6, 2011
  • 57. What is RabbitMQ?Monday, June 6, 2011
  • 58. RabbitMQ • Enterprise Messaging System • Open Source MPL • Written in Erlang/OTP • Commercial Support • Messaging via AMQPMonday, June 6, 2011
  • 59. Features • Reliable and High Scalable • Easy To install • Easy To Cluster • Runs on: Windows, Solaris, Linux, OSX • AMQP 0.8 - 0.9.1Monday, June 6, 2011
  • 60. Client Libraries • Java • .NET/C# • Erlang • Ruby, Python, PHP, Perl, AS3, Lisp, Scala, Clojure, HaskellMonday, June 6, 2011
  • 61. AMQP • Advanced Message Queuing Protocol • Suits Interoperability • Completely Open Protocol • Binary ProtocolMonday, June 6, 2011
  • 62. Message Flow http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.htmlMonday, June 6, 2011
  • 63. AMQP Model • Exchanges • Message Queues • Bindings • Rules for binding themMonday, June 6, 2011
  • 64. Exchange Types • Fanout • Direct • TopicMonday, June 6, 2011
  • 65. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Fanout_Exchange.htmlMonday, June 6, 2011
  • 66. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Direct_Exchange.htmlMonday, June 6, 2011
  • 67. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Topic_Exchange.htmlMonday, June 6, 2011
  • 68. Messaging PatternsMonday, June 6, 2011
  • 69. There are many messaging patterns http://www.eaipatterns.com/Monday, June 6, 2011
  • 70. Basic PatternsMonday, June 6, 2011
  • 71. Competing Consumers How can a messaging client process multiple messages concurrently?Monday, June 6, 2011
  • 72. Competing Consumers Create multiple Competing Consumers on a single channel so that the consumers can process multiple messages concurrently.Monday, June 6, 2011
  • 73. Competing ConsumersMonday, June 6, 2011
  • 74. Publisher Code init(Exchange, Queue) -> #exchange.declare{exchange = Exchange, type = <<"direct">>, durable = true}, #queue.declare{queue = Queue, durable = false}, #queue.bind{queue = Queue, exchange = Exchange}. publish_msg(Exchange, Payload) -> Props = #P_basic{content_type = <<"application/json">>, delivery_mode = 2}, %% persistent publish(Exchange, #amqp_msg{props = Props, payload = Payload}).Monday, June 6, 2011
  • 75. Consumer Code init_consumer(Exchange, Queue) -> init(Exchange, Queue), #basic.consume{ticket = 0, queue = Queue}. on(#basic.deliver{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> do_something_with_msg(Msg), #basic.ack{delivery_tag = DeliveryTag}.Monday, June 6, 2011
  • 76. Publish/Subscribe How can the sender broadcast an event to all interested receivers?Monday, June 6, 2011
  • 77. Publish/Subscribe Send the event on a Publish-Subscribe Channel, which delivers a copy of a particular event to each receiver.Monday, June 6, 2011
  • 78. Publish/SubscribeMonday, June 6, 2011
  • 79. Publisher Code init(Exchange, Queue) -> #exchange.declare{exchange = Exchange, type = <<"fanout">>, %% different type durable = true} %% same as before ... publish_msg(Exchange, Payload) -> Props = #P_basic{content_type = <<"application/json">>, delivery_mode = 2}, %% persistent publish(Exchange, #amqp_msg{props = Props, payload = Payload}).Monday, June 6, 2011
  • 80. Consumer Code A init_consumer(Exchange, ResizeImageQueue) -> init(Exchange, ResizeImageQueue), #basic.consume{queue = ResizeImageQueue}. on(#basic.deliver{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> resize_message(Msg), #basic.ack{delivery_tag = DeliveryTag}.Monday, June 6, 2011
  • 81. Consumer Code B init_consumer(Exchange, NotifyFriendsQueue) -> init(Exchange, NotifyFriendsQueue), #basic.consume{queue = NotifyFriendsQueue}. on(#basic.deliver{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> notify_friends(Msg), #basic.ack{delivery_tag = DeliveryTag}.Monday, June 6, 2011
  • 82. Consumer Code C init_consumer(Exchange, LogImageUpload) -> init(Exchange, LogImageUpload), #basic.consume{queue = LogImageUpload}. on(#basic.deliver{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> log_image_upload(Msg), #basic.ack{delivery_tag = DeliveryTag}.Monday, June 6, 2011
  • 83. Request/Reply When an application sends a message, how can it get a response from the receiver?Monday, June 6, 2011
  • 84. Request/Reply Send a pair of Request-Reply messages, each on its own channel.Monday, June 6, 2011
  • 85. Request/ReplyMonday, June 6, 2011
  • 86. Return Address How does a replier know where to send the reply?Monday, June 6, 2011
  • 87. Return Address The request message should contain a Return Address that indicates where to send the reply message.Monday, June 6, 2011
  • 88. Return AddressMonday, June 6, 2011
  • 89. Correlation Identifier How does a requestor that has received a reply know which request this is the reply for?Monday, June 6, 2011
  • 90. Correlation Identifier Each reply message should contain a Correlation Identifier, a unique identifier that indicates which request message this reply is for.Monday, June 6, 2011
  • 91. Correlation IdentifierMonday, June 6, 2011
  • 92. Putting it all togetherMonday, June 6, 2011
  • 93. RPC Client init() -> #queue.declare_ok{queue = SelfQueue} = #queue.declare{exclusive = true, auto_delete = true}, #basic.consume{queue = SelfQueue, no_ack = true}, SelfQueue.Monday, June 6, 2011
  • 94. RPC Client init() -> #queue.declare_ok{queue = SelfQueue} = #queue.declare{exclusive = true, auto_delete = true}, #basic.consume{queue = SelfQueue, no_ack = true}, SelfQueue. request(Payload, RequestId) -> Props = #P_basic{correlation_id = RequestId, reply_to = SelfQueue}, publish(ServerExchange, #amqp_msg{props = Props, payload = Payload}).Monday, June 6, 2011
  • 95. RPC Client init() -> #queue.declare_ok{queue = SelfQueue} = #queue.declare{exclusive = true, auto_delete = true}, #basic.consume{queue = SelfQueue, no_ack = true}, SelfQueue. request(Payload, RequestId) -> Props = #P_basic{correlation_id = RequestId, reply_to = SelfQueue}, publish(ServerExchange, #amqp_msg{props = Props, payload = Payload}). on(#basic.deliver{}, #amqp_msg{props = Props, payload = Payload}) -> CorrelationId = Props.correlation_id, do_something_with_reply(Payload).Monday, June 6, 2011
  • 96. RPC Server on(#basic.deliver{}, #amqp_msg{props = Props, payload = Payload}) -> CorrelationId = Props.correlation_id, ReplyTo = Props.reply_to, Reply = process_request(Payload), NewProps = #P_basic{correlation_id = CorrelationId}, publish("", %% anonymous exchange #amqp_msg{props = NewProps, payload = Reply}, ReplyTo). %% routing keyMonday, June 6, 2011
  • 97. Advanced PatternsMonday, June 6, 2011
  • 98. Control Bus How can we effectively administer a messaging system that is distributed across multiple platforms and a wide geographic area?Monday, June 6, 2011
  • 99. Control Bus Use a Control Bus to manage an enterprise integration system.Monday, June 6, 2011
  • 100. Control Bus • Send Configuration Messages • Start/Stop Services • Inject Test Messages • Collect StatisticsMonday, June 6, 2011
  • 101. Control BusMonday, June 6, 2011
  • 102. Control Bus Make Services “Control Bus” EnabledMonday, June 6, 2011
  • 103. Detour How can you route a message through intermediate steps to perform validation, testing or debugging functions?Monday, June 6, 2011
  • 104. Detour Construct a Detour with a context-based router controlled via the Control Bus. In one state the router routes incoming messages through additional steps while in the other it routes messages directly to the destination channel.Monday, June 6, 2011
  • 105. DetourMonday, June 6, 2011
  • 106. Wire Tap How do you inspect messages that travel on a point-to-point channel?Monday, June 6, 2011
  • 107. Wire Tap Insert a simple Recipient List into the channel that publishes each incoming message to the main channel and a secondary channel.Monday, June 6, 2011
  • 108. Wire Tap How do you inspect messages that travel on a point-to-point channel?Monday, June 6, 2011
  • 109. Wire Tap Insert a simple Recipient List into the channel that publishes each incoming message to the main channel and a secondary channel.Monday, June 6, 2011
  • 110. Wire TapMonday, June 6, 2011
  • 111. Smart Proxy How can you track messages on a service that publishes reply messages to the Return Address specified by the requestor?Monday, June 6, 2011
  • 112. Smart Proxy Use a Smart Proxy to store the Return Address supplied by the original requestor and replace it with the address of the Smart Proxy. When the service sends the reply message route it to the original Return Address.Monday, June 6, 2011
  • 113. Smart ProxyMonday, June 6, 2011
  • 114. Credits Pattern graphics and description taken from: http://www.eaipatterns.com/Monday, June 6, 2011
  • 115. Thanks! @old_sound http://vimeo.com/user1169087 http://www.slideshare.net/old_soundMonday, June 6, 2011