Easy enterprise application integration with RabbitMQ and AMQP


Published on

VMware vFabric RabbitMQ Technical Webinar December 2010 by VMware engineer Emile Joubert. Covers common integration patterns, and how RabbitMQ makes these easily implemented, using AMQP as a communications mechanism.

You can view a recording of this presentation on YouTube: http://www.youtube.com/user/SpringSourceDev#p/c/5956C6D9EC319817/0/ABGMjX4K0D8

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Easy enterprise application integration with RabbitMQ and AMQP

  1. 1. Emile Joubert December 2010 The Advanced Message Queuing Protocol (AMQP) is an application layer protocol for messaging. This talk introduces RabbitMQ, its prominent multi-lingual open-source implementation.
  2. 2. Disclaimer <ul><li>This session may contain product features that are currently under development. </li></ul><ul><li>This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. </li></ul><ul><li>Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. </li></ul><ul><li>&quot;THESE FEATURES ARE REPRESENTATIVE OF FEATURE AREAS UNDER DEVELOPMENT. FEATURE COMMITMENTS ARE SUBJECT TO CHANGE, AND MUST NOT BE INCLUDED IN CONTRACTS, PURCHASE ORDERS, OR SALES AGREEMENTS OF ANY KIND. TECHNICAL FEASIBILITY AND MARKET DEMAND WILL AFFECT FINAL.&quot; </li></ul>
  3. 3. About me... <ul><li>Emile studied computer science and mathematics before following a career that included electronic publishing; joining LShift in 2008 and SpringSource in 2010. </li></ul>
  4. 4. Outline <ul><li>Messaging </li></ul><ul><li>AMQP </li></ul><ul><li>Publishing walk-through </li></ul><ul><li>RabbitMQ and the benefits of Erlang </li></ul><ul><li>Large scale sample uses </li></ul>
  5. 5. Messaging Use Cases Decoupling
  6. 6. Messaging Use Cases Bidirectional Decoupling e.g. Remote Procedure Call
  7. 7. Messaging Use Cases Pipelining and Decoupling
  8. 8. Messaging Use Cases Work-distribution and Decoupling e.g. duplication and round-robin
  9. 9. Messaging Use Cases Work aggregation, distribution and decoupling
  10. 10. Why Messaging? <ul><li>SQL : : Databases : : Filesystem </li></ul><ul><li>AMQP : : Messaging : : Network </li></ul><ul><li>Logical decoupling </li></ul><ul><li>Physical decoupling </li></ul><ul><li>Temporal decoupling </li></ul>
  11. 11. AMQP <ul><li>An open, wire-level binary protocol for Message Orientated Middleware </li></ul><ul><li>Created by users and technologists working together, to satisfy real needs </li></ul><ul><li>A straight-forward and complete solution for business messaging: SMTP for business </li></ul>
  12. 12. Transport Framing AMQP Wire Format AMQP Model Network-friendly Network-optimized Multi-vendor interoperability Flexible routing/ queuing parameters AMQP Model
  13. 13. AMQP Specification Sample < method name=&quot; publish &quot; content=&quot; 1 &quot; index=&quot; 40 &quot;> < chassis name=&quot; server &quot; implement=&quot; MUST &quot;/> < field name=&quot; reserved-1 &quot; type=&quot; short “ /> < field name=&quot; exchange &quot; domain=&quot; exchange-name &quot;/> < field name=&quot; routing-key &quot; domain=&quot; shortstr &quot;/> < field name=&quot; mandatory &quot; domain=&quot; bit &quot;/> < field name=&quot; immediate &quot; domain=&quot; bit &quot;/> </ method >
  14. 14. AMQP Model <ul><li>Producers </li></ul><ul><li>Exchanges </li></ul><ul><li>Bindings </li></ul><ul><li>Queues </li></ul><ul><li>Consumers </li></ul>
  15. 15. <ul><li>Exchanges, queues & bindings are created dynamically by producers & consumers </li></ul>Publishing: Setup
  16. 16. Publishing
  17. 17. Publishing
  18. 18. Publishing Publishing a second message
  19. 19. Consuming
  20. 20. Consuming Consuming second message
  21. 21. Publishing Multiple Messages
  22. 22. Publishing Multiple Messages ...to several consumers on the same queue
  23. 23. Publishing to Multiple Queues
  24. 24. Publishing to Multiple Queues Sending
  25. 25. Publishing to Multiple Queues Messages go to every queue bond to a fanout exchange
  26. 26. Publishing to Multiple Queues Queues can be consumed from at different rates
  27. 27. Publishing: Topic Routing <ul><li>* matches 1 word </li></ul><ul><li># matches 0 or more words </li></ul>
  28. 28. Publishing: Topic Routing
  29. 29. AMQP Model Elements <ul><li>Exchange Properties </li></ul><ul><ul><li>passive </li></ul></ul><ul><ul><li>durable </li></ul></ul><ul><ul><li>auto-delete </li></ul></ul><ul><li>Exchange Types </li></ul><ul><ul><li>direct </li></ul></ul><ul><ul><li>fanout </li></ul></ul><ul><ul><li>topic </li></ul></ul><ul><ul><li>headers </li></ul></ul><ul><li>Queue Properties </li></ul><ul><ul><li>passive </li></ul></ul><ul><ul><li>durable </li></ul></ul><ul><ul><li>exclusive </li></ul></ul><ul><ul><li>auto-delete </li></ul></ul><ul><li>Message Properties </li></ul><ul><ul><li>routing key </li></ul></ul><ul><ul><li>immediate </li></ul></ul><ul><ul><li>delivery-mode </li></ul></ul><ul><ul><li>priority </li></ul></ul><ul><ul><li>expiration </li></ul></ul>
  30. 30. Prominent AMQP Implementations RabbitMQ (Erlang) OpenAMQ (C++) Apache Qpid (Java and C++) RedHat MRG (Messaging, Realtime & Grid) (C++)
  31. 31. Why Erlang? <ul><li>Concise (binary matching) </li></ul><ul><li>Sane concurrency (no shared state) </li></ul><ul><li>Built-in distributed database (Mnesia) </li></ul><ul><li>Multi-platform (Windows, Unix) </li></ul>
  32. 32. Erlang binary matching decode_method ('basic.publish', << F0 : 16 /unsigned, F1Len :8/unsigned, F1 : F1Len /binary, F2Len :8/unsigned, F2 : F2Len /binary, F3Bits :8>>) -> F3 = ( F3Bits band 1) /= 0, F4 = ( F3Bits band 2) /= 0, #'basic.publish'{ticket = F0 , exchange = F1 , routing_key = F2 , mandatory = F3 , immediate = F4 };
  33. 33. What is RabbitMQ? <ul><li>RabbitMQ consists of the broker (server) and several clients (Java, .net and others) </li></ul><ul><li>They speak the AMQP protocol to each other </li></ul><ul><li>Anyone can write other clients which also speak AMQP and work with RabbitMQ </li></ul><ul><ul><li>e.g. Ruby, Java, Spring, Python, C, C#, Perl, Erlang, Lisp, Haskell, PHP, <your language here> </li></ul></ul><ul><li>It's in many popular Linux distributions and can easily be installed on OS X </li></ul><ul><li>Released under MPL </li></ul>
  34. 34. What is RabbitMQ? <ul><li>17k lines of Erlang code in the broker </li></ul><ul><li>Supports clustering for increased scalability </li></ul><ul><li>Can work with Pacemaker and associated tools to provide High Availability </li></ul><ul><li>Supports several extension points via plug-ins, e.g. exchange types and backing queue storage </li></ul>
  35. 35. AMQP compared to JMS <ul><li>AMQP has a wire-level binary protocol, instead of an API </li></ul><ul><li>AMQP offers multi-language interoperability </li></ul><ul><li>AMQP message payloads are opaque </li></ul><ul><li>Use Spring-AMQP to migrate </li></ul>
  36. 36. Protocols AMQP (SSL) STOMP XMPP (Rabbiter) 0MQ REST
  37. 37. Real-world features <ul><li>Unroutable messages </li></ul><ul><li>Client acknowledgements </li></ul><ul><li>Persistence </li></ul><ul><li>Transactions </li></ul><ul><li>Flow control </li></ul><ul><li>Security </li></ul>
  38. 38. Management & Monitoring <ul><li>RabbitMQ plugin </li></ul><ul><li>Inspect and manage resources </li></ul><ul><li>Load & save configuration </li></ul><ul><li>HTTP API </li></ul><ul><li>Commandline tool </li></ul>
  39. 39. Management & Monitoring
  40. 40. Management & Monitoring
  41. 41. Cloud Ready <ul><li>Messaging service hosted in cloud </li></ul><ul><li>Cloud infrastructure services </li></ul><ul><ul><li>e.g. Nasa Nebula </li></ul></ul><ul><li>Providing a service in the cloud </li></ul><ul><ul><li>e.g. SQS on Amazon </li></ul></ul>
  42. 42. Nebula Architecture
  43. 43. UIDAI
  44. 44. UIDAI <ul><li>Unique Identification Authority of India (UIDAI) will issue a 12-digit for all residents </li></ul><ul><li>One of the largest IT projects ever </li></ul><ul><li>Largest online identity DB ever </li></ul><ul><li>200 million messages per day peak </li></ul><ul><li>600 million residents over 4 years </li></ul><ul><li>Spring </li></ul><ul><li>Tomcat </li></ul><ul><li>Hibernate </li></ul><ul><li>MySQL </li></ul><ul><li>Hadoop </li></ul><ul><li>Mule </li></ul><ul><li>Apache httpd </li></ul><ul><li>RabbitMQ </li></ul>
  45. 45. The future <ul><li>AMQP 1.0 </li></ul><ul><ul><li>Connect-a-thon </li></ul></ul><ul><li><Your suggestion here> </li></ul><ul><ul><li>Get in touch! </li></ul></ul><ul><ul><li>http://www.rabbitmq.com / </li></ul></ul><ul><li>This presentation will be available at </li></ul><ul><li>http://www.springsource.com/webinars </li></ul><ul><li>from Monday 13 Deccember 2010 </li></ul>