Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff


Published on

2011-11-02 | 05:45 PM - 06:35 PM
The JMS standard is 9 years old - but outside the Java community innovation is happening. The AMQP standard with implementations like RabbitMQ is gaining more and more traction. This session explains the standard and its advantages. It will also show how an AMQP application can be implemented using Java.

Published in: Technology

Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff

  1. 1. The Future of Messaging: RabbitMQ and AMQP Eberhard WolffArchitecture and Technology Manager adesso AG, Germany
  2. 2. Overview•  Why Messaging, AMQP and RabbitMQ•  Basic AMQP•  Exchanges•  More on Spring-AMQP
  3. 3. RPC•  Predominant approach –  RMI, SOAP Web Services, CORBA, HttpInvoker, Burlap, Hessian•  Calls remote methods with parameter•  …and waits for response
  4. 4. RPC•  Problems: –  Explicitly tells the server what to do i.e. tight coupling –  What about network failures? –  What about long latencies?
  5. 5. Why Messaging?•  Decoupling –  Data, no action i.e. receiver Component can react arbitrarily –  Asynchronous i.e. decoupled by time•  Reliable –  Message can be stored-and- Component forwarded Messages –  Redelivery until message processed•  Solves typical problems of distributed systems
  6. 6. Why Messaging?•  But: Requires different architecture•  Very different from calling remote methods•  Asynchronous•  AJAX has the same model•  See for example “Patterns of Enterprise Integration”
  7. 7. Why AMQP?•  Open standard protocol•  Standard wire protocol•  i.e. just one client library – no matter which implementation you are using•  Less vendor lock in•  Efficient –  Binary wire protocol•  Support in all major languages•  Supported on most OS platforms
  8. 8. What about JMS?•  JMS has been the default for Java messaging system for 10+ years•  But: –  Only standardized on the API level –  Less flexible than AMQP•  Mapping AMQP/JMS is being defined
  9. 9. Why Rabbit?•  Because it has a kewl name•  Numerous protocols supported•  Most popular choice on EC2•  Foundation for demanding systems e.g. NASA’s cloud initiative Nebula•  Implemented in Erlang•  Clustering built in•  Currently in 2.6.1•  Supports AMQP 0.8, 0.9, 0.9.1•  1.0 as a prototype Plug In
  10. 10. Broad Support in RabbitMQ
  11. 11. Broad Support in the JVM Space•  Grails Plug In•  Java Client•  Scala / Lift support•  We will discuss Spring support in detail•  Spring AMQP project 1.0.0• amqp
  12. 12. Why Erlang?•  Originally designed for telephone switches by Ericsson•  Much easier to develop scalable and fault tolerant systems (by factors)•  See Motorola presentation: comparing-cpp-and-erlang-for-motorola- telecoms-software•  Good tool for reliable and scalable systems
  13. 13. Erlang‘s Model MonitorLink to monitor,restart Light Light Light weight Messages weight Messages weight process process process with with with state state state
  14. 14. Why Erlang?•  Let it crash –  If a process fails, it can be easily restarted –  Different approach to fault tolerance –  Otherwise lots of error handling•  Message Passing in the Core –  RabbitMQ is a messaging system…•  Light-weight process model –  Scalabiliy
  15. 15. Very Basic AMQP•  Queues: Store messages•  Queues might be –  Durable: Survive server restarts –  Exclusive: For one connection –  autoDelete: Deleted if connection closes•  Queue usually created by consumer•  All resources are dynamic•  Producer sends a message to a Queue
  16. 16. CodeConnectionFactory conFactory = new CachingConnectionFactory ("localhost"); RabbitAdmin admin = new RabbitAdmin(conFactory); admin.declareQueue(
 new Queue("myQueue", false, true, true)); RabbitTemplate template = new RabbitTemplate(conFactory); template.convertAndSend("myQueue", "Hi AMQP!"); String receive = (String) template.receiveAndConvert("myQueue"); Assert.assertEquals("Hi AMQP!", receive);
  17. 17. Spring’s RabbitTemplate•  Send & receive message•  AmqpTemplate: Generic AMQP interface•  RabbitOperations: Rabbit specific interface: (adds just a callback)•  RabbitTemplate: Implementation•  Spring might provide support for other AMQP implementations later
  18. 18. Spring’s MessageConverter•  Messages are binary data•  RabbitTemplate uses MessageConverter to convert between objects and messages•  Can also send binary data if preferred
  19. 19. Spring’s MessageConverter•  Default: SimpleMessageConverter –  byte[] directly transferred –  String converted with configurable encoding –  Serializable are serialized –  Content type set accordingly•  JsonMessageConverter converts from / to JSON using Jackson•  MarshallingMessageConverter converts from / to XML using Springs OXM mapping•  SerializerMessageConverter uses Spring’s Serializer abstraction
  20. 20. Spring‘s AdminTemplate•  Main purpose: Configure the AMQP infrastructure•  E.g. create queues•  AmpqAdmin: Generic AMQP interface•  RabbitAdmin: Rabbit specific
  21. 21. Basics of AMQP•  Sending messages directly to queues is not enough•  What about e.g. pub / sub?•  Exchange: Route messages (stateless)•  Messages are byte-streams•  Example used the default exchange•  More dynamic, flexible and cleaner than JMS
  22. 22. AMQP  in  a  nutshell  Exchange routes messageStatelessUsually created by producerNo queue: Message discarded X Binding binds an Exchange to a Queue Queues buffer messages Usually created by consumer
  23. 23. AMQP  in  a  nutshell  Producer and Consumer might be written in Java, C#,Python, Ruby … C P X C AMQP RabbitMQ AMQP protocol protocol
  24. 24. Exchange: Route Messages X•  The type of Exchange defined the routing algorithm used•  Binding provides selector for routing•  Exchange is addressed by name•  Some standard types•  Can provide additional ones
  25. 25. Fanout Exchange X•  Broadcast to all bound queues•  Fast•  Simple•  amq.fanout is mandatory•  To broadcast information
  26. 26. Fanout Exchange X CP X C Fanout C
  27. 27. Queue fanoutQueue = new Queue("fanoutQueue"); admin.declareQueue(fanoutQueue); FanoutExchange fanoutExchange= new FanoutExchange("myFanout"); admin.declareExchange(fanoutExchange); admin.declareBinding( BindingBuilder.bind(fanoutQueue). to(fanoutExchange)); template.setExchange("myFanout"); template.convertAndSend("Hi Fanout!"); String receive = (String) template.receiveAndConvert("fanoutQueue"); Assert.assertEquals("Hi Fanout!", receive);
  28. 28. Direct Exchange X•  Routing based on one routing key• and the default Exchange (no name) always exist•  To send work orders to a specific worker
  29. 29. Direct Exchangenormalexpress Direct Exchange C express P X C normal C
  30. 30. Queue directQueue = new Queue("direct"); admin.declareQueue(directQueue); admin.declareBinding(BindingBuilder .bind(directQueue) .to(new DirectExchange("")) .with("helloKey")); template.setExchange(""); template.convertAndSend("","dropMe", "I will be dropped!"); template.convertAndSend("","helloKey", "Hi Direct!"); Assert.assertEquals("Hi Direct!", template.receiveAndConvert("direct")); Assert.assertNull( template.receiveAndConvert("direct"));
  31. 31. Topic Exchange X•  Routing based on routing pattern•  amq.topic is mandatory•  E.g. for public / subscribe scenarios
  32. 32. Topic Exchange   order.DEinvoice.USD Topic Exchange order.* P X C invoice.* C
  33. 33. Headers Exchange X•  Routing based on one or more headers and an expression•  amqp.match is mandatory•  Complex routing roles
  34. 34. Other Features•  Message can be persistent•  Request / response using correlations possible•  Redelivery / acknowledgement possible•  Clustering with e.g. Linux HA possible•  ...or send message through multiple channels and drop duplicates
  35. 35. More aboutRabbitMQ and Spring
  36. 36. Configuring Rabbit Resources with Spring•  Spring enables decoupling of your application code from the underlying infrastructure•  The container provides the resources•  The application is simply coded against the API
  37. 37. Configuring a ConnectionFactory Create an objectwith the given name and class<bean id="connectionFactory" Call setUsername() with the given value class="org.sfw.amqp.rabbit.connection.CachingConnectionFactory"> <property name="username" value="guest"/> <property name="password" value="guest"/> <constructor-arg value="localhost" /></bean> Parameter for the constructor•  Can easily modify configuration options
  38. 38. Using a ConnectionFactory from Cloud Foundry<cloud:rabbit-connection-factory id="rabbitConnectionFactory" />ConnectionFactory connectionFactory = new RabbitServiceCreator(new CloudEnvironment()) .createSingletonService().service; •  Will be provided by Cloud Foundry
  39. 39. Defining a RabbitTemplate Bean •  Provide a reference to the ConnectionFactory •  Optionally provide other references –  MessageConverter –  Routing key and exchange to be used if none is specified<bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate"> <constructor-arg ref="connectionFactory" /> <property name="routingKey" value=”invoice.USD" /></bean>
  40. 40. The MessageListener•  So far: Calling receive() on RabbitTemplate•  Needed: Something that is called when a new message appears•  The API defines this interface for asynchronous reception of messages public interface MessageListener { public void onMessage(Message) { // handle the message } }
  41. 41. Spring’s MessageListener Container•  Spring provides lightweight containers to call MessageListeners•  SimpleMessageListenerContainer•  Advanced scheduling and endpoint management options available•  i.e. thread pools, concurrent consumers, transaction handling
  42. 42. Defining a Message Listener Container<bean class="org.sfw.amqp.rabbit.listener.SimpleMessageListenerContainer"> <property name="connectionFactory" ref="connectionFactory" /> <property name="queueNames" value="my.amqp.queue" /> <property name="messageListener" ref="messageListener" /></bean>•  Every time a new message appears on my.amqp.queue the messageListener is called
  43. 43. Springs message-driven objects•  MessageListener means the receiver depends on Spring API•  Why not just a POJO?•  MessageListenerAdapter takes a POJO and makes it a MessageListener•  i.e. calls consume on Bean consumer<bean id="messageListenerAdapter" class="org.sfw.amqp.rabbit.listener.adapter.MessageListenerAdapter"> <property name="delegate" ref="consumer" /> <property name="defaultListenerMethod" value="consume" /> <property name="messageConverter" ref="jsonMessageConverter" /></bean>
  44. 44. Easier Using Namespaces<rabbit:listener-container connection-factory="connectionFactory“ message-converter="jsonMessageConverter"> <rabbit:listener ref="consumer" method="consume" queue-names="my.amqp.queue2" /></rabbit:listener-container>•  Results in the same Spring Beans
  45. 45. @Component Consumer codepublic class Consumer { public String consume(String message) { return …; } }•  No dependency on AMQP!•  But: What about the result of the method?•  Send to the Reply-To address given in message properties with same correlationId as original method
  46. 46. Client CodeString response = (String) rabbitTemplate.convertSendAndReceive( "my.fanout", "", "test");•  Message sent to destination with routing key•  Reply-To set to exclusive, autodelete, non- durable queue•  Response received through Reply-To converted and returned•  Easy request-response!•  Beware of potential latency
  47. 47. Create Environment using Namespaces •  ...if you don‘t like API calls<rabbit:fanout-exchange name="my.fanout2"> <rabbit:bindings> <rabbit:binding queue="my.amqp.queue2" /> </rabbit:bindings></rabbit:fanout-exchange><rabbit:queue name="my.amqp.queue2" /><rabbit:admin connection-factory="rabbitConnectionFactory" />
  50. 50. New Elements•  Links: unidirectional transport between source and target –  Flow control –  Settling transfers for different semantics (at most once etc)•  Processing nodes have distribution modes –  Copy: Copy message to each link –  Move: Move it to just one link
  51. 51. AMQP: Point to Point Incoming Link Receiver NodeSender mode : Receiver Outgoing move Incoming Link Link Broker
  52. 52. AMQP: Publish / Subscribe Incoming Link Receiver NodeSender mode : Receiver Outgoing copy Incoming Link Link Broker
  53. 53. Conclusion: AMQP•  Ubiquitous Messaging•  AMQP: Protocol standard•  Better scalability•  Dynamic resources
  54. 54. Conclusion: Spring AMQP•  Easy to use•  Flexible (e.g. message encoding)•  Allows scalable message handling•  Full support for AMQP and RabbitMQ
  55. 55. More••  Also a .NET version available•  …and support Spring Integration• 2011/04/01/routing-topologies-for- performance-and-scalability-with- rabbitm•  Transaction support
  56. 56. Questions? @ewolff