Messaging in the Cloud - AMQP, RabbitMQ and Spring

Messaging in the Cloud   Need new levels of scalability   Need a standardized wire protocol •  Won't ship a specific client for each environment •  Better interoperability   JMS only defines an API and behavior, but no protocol   AMQP defines a protocol 2

Why AMQP?   Interoperability – like TCP and unlike JMS •  RabbitMQ leading the 2009 interoperability push around AMQP 0-9-1 towards AMQP 1.0   Multiple vendors on one open, royalty-free standard •  You are not locked in •  Lower risk, lower price because of competition, easier to compare •  Products specialized around different areas of value e.g low latency, high stability, wide area   Efficient – designed for today’s pubsub and queueing needs •  Binary wire protocol •  Support in all major languages •  Supported on most OS platforms   Already in use by many major companies   Future proof – backed by Cisco, Microsoft, VMware and many more 3

Broad platform and vendor support... 4

RabbitMQ is used a lot in the Amazon Cloud   RabbitMQ preferred to Amazon SQS 5

Key AMQP messaging protocol requirements   Internet protocol - like HTTP, TCP - but ASYNCHRONOUS   Ubiquity: Open, easy & low barrier to use, understand and implement   Safety: Secure and trusted global transaction network   Fidelity: Well-stated message queuing, ordering and delivery semantics   Applicability: any broker can talk to any client, support common messaging pattern and topologies   Interoperability   Manageability: Binary, scalable 6

AMQP in a nutshell Different languages Ruby, Java … C P X C P X C AMQP AMQP protocol protocol 7

Exchange Types: Matching Algorithms   Direct •  Routes on a routing key •  Two direct exchanges always exist •  amq.direct and the default exchange (with no public name) are mandatory   Topic •  Routes on a routing pattern •  amq.direct is mandatory if the server supports topic exchanges •  Which it should according to the spec (whereas direct and fanout must be supported)   Fanout •  Simple broadcast to all bound queues (no args when binding). Fast •  amq.fanout is mandatory Any Exchange that routes Messages to more than one queue will create multiple instances of the Message. 8

AMQP in more detail Messages are stateless C P X C P X C 9

AMQP in more detail Queues buffer messages for Exchanges are push to consumers X stateless routing tables. Queues are stateful, ordered, and can be persistent, transient, private, shared. Order might change if messages are redelivered C P X C P X C 10

AMQP in more detail Queues are bound to named exchanges Binding can have a pattern e.g. “tony” (direct exchange) or “*.ibm.*” (topic exchange) C P X C P X C 11

AMQP in more detail P Producers send messages to exchanges with routing key e.g. “tony”, or ordered set of keys e.g. “buy.ibm.nyse” Exchanges route messages to queues whose binding X pattern matches the message routing key or keys C P X C P X C 12

Twi1er style pubsub message ﬂow C Anders is at P X work is at work Tony P X is at is at is at work work work C Evan Evan and Anders want to follow what Tony says: •  bind queues to a RabbitMQ exchange •  pattern “tony”. Tony publishes “is at work” to exchange using routing key “tony”. Exchange updates Evan’s and Anders’ queues Other patterns are possible e.g. for filtering by topic similar to this: http://jchris.mfdz.com/posts/64 13

Producers and consumers logically interact through a broker cloud C P C P P X X P P P C C 14

Configuring Rabbit Resources with Spring   Spring enables decoupling of your application code from the underlying infrastructure   The container provides the resources   The application is simply coded against the API 16

Configuring a ConnectionFactory <bean id="connectionFactory" class="org.sfw.amqp.rabbit.connection.CachingConnectionFactory"> <property name="username" value="guest"/> <property name="password" value="guest"/> <property name="channelCacheSize" value="42"/> <property name="hostName" value="localhost"/> </bean> Caches connection i.e. connection has to be stateless i.e. can only be used for transactional access only (connection is stateful) Otherwise use com.rabbitmq.client.ConnectionFactory 17

Queues in RabbitMQ   Queue deliver messages to at max one consumer   Messages are sent to an exchange and can be routed to one or multiple queues   Meta data about RabbitMQ Queues is stored in org.springframework.amqp.core.Queue: •  String name •  boolean durable •  boolean exclusive : private to one consumer •  boolean autoDelete   Meta data can be used with RabbitAdminTemplate •  Call declare() to actually start using the Queue   Afterwards they are identified by name   Queues are bound to exchanges with routing keys   Default: Bound using the name of the queues as routing key 18

Exchanges in RabbitMQ   Meta data about RabbitMQ Exchanges is stored •  DirectExchange : String as routing key, Queue binds to exchange with key •  FanoutExchange : No routing, what goes in must go out •  TopicExchange : Pattern as routing key   String name   boolean durable   boolean autoDelete   Each message received by an exchange will be delivered to each (qualifying) Queue bound to it 19

Spring’s Templates   AmqpTemplate: Generic AMQP interface   RabbitOperations: Rabbit specific interface: (adds just a callback)   RabbitTemplate: Implementation   Spring might provide support for other AMQP implementations later   Common interface 20

Spring’s Templates   Central point to send and receive messages   Manages resources transparently   Throws runtime exceptions   Provides convenience methods and callbacks public Message receive() public Message receive(final String queueName) public void send(MessageCreator messageCreator) public void send(String routingKey, MessageCreator messageCreator) public void send(String exchange, String routingKey, MessageCreator messageCreator) 21

MessageConverter   The RabbitTemplate uses a MessageConverter to convert between objects and messages   The default SimpleMessageConverter handles basic types •  byte[] directly transfered •  String converted to byte[] •  Serializable serialized to byte[] •  Content type set accordingly   JsonMessageConverter converts from / to JSON using Jackson   MarshallingMessageConverter converts from / to XML using Spring's OXM mapping 22

Defining a RabbitTemplate Bean   Provide a reference to the ConnectionFactory   Optionally provide other references •  MessageConverter •  Routing key and exchange to be used if none is specified <bean id=“rabbitTemplate” class=“org.springframework.amqp.rabbit.core.RabbitTemplate”> <property name=“connectionFactory” ref=“connectionFactory”/> <property name=“messageConverter” ref=“messageConverter”/> <property name=“routingKey” value=“app.stock.request”/> </bean> 23

Sending Messages   The template provides options •  One line methods that leverage the template’s MessageConverter •  Callback-accepting methods that offer more flexibility   Use the simplest option for the task at hand 24

Sending Messages with Conversion   Leveraging the template’s MessageConverter public void convertAndSend(Object object); public void convertAndSend(String routingKey, Object object); public void convertAndSend(String exchange, String routingKey, Object object); 25

Sending Messages with Callbacks   When more control is needed, use callbacks public void convertAndSend(String routingKey, Object message, MessagePostProcessor messagePostProcessor); public void send(MessageCreator messageCreator); public void send(String routingKey, MessageCreator messageCreator); public void send(String exchange, String routingKey, MessageCreator messageCreator); Message createMessage() {…} 26

Setting the reply to / correlation ID   Allows request / reply schema i.e. wait for the reply to the specific message   Planned: sendAndReceive() as a direct implementation of this pattern getRabbitTemplate().convertAndSend(tradeRequest, new MessagePostProcessor() { public Message postProcessMessage(Message message) throws AmqpException { message.getMessageProperties().setReplyTo( new Address(defaultReplyToQueue)); try { message.getMessageProperties().setCorrelationId( UUID.randomUUID().toString().getBytes("UTF-8")); } catch (UnsupportedEncodingException e) { throw new AmqpExcpetion(e); } return message; } 27 });

Synchronous Message Reception   The RabbitTemplate can receive messages also •  receive() : Message •  receive(String queueName) •  receiveAndConvert() •  receiveAndConvert(String queueName)   If no message is on the queue null is returned   If no queueName is provided the queue name or queue set at the template will be used   The MessageConverter can be leveraged for message reception as well Object someSerializable = rabbitTemplate.receiveAndConvert(); 28

The MessageListener   The API defines this interface for asynchronous reception of messages public void onMessage(Message) { // handle the message } 29

Spring’s MessageListener Containers   Spring provides lightweight containers to call MessageListeners   SimpleMessageListenerContainer   Advanced scheduling and endpoint management options available 30

Defining a plain Message Listener <bean id="messageListenerContainer" class="org.sfw.amqp.rabbit.listener.SimpleMessageListenerContainer"> <property name="connectionFactory" ref="connectionFactory" /> <property name="queueName" value="#{marketDataQueue.name},#{traderJoeQueue.name}"/> <property name="concurrentConsumers" value="5" /> <property name="messageListener" ref="messageListenerAdapter" /> </bean> 31

Spring's message-driven objects   Spring also allows you to specify a plain Java object that can serve as a listener <bean id="messageListenerAdapter" class="org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter"> <property name="delegate" ref="clientHandler" /> <property name="messageConverter" ref="jsonMessageConverter" /> </bean>   Parameter is automatically converted using a MessageConverter   Return value sent to response-destination or the reply to   Method with matching parameters is automatically called 32

Demo: Hello World   Producer send message using the RabbitTemplate   Routing key : helloWorldQueue name "hello.world.queue"   Goes to Default Exchange   …and therefore to the helloWorldQueue (routing by name)   Consumer receives message using the default receive queue (helloWorldQueue) Routing key: hello.world.queue P X C Default helloWorldQueue Exchange "hello.world.queue" 33

Demo: Stock Exchange   Server sends stock prices   Client receives stock prices   Client can issue orders   Orders are processed by the server   Client receives confirmation of the trade   Queues are private for one client 34

Server sends stock prices Routing key: RabbitMarketDataGateway app.stockes.quote.?.? e.g. called periodicaly app.stockes.quote.nasdaq.ORCL P X Topic Exchange app.stock.marketdata 35

Client receives stock prices   Binding is created to attach the exchange to the queue using the routing key Routing key: app.stock.quotes.nasdaq.* X C Topic marketDataQueue Exchange (private per consumer) app.stock.marketdata ClientHandler in SimpleMessageListenerContainer with jsonMessageConverter and a MessageListenerAdapter 36

Client can issue orders   traderJoeQueue is set as reply to for the message Routing key: app.stock.request P X Default Exchange 37

Server processes order   Reply (i.e. confirmation) is sent to the reply to (trader Joe queue) X C Default stockRequestQueue ServerHandler in Exchange "app.stock.request" SimpleMessageListenerContainer with jsonMessageConverter and a MessageListenerAdapter 38

Client receives confirmation of the trade   Binding is created to attach the exchange to the queue using the routing key X C Default Exchange traderJoeQueue ClientHandler in SimpleMessageListenerContainer with jsonMessageConverter and a MessageListenerAdapter 39

Conclusion   Ubiquitous Messaging   AMQP: Protocol standard   Better scalability   http://springsource.org/spring-amqp   Also a .NET version available 40

Messaging in the Cloud - AMQP, RabbitMQ and Spring

by Eberhard Wolff on Jul 15, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 78

Statistics

Messaging in the Cloud - AMQP, RabbitMQ and Spring — Presentation Transcript

http://www.lifeyun.com	77
http://www.slideshare.net	1