intro and messaging patterns
javier arias losada @javier_arilos
amqp and rabbitmq
introduction to messaging
messaging
● provides asynchronous communications
● loosely coupled modules / processes
● MoM => ...
amqp concepts (i)
amqp is a message oriented wire-level protocol
● message broker: receives and dispatches msgs
using AMQP...
amqp concepts (ii)
consumerproducer
producer consumer
Clients produce and
consume messages.
Exchanges Route and filter
mes...
escenario 1
● producing and consuming, simplest routing
examples using Python, pika and rabbitmq, everything should apply ...
step 1- connect & channel setup
from pika import BlockingConnection, ConnectionParameters
conn = BlockingConnection(Connec...
step 2- declare exchange
Declare an exchange, important parameters:
● name: if exists, nothing is done
● type: direct, fan...
step 3- declare queue
Declare a queue, important parameters:
● name: if exists, nothing is done
● durable: must queue decl...
step 4- bind queue and exchange
ch.queue_bind(exchange='important', queue='important-jobs', routing_key='important')
consu...
step 5- send the message
ch.basic_publish(exchange='important', routing_key='important', body='new important task')
consum...
step 6- consume the message
method_frame, header_frame, body = ch.basic_get('important-jobs')
print body
ch.basic_ack(meth...
escenario 2
● default exchange in more detail
● message to two queues, depending on routing key
‘important’ messages must ...
step 1- create & bind new queue
ch.queue_declare(queue='traces')
ch.queue_bind(exchange='important', queue='traces', routi...
step 2- consuming from both
consumerproducer
exchange: important
type: direct
queue: important-jobs
consumer
routing_key:
...
escenario 3
● 2 x (binding + routing-key) => to 1 queue
● exchange to exchange binding
● headers exchange
‘customer’ messa...
step 1- bind traces, declare q’s
routing_key:
customer traces
consumer
producer
update
consumer
signup
consumer
exchange: ...
exchange: important
type: direct
traces
consumer
producer
update
consumer
signup
consumer
queue: traces
queue: signup
queu...
step 3- customer signup message
ch.basic_publish(exchange='important', routing_key='customer',
body='cust num=25', propert...
bonus: topic exchanges
traces
consumer
producer
customer
consumer
signup
consumerexchange: operations
type: topic
queue: t...
escenario 4
● DeadLetter exchanges in RabbitMQ
What will we do with a message RabbitMQ cannot deliver?
(on client rejectio...
step 1- create rejected-jobs
ch.exchange_declare(exchange='rejected-jobs', type='direct')
ch.queue_declare(queue='rejected...
step 2- deadlettr important-jobs
ch.queue_delete('important-jobs')
ch.queue_declare(queue='important-jobs', arguments={'x-...
step 3- consumer rejects job
ch.basic_publish(exchange='important', routing_key='important', body='[unparseable message]')...
What are you waiting to start working with RabbitMQ?
It’s fun!
Questions?
Thank you for attending.
Upcoming SlideShare
Loading in …5
×

Rabbitmq, amqp Intro - Messaging Patterns

2,195
-1

Published on

Introduction to RabbitMQ, Amqp and some messaging patterns.
Easy to follow code examples step by step provided.
Sample code and escenarios can be found at: https://gist.github.com/javierarilos/9348168

Published in: Software
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,195
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
58
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Rabbitmq, amqp Intro - Messaging Patterns

  1. 1. intro and messaging patterns javier arias losada @javier_arilos amqp and rabbitmq
  2. 2. introduction to messaging messaging ● provides asynchronous communications ● loosely coupled modules / processes ● MoM => Message Oriented Middleware
  3. 3. amqp concepts (i) amqp is a message oriented wire-level protocol ● message broker: receives and dispatches msgs using AMQP ● connection physical connection (eg: tcp/ip) ● channel: allows n clients over one connection consumerproducer producer consumer
  4. 4. amqp concepts (ii) consumerproducer producer consumer Clients produce and consume messages. Exchanges Route and filter messages to queues: binding rules, direct, one-to-one, fanout, topic, headers Queues buffer messages between producers and consumers Messages are always: >> sent to exchanges >> consumed from queues
  5. 5. escenario 1 ● producing and consuming, simplest routing examples using Python, pika and rabbitmq, everything should apply to other languages, libraries and amqp brokers ‘important’ messages must be sent to queue ‘important-jobs’ consumerproducer source code for the example: https://gist.github.com/javierarilos/9348168
  6. 6. step 1- connect & channel setup from pika import BlockingConnection, ConnectionParameters conn = BlockingConnection(ConnectionParameters('localhost')) ch = conn.channel() consumerproducer Same code for consumer and producer
  7. 7. step 2- declare exchange Declare an exchange, important parameters: ● name: if exists, nothing is done ● type: direct, fanout, pub-sub, headers ● durable: must exchange be kept between server restarts? ● autodelete: must exchange be deleted when not in use? ● internal: is it for internal routing use, or public to clients? ch.exchange_declare(exchange='important', type='direct') consumerproducer exchange: important type: direct
  8. 8. step 3- declare queue Declare a queue, important parameters: ● name: if exists, nothing is done ● durable: must queue declaration be kept between server restarts? ● exclusive: Is this the only connection that can consume from the queue? ● autodelete: must queue be deleted when not in use? ch.queue_declare(queue='important-jobs') consumerproducer exchange: important type: direct queue: important-jobs
  9. 9. step 4- bind queue and exchange ch.queue_bind(exchange='important', queue='important-jobs', routing_key='important') consumerproducer exchange: important type: direct queue: important-jobs routing_key: important Binding a queue and exchange: ● establishes a route (exchange => queue) ● based on a criteria (exchange type + routing key) Here: when producer sends a message to ‘important’ exchange with routing key ‘important’, message will be forwarded to queue ‘important-jobs’.
  10. 10. step 5- send the message ch.basic_publish(exchange='important', routing_key='important', body='new important task') consumerproducer exchange: important type: direct queue: important-jobs routing_key: important Bonus track: default exchange, forwards to a queue, without any exchange declaration: default exchange’s name is empty string = ‘’ routing_key is the queue name ‘important-jobs’ ch.basic_publish(exchange=’’, routing_key='important-jobs', body=’def exch important task')
  11. 11. step 6- consume the message method_frame, header_frame, body = ch.basic_get('important-jobs') print body ch.basic_ack(method_frame.delivery_tag) consumerproducer exchange: important type: direct queue: important-jobs routing_key: important Messages must be acknowledged
  12. 12. escenario 2 ● default exchange in more detail ● message to two queues, depending on routing key ‘important’ messages must be sent to queues ‘important-jobs’ and ‘traces’ consumerproducer consumer
  13. 13. step 1- create & bind new queue ch.queue_declare(queue='traces') ch.queue_bind(exchange='important', queue='traces', routing_key='important') ch.basic_publish(exchange='important', routing_key='important', body='[another task to be handled]') consumerproducer exchange: important type: direct queue: important-jobs routing_key: important consumer routing_key: important queue: traces
  14. 14. step 2- consuming from both consumerproducer exchange: important type: direct queue: important-jobs consumer routing_key: important queue: traces routing_key: important ● Binding the new ‘traces’ queue to existing ‘important’ exchange does not affect publishing code. ● Consumers and queues may be added dynamically without affecting the producer.
  15. 15. escenario 3 ● 2 x (binding + routing-key) => to 1 queue ● exchange to exchange binding ● headers exchange ‘customer’ messages to ‘important’ exchange must be sent to different queues depending on the operation to perform (‘signup’, ‘update’) and to ‘traces’ traces consumer producer update consumer signup consumer
  16. 16. step 1- bind traces, declare q’s routing_key: customer traces consumer producer update consumer signup consumer exchange: important type: direct queue: traces queue: signup queue: update ‘traces’ queue is bound to ‘important’ exch with two routing keys: ‘customer’ and ‘important’ ch.queue_bind(exchange='important', queue='traces', routing_key='customer') ch.queue_declare(queue='signup') ch.queue_declare(queue='update')
  17. 17. exchange: important type: direct traces consumer producer update consumer signup consumer queue: traces queue: signup queue: update exchange: customer type: headers routing_key: customer step 2- customer exchange & bind ch.exchange_declare(exchange='customer', type='headers') ch.exchange_bind(source='important', destination='customer', routing_key='customer') ch.queue_bind(exchange='customer', queue='signup', arguments={'operation': 'signup', 'x-match':'any'}) ch.queue_bind(exchange='customer', queue='update', arguments={'operation': 'update', 'x-match':'any'})
  18. 18. step 3- customer signup message ch.basic_publish(exchange='important', routing_key='customer', body='cust num=25', properties=BasicProperties(headers=s{'operation': 'signup'})) method_frame, header_frame, msg = ch.basic_get('signup') print "msg received from queue 'signup' : ", msg ch.basic_ack(method_frame.delivery_tag) routing_key: customer traces consumer producer update consumer signup consumer exchange: important type: direct queue: traces queue: signup queue: update exchange: customer type: headers msg also routed to traces queue
  19. 19. bonus: topic exchanges traces consumer producer customer consumer signup consumerexchange: operations type: topic queue: traces queue: signup queue: update r_k: # r_k: *.signup r_k: customer.* topic exchanges allow to route messages based on topics. Examples from escenario 3: ○ Producer sends with routing keys: ‘customer.signup’, ‘customer.update’ ○ ‘traces’ consumer subscribes to ‘#’ that means all routing keys ○ ‘signup’ customer consumer subscribes to ‘customer.signup’ ○ Consumer wanting all customer operations: ‘customer.*’ ○ Consumer wanting all signup operations: *.signup’
  20. 20. escenario 4 ● DeadLetter exchanges in RabbitMQ What will we do with a message RabbitMQ cannot deliver? (on client rejection, timeout, queue length exceeded) By default those messages are dropped, we want not to lose them consumerproducer
  21. 21. step 1- create rejected-jobs ch.exchange_declare(exchange='rejected-jobs', type='direct') ch.queue_declare(queue='rejected-jobs') ch.queue_bind(exchange='rejected-jobs', queue='rejected-jobs', routing_key='important') consumerproducer queue: rejected-jobs exchange: rejected-jobs type: direct
  22. 22. step 2- deadlettr important-jobs ch.queue_delete('important-jobs') ch.queue_declare(queue='important-jobs', arguments={'x-dead-letter-exchange': 'rejected-jobs'}) ch.queue_bind(exchange='important', queue='important-jobs', routing_key='important') consumerproducer queue: rejected-jobs exchange: rejected-jobs type: direct exchange: important type: direct routing_key: important x-dead-letter-exchange: rejected-jobs
  23. 23. step 3- consumer rejects job ch.basic_publish(exchange='important', routing_key='important', body='[unparseable message]') method_frame, header_frame, important_job = ch.basic_get('important-jobs') ch.basic_reject(method_frame.delivery_tag, requeue=False) method_frame, header_frame, rejected_job = ch.basic_get('rejected-jobs') ch.basic_ack(method_frame.delivery_tag) consumerproducer queue: rejected-jobs exchange: rejected-jobs type: direct exchange: important type: direct routing_key: important x-dead-letter-exchange: rejected-jobs
  24. 24. What are you waiting to start working with RabbitMQ? It’s fun!
  25. 25. Questions? Thank you for attending.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×