Your SlideShare is downloading. ×

Practical Message Queuing Using RabbitMQ (PHPem, 3rd July 2014)

765

Published on

RabbitMQ is a message broker - an application that allows communication between applications by way of a message queuing system. In this talk, we’ll set up an RabbitMQ instance, take an …

RabbitMQ is a message broker - an application that allows communication between applications by way of a message queuing system. In this talk, we’ll set up an RabbitMQ instance, take an intermediate-level look into the technical features it provides and also how you can apply RabbitMQ in your in applications to scale them efficiently.

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
765
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
16
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Practical Message Queueing Using RabbitMQ James Titcumb PHPem 3rd July 2014
  • 2. James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim Who is this guy?
  • 3. Who are you? https://www.flickr.com/photos/akrabat/10168019755/
  • 4. What is message queueing?
  • 5. Separation of Concerns
  • 6. Scaling with Rabbit RabbitMQApplication Background processing
  • 7. Scaling with Rabbit RabbitMQApplication Background processing Background processing
  • 8. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing
  • 9. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing
  • 10. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing Background processing
  • 11. ● Fast logging solution Some real world uses...
  • 12. ● Fast logging solution ● Background Processing Some real world uses...
  • 13. ● Fast logging solution ● Background Processing ○ Sending emails Some real world uses...
  • 14. ● Fast logging solution ● Background Processing ○ Sending emails ○ Sending SMS Some real world uses...
  • 15. ● Fast logging solution ● Background Processing ○ Sending emails ○ Sending SMS ○ Analytics, reporting Some real world uses...
  • 16. (on precise64, other OSs may vary) Installing RabbitMQ
  • 17. ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart Using Apt
  • 18. http://localhost:15672/
  • 19. Basic Message Queuing
  • 20. Objective: Basic Queuing Producer Consumer test_queue 1 2 3 4 5
  • 21. composer.json { "require": { "videlalvaro/php-amqplib": "2.*" } } then composer install
  • 22. Please wait, connecting... use PhpAmqpLibConnectionAMQPConnection; $connection = new AMQPConnection( 'localhost', 5672, 'guest', 'guest', '/' ); $channel = $connection->channel();
  • 23. basic/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->queue_declare( 'test_queue', false, true, false, false); $message = new AMQPMessage('my test message'); $channel->basic_publish($message, '', 'test_queue');
  • 24. basic/consumer.php $channel->basic_consume( 'test_queue', // Queue to consume '', // Consumer identifier false, true, // No-ack means messages are "auto acknowledged" false, // Exclusive - no other consumers can use the queue false, function(AMQPMessage $message) { echo $message->body . "n"; } ); while (count($channel->callbacks)) { $channel->wait(); }
  • 25. What to expect...
  • 26. Exchanges: Fanout
  • 27. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  • 28. fanout/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $message = new AMQPMessage('my test message #' . $id); $channel->basic_publish($message, 'test_exchange');
  • 29. fanout/consumer.php $q = $channel->queue_declare( '', // Lets RabbitMQ pick a name for queue false, false, false, true // Delete this queue ); $queue_name = $q[0]; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $channel->queue_bind($queue_name, 'test_exchange');
  • 30. What to expect...
  • 31. A word on Temporary Queues ● Only the “current” flow ● Specific use cases test_exchangeProducer Messages go nowhere
  • 32. Exchanges: Direct
  • 33. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer 3 Message Routing Keys 1 = orange 2 = banana 3 = apple 2 3 BK = orange, banana, apple Consumer1 2 3
  • 34. direct/producer.php $channel->exchange_declare( 'test_direct', 'fanout', false, false, false); $messageContent = 'my test message, key=' . $routingKey; $message = new AMQPMessage($messageContent); $channel->basic_publish($message, 'test_direct', $routingKey);
  • 35. direct/consumer.php $q = $channel->queue_declare('', false, false, false, true); $queue_name = $q[0]; $channel->exchange_declare( 'test_direct', 'direct', false, false, false); // Bind for each routing key we want (BINDING KEY) $channel->queue_bind($queue_name, 'test_direct', 'apple'); $channel->queue_bind($queue_name, 'test_direct', 'orange'); $channel->queue_bind($queue_name, 'test_direct', 'banana');
  • 36. What to expect...
  • 37. Exchanges: Topic
  • 38. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer 1 2 Message Routing Keys 1 = red.vegetable 2 = green.vegetable 3 = green.fruit 4 = red.meat 5 = green.grass.long 1 2 3 4 5 BK = green.# Consumer2 3 5 BK = *.grass.* / *.*.long Consumer5
  • 39. Real World Example
  • 40. Fetch message Logging Sequence ApplicationBrowser Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  • 41. Flexibility! ● Temporary Queues ○ e.g. inspect “debug” messages
  • 42. Flexibility! ● Temporary Queues ● Queues to log to DB
  • 43. Flexibility! ● Temporary Queues ● Queues to log to DB ● Queues to email “alert/emergency”
  • 44. Flexibility! ● Temporary Queues ● Queues to log to DB ● Queues to email “alert/emergency” ● Get creative with routing keys ○ RK = app.api.error … BK = #.api.error ○ RK = app.form.debug … BK = #.debug
  • 45. Problem: SPOF
  • 46. Solution 1: Clustering
  • 47. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 5 RabbitMQ Node 6 Load Balance / Floating IP / Low TTL DNS etc.
  • 48. Clustering ● Everything replicated, except queues
  • 49. Clustering ● Everything replicated, except queues ● Types: ○ RAM ○ Disk
  • 50. Clustering ● Everything replicated, except queues ● Types: ○ RAM ○ Disk ● Configuration: ○ CLI (rabbitmqctl) ○ Configuration files
  • 51. Creating a cluster node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}] ...done. node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}] ...done.
  • 52. node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done. Creating a cluster
  • 53. node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done. Managing Nodes ● Stopping/starting nodes ● Removing nodes:
  • 54. Solution 2: HA
  • 55. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  • 56. https://github.com/asgrim/rmq-slides Have a go yourself!
  • 57. Questions?
  • 58. James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim Thanks for watching!

×