Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014)

1,808 views

Published on

RabbitMQ is a message broker – an application that allows communication between applications by way of a message queuing system. In this talk, we’ll set up a RabbitMQ instance, take an intermediate-level look into the technical features it provides and also how you can apply RabbitMQ in your applications to scale them efficiently.

Published in: Technology
  • Be the first to comment

Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014)

  1. 1. Practical Message Queueing Using RabbitMQ James Titcumb Nomad PHP EU December 2014
  2. 2. James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim Who is this guy?
  3. 3. What is message queueing?
  4. 4. Separation of Concerns
  5. 5. Scaling with Rabbit RabbitMQApplication Background processing
  6. 6. Scaling with Rabbit RabbitMQApplication Background processing Background processing
  7. 7. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing
  8. 8. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing
  9. 9. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing Background processing
  10. 10. Real world uses?
  11. 11. Why RabbitMQ?
  12. 12. (on precise64, other OSs may vary) Installing RabbitMQ
  13. 13. ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart Using Apt
  14. 14. http://localhost:15672/
  15. 15. http://localhost:15672/
  16. 16. http://localhost:15672/
  17. 17. http://localhost:15672/
  18. 18. http://localhost:15672/
  19. 19. Basic Message Queuing
  20. 20. Objective: Basic Queuing Producer Consumer test_queue 1 2 3 4 5
  21. 21. composer.json 1 { 2 "require": { 3 "videlalvaro/php-amqplib": "2.*" 4 } 5 } then composer install
  22. 22. Please wait, connecting... 1 use PhpAmqpLibConnectionAMQPConnection; 2 3 $connection = new AMQPConnection( 4 'localhost', 5 5672, 6 'guest', 7 'guest', 8 '/' 9 ); 10 $channel = $connection->channel();
  23. 23. basic/producer.php 1 use PhpAmqpLibMessageAMQPMessage; 2 3 $channel->queue_declare( 4 'test_queue', 5 false, 6 true, 7 false, 8 false); 9 10 $message = new AMQPMessage('my test message'); 11 $channel->basic_publish($message, '', 'test_queue');
  24. 24. basic/consumer.php 1 $channel->basic_consume( 2 'test_queue', // Queue to consume 3 '', // Consumer identifier 4 false, 5 true, // No-ack means messages are "auto acknowledged" 6 false, // Exclusive - no other consumers can use the queue 7 false, 8 function(AMQPMessage $message) { 9 echo $message->body . "n"; 10 } 11 ); 12 13 while (count($channel->callbacks)) { 14 $channel->wait(); 15 }
  25. 25. What to expect...
  26. 26. Exchanges: Fanout
  27. 27. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  28. 28. 1 use PhpAmqpLibMessageAMQPMessage; 2 3 $channel->exchange_declare( 4 'test_exchange', 5 'fanout', 6 false, 7 false, 8 false); 9 10 $message = new AMQPMessage('my test message #' . $id); 11 $channel->basic_publish($message, 'test_exchange'); fanout/producer.php
  29. 29. fanout/consumer.php 1 $q = $channel->queue_declare( 2 '', // Lets RabbitMQ pick a name for queue 3 false, false, false, 4 true // Delete this queue 5 ); 6 $queue_name = $q[0]; 7 8 $channel->exchange_declare( 9 'test_exchange', 'fanout', false, false, false); 10 $channel->queue_bind($queue_name, 'test_exchange');
  30. 30. What to expect...
  31. 31. A word on Temporary Queues test_exchangeProducer Messages go nowhere
  32. 32. Exchanges: Direct
  33. 33. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  34. 34. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  35. 35. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  36. 36. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  37. 37. direct/producer.php 1 $channel->exchange_declare( 2 'test_direct', 'fanout', false, false, false); 3 4 $messageContent = 'my test message, key=' . $routingKey; 5 $message = new AMQPMessage($messageContent); 6 $channel->basic_publish( 7 $message, 8 'test_direct', 9 $routingKey 10 );
  38. 38. direct/consumer.php 1 $q = $channel->queue_declare('', false, false, false, true); 2 $queue_name = $q[0]; 3 $channel->exchange_declare( 4 'test_direct', 'direct', false, false, false); 5 6 // Bind for each routing key we want (BINDING KEY) 7 $channel->queue_bind($queue_name, 'test_direct', 'apple'); 8 $channel->queue_bind($queue_name, 'test_direct', 'orange'); 9 $channel->queue_bind($queue_name, 'test_direct', 'banana');
  39. 39. What to expect...
  40. 40. Exchanges: Topic
  41. 41. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  42. 42. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  43. 43. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  44. 44. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  45. 45. Real World Example
  46. 46. Fetch message Logging Sequence ApplicationBrowser Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  47. 47. Flexibility!
  48. 48. Parallel Processing
  49. 49. Message Acknowledgement
  50. 50. RPC
  51. 51. TTL
  52. 52. DLX
  53. 53. http://tryrabbitmq.com/
  54. 54. Infrastructure
  55. 55. Problem: SPOF
  56. 56. Solution 1: Clustering
  57. 57. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 5 RabbitMQ Node 6 Load Balance / Floating IP / Low TTL DNS etc.
  58. 58. Everything Replicates (except queues…)
  59. 59. RAM / Disk
  60. 60. Configuration...
  61. 61. /etc/rabbitmq/rabbitmq.config [ {rabbit, [ {loopback_users, []}, {vm_memory_high_watermark, 0.8} ]} ].
  62. 62. Creating a cluster node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}] ...done. node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}] ...done.
  63. 63. node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done. Creating a cluster
  64. 64. Starting/Stopping Nodes
  65. 65. node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done. Removing Nodes
  66. 66. Solution 2: HA
  67. 67. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  68. 68. https://github.com/asgrim/rmq-slides Have a go yourself!
  69. 69. Questions?
  70. 70. https://joind.in/13093 James Titcumb @asgrim Thanks for watching!

×