Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

858 views

Published on

As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ. In this talk, we will take a look at the techniques you can use to vastly enhance inter-application communication, learn about the core concepts of RabbitMQ, cover how you can scale different parts of your application separately, and modernise your development using a message-oriented architecture

Published in: Technology
  • Be the first to comment

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

  1. 1. Adding 1.21 Gigawatts to Applications with RabbitMQ James Titcumb PHPNW December 2014
  2. 2. Who is this guy? James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim
  3. 3. What is message queueing?
  4. 4. Separation of Concerns
  5. 5. Scaling with Rabbit Application RabbitMQ Background processing
  6. 6. Scaling with Rabbit Application RabbitMQ Background processing Background processing
  7. 7. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing
  8. 8. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing Background processing
  9. 9. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing Background processing Background processing
  10. 10. Real world uses?
  11. 11. Why RabbitMQ?
  12. 12. Installing RabbitMQ (on precise64, other OSs may vary)
  13. 13. Using Apt ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart
  14. 14. http://localhost:15672/
  15. 15. Basic Message Queuing
  16. 16. Objective: Basic Queuing test_queue 1 2 3 4 5 Producer Consumer
  17. 17. composer.json { "require": { "videlalvaro/php-amqplib": "2.*" } } then composer install
  18. 18. Please wait, connecting... use PhpAmqpLibConnectionAMQPConnection; $connection = new AMQPConnection( 'localhost', 5672, 'guest', 'guest', '/' ); $channel = $connection->channel();
  19. 19. basic/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->queue_declare( 'test_queue', false, true, false, false); $message = new AMQPMessage('my test message'); $channel->basic_publish($message, '', 'test_queue');
  20. 20. basic/consumer.php $channel->basic_consume( 'test_queue', // Queue to consume '', // Consumer identifier false, true, // No-ack means messages are "auto acknowledged" false, // Exclusive - no other consumers can use the queue false, function(AMQPMessage $message) { echo $message->body . "n"; } ); while (count($channel->callbacks)) { $channel->wait(); }
  21. 21. What to expect...
  22. 22. Exchanges: Fanout
  23. 23. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  24. 24. fanout/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $message = new AMQPMessage('my test message #' . $id); $channel->basic_publish($message, 'test_exchange');
  25. 25. fanout/consumer.php $q = $channel->queue_declare( '', // Lets RabbitMQ pick a name for queue false, false, false, true // Delete this queue ); $queue_name = $q[0]; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $channel->queue_bind($queue_name, 'test_exchange');
  26. 26. What to expect...
  27. 27. A word on Temporary Queues Producer test_exchange Messages go nowhere
  28. 28. Exchanges: Direct
  29. 29. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  30. 30. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  31. 31. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  32. 32. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  33. 33. direct/producer.php $channel->exchange_declare( 'test_direct', 'fanout', false, false, false); $messageContent = 'my test message, key=' . $routingKey; $message = new AMQPMessage($messageContent); $channel->basic_publish($message, 'test_direct', $routingKey);
  34. 34. direct/consumer.php $q = $channel->queue_declare('', false, false, false, true); $queue_name = $q[0]; $channel->exchange_declare( 'test_direct', 'direct', false, false, false); // Bind for each routing key we want (BINDING KEY) $channel->queue_bind($queue_name, 'test_direct', 'apple'); $channel->queue_bind($queue_name, 'test_direct', 'orange'); $channel->queue_bind($queue_name, 'test_direct', 'banana');
  35. 35. What to expect...
  36. 36. Exchanges: Topic
  37. 37. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  38. 38. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  39. 39. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  40. 40. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  41. 41. Real World Example
  42. 42. Fetch message Logging Sequence Browser Application Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  43. 43. Flexibility!
  44. 44. RPC
  45. 45. TTL
  46. 46. DLX
  47. 47. Infrastructure
  48. 48. Problem: SPOF
  49. 49. Solution 1: Clustering
  50. 50. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 6 RabbitMQ Node 5 Load Balance / Floating IP / Low TTL DNS etc.
  51. 51. Everything Replicates (except queues…)
  52. 52. RAM / Disk
  53. 53. Configuration...
  54. 54. Creating a cluster node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}] ...done. node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}] ...done.
  55. 55. Creating a cluster node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done.
  56. 56. Starting/Stopping Nodes
  57. 57. Removing Nodes node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done.
  58. 58. Solution 2: HA
  59. 59. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  60. 60. Have a go yourself! https://github.com/asgrim/rmq-slides
  61. 61. Questions?
  62. 62. Thanks for watching! James Titcumb @asgrim

×