Successfully reported this slideshow.
Your SlideShare is downloading. ×

Adding 1.21 Gigawatts to Applications with RabbitMQ (DPC 2015)

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 83 Ad

Adding 1.21 Gigawatts to Applications with RabbitMQ (DPC 2015)

Download to read offline

As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ.

In this talk, we will take a look at the techniques you can use to vastly enhance inter-application communication, learn about the core concepts of RabbitMQ, cover how you can scale different parts of your application separately, and modernise your development using a message-oriented architecture.

As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ.

In this talk, we will take a look at the techniques you can use to vastly enhance inter-application communication, learn about the core concepts of RabbitMQ, cover how you can scale different parts of your application separately, and modernise your development using a message-oriented architecture.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Viewers also liked (20)

Advertisement

Similar to Adding 1.21 Gigawatts to Applications with RabbitMQ (DPC 2015) (20)

More from James Titcumb (20)

Advertisement

Recently uploaded (20)

Adding 1.21 Gigawatts to Applications with RabbitMQ (DPC 2015)

  1. 1. Adding 1.21 Gigawatts to Applications with RabbitMQ James Titcumb Dutch PHP Conference 2015
  2. 2. James Titcumb www.jamestitcumb.com www.roave.com www.phphants.co.uk www.phpsouthcoast.co.uk @asgrim Who is this guy?
  3. 3. What is a message?
  4. 4. What is message queueing?
  5. 5. Separation of Concerns
  6. 6. Scaling with Rabbit RabbitMQApplication Background processing
  7. 7. Scaling with Rabbit RabbitMQApplication Background processing Background processing
  8. 8. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing
  9. 9. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing
  10. 10. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing Background processing
  11. 11. Real world uses?
  12. 12. SOA
  13. 13. Why RabbitMQ?
  14. 14. https://github.com/asgrim/rmq-slides Follow along here...
  15. 15. (on trusty64 Vagrant, other OSs may vary) Installing RabbitMQ
  16. 16. ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart Using Apt
  17. 17. http://localhost:15672/
  18. 18. http://localhost:15672/
  19. 19. http://localhost:15672/
  20. 20. http://localhost:15672/
  21. 21. http://localhost:15672/
  22. 22. Basic Message Queue
  23. 23. Objective: Basic Queuing Producer Consumer test_queue 1 2 3 4 5
  24. 24. composer.json composer require videlalvaro/php-amqplib Approved
  25. 25. Please wait, connecting... 1 use PhpAmqpLibConnectionAMQPConnection; 2 3 $connection = new AMQPConnection( 4 'localhost', 5672, 'guest', 'guest', '/'); 5 6 $channel = $connection->channel();
  26. 26. basic/producer.php 1 use PhpAmqpLibMessageAMQPMessage; 2 3 $channel->queue_declare( 4 'test_queue', false, true, false, false); 5 6 $message = new AMQPMessage('my test message'); 7 $channel->basic_publish($message, '', 'test_queue');
  27. 27. basic/consumer.php 1 $channel->basic_consume( 2 'test_queue', '', false, true, false, false, 3 function(AMQPMessage $message) { 4 echo $message->body . "n"; 5 }); 6 7 while (count($channel->callbacks)) 8 $channel->wait();
  28. 28. What to expect...
  29. 29. Exchanges: Fanout
  30. 30. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  31. 31. 1 use PhpAmqpLibMessageAMQPMessage; 2 3 $channel->exchange_declare( 4 'test_exchange', 'fanout', false, false, false); 5 6 $message = new AMQPMessage('test msg #' . $id); 7 $channel->basic_publish($message, 'test_exchange'); fanout/producer.php
  32. 32. fanout/consumer.php 1 $q = $channel->queue_declare( 2 '', false, false, false, true); 3 $queue_name = $q[0]; 4 5 $channel->exchange_declare( 6 'test_exchange', 'fanout', false, false, false); 7 $channel->queue_bind($queue_name, 'test_exchange');
  33. 33. What to expect...
  34. 34. A word on Temporary Queues test_exchangeProducer Messages go nowhere
  35. 35. Exchanges: Direct
  36. 36. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  37. 37. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  38. 38. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  39. 39. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  40. 40. direct/producer.php 1 $channel->exchange_declare( 2 'test_direct', 'fanout', false, false, false); 3 4 $messageContent = 'test msg, key=' . $routingKey; 5 $message = new AMQPMessage($messageContent); 6 $channel->basic_publish( 7 $message, 'test_direct', $routingKey);
  41. 41. direct/consumer.php 1 $q = $channel->queue_declare('', false, false, false, true); 2 $queue_name = $q[0]; 3 $channel->exchange_declare( 4 'test_direct', 'direct', false, false, false); 5 6 // Bind for each routing key we want (BINDING KEY) 7 $channel->queue_bind($queue_name, 'test_direct', 'apple'); 8 $channel->queue_bind($queue_name, 'test_direct', 'orange'); 9 $channel->queue_bind($queue_name, 'test_direct', 'banana');
  42. 42. What to expect...
  43. 43. Exchanges: Topic
  44. 44. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  45. 45. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  46. 46. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  47. 47. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  48. 48. Real World Example
  49. 49. Fetch message Logging Sequence ApplicationBrowser Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  50. 50. Flexibility!
  51. 51. Parallel Processing
  52. 52. test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Producer 1 1 2 2 3 3 4 4 5 5 Consumer Consumer Consumer Consumer Consumer Consumer Consumer Parallel Processing
  53. 53. Message Acknowledgement
  54. 54. 1 $channel->basic_consume( 2 'test_queue', '', false, false, false, false, 3 function(AMQPMessage $message) { 4 echo $message->body . "n"; 5 throw new Exception('oh noes!!!'); 6 7 $channel = $message->delivery_info['channel']; 8 $tag = $message->delivery_info['delivery_tag']; 9 $channel->basic_ack($tag); 10 }); Message Acknowledgement
  55. 55. RPC
  56. 56. RPC example Producer Consumer test_queue 1 reply_to: amq.gen-Xa2 1’
  57. 57. TTL
  58. 58. TTL - per queue message Producer Consumer test_queue 1 2 3 4 5 } 10s
  59. 59. TTL - per message Producer Consumer test_queue 1 2 3 4 5 5s 3s 7s 1s 9s
  60. 60. TTL - queue Producer test_queue } 5s (if no consumers)
  61. 61. DLX
  62. 62. DLX test_exchange amq.KfgPZ3PE Producer 1 dlx_exchange dlx_queue 1
  63. 63. Scheduling / Delayed messages
  64. 64. Priority
  65. 65. http://tryrabbitmq.com/
  66. 66. Infrastructure
  67. 67. Problem: SPOF
  68. 68. Solution 1: Clustering
  69. 69. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 5 RabbitMQ Node 6 Load Balance / Floating IP / Low TTL DNS etc.
  70. 70. Everything Replicates (except queues…)
  71. 71. Disk / RAM
  72. 72. Configuration...
  73. 73. /etc/rabbitmq/rabbitmq.config [ {rabbit, [ {loopback_users, []}, {vm_memory_high_watermark, 0.8} ]} ].
  74. 74. /etc/rabbitmq/rabbitmq.config [{{{[{{[{{}}{][[[{[{{}[[}{[[{}[][}{}}}{}}{{,},]{ [[{rabbit, [{{}[[}{,,{}[][}{[][][{}{{{{}}}}[[}{{ {{}}{loopback_users, []},[][][]{}{}{}<}{[}[][][} [{{[{vm_memory_high_watermark, 0.8}]]{}{[[[]]{}] {{]}[{[{{}[[}{]]{}[][,{}[][}{[][][{}.[]}{]][][]} ]...{}[][,]{.}[][}{}[[[{}{][]}{}{}[}{}{}{]{}{}}[
  75. 75. node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. Creating a cluster
  76. 76. node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done. Creating a cluster
  77. 77. Starting/Stopping Nodes
  78. 78. node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done. Removing Nodes
  79. 79. Solution 2: HA
  80. 80. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  81. 81. https://github.com/asgrim/rmq-slides That link again...
  82. 82. Any questions? :) https://joind.in/14253 James Titcumb @asgrim

×