Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Adding 1.21 Gigawatts to Applications with RabbitMQ (Bulgaria PHP 2016 - Tutorial)

295 views

Published on

As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ. In this tutorial, I will introduce RabbitMQ as a solution to scalable, interoperable and flexible applications.

This tutorial is perfect for those who would like a deep dive into RabbitMQ with little or no pre-existing knowledge about message queuing systems. Once you’ve finished the tutorial, you will have learnt how to set up basic publish/subscribe message queues, control the flow of messages using various exchanges, and understand various features of RabbitMQ such as RPC, TTL, and DLX.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Adding 1.21 Gigawatts to Applications with RabbitMQ (Bulgaria PHP 2016 - Tutorial)

  1. 1. @asgrim this slide is intentionally left blank
  2. 2. @asgrim Before we begin... Copy folder from USB to your local machine. $ cd /path/to/the/folder/you/copied/from/usb/ $ vagrant box add asgrim/rmq-vm rmq-vm.box $ vagrant up --no-provision Test it in your browser... http://192.168.33.99:15672/
  3. 3. @asgrim Introducing Practical RabbitMQ James Titcumb
  4. 4. James Titcumb www.jamestitcumb.com www.roave.com www.phphants.co.uk www.phpsouthcoast.co.uk @asgrim Who is this guy?
  5. 5. @asgrim Photo: http://spitalfieldslife.com/2011/08/27/a-fox-in-hoxton-2/
  6. 6. @asgrim What is a message?
  7. 7. @asgrim What is message queueing?
  8. 8. @asgrim Separation of Concerns
  9. 9. @asgrim Scaling with Rabbit RabbitMQApplication Background processing
  10. 10. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing
  11. 11. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing
  12. 12. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing
  13. 13. @asgrim Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing Background processing
  14. 14. @asgrim AMQP
  15. 15. @asgrim AMQP Low Level Frame 1 1 67 <frame payload> 0xCE
  16. 16. @asgrim AMQP Frame: Basic.Publish <frame payload> Publish <exchange> <routing key> <flag> 1 1 67 0xCE Basic
  17. 17. @asgrim AMQP Message: Multiple Frames 1 1 67 Basic.Publish 0xCE 2 1 102 Content Header 0xCE 3 1 1024 Body 0xCE 3 1 1024 Body 0xCE 3 1 1024 Body 0xCE 3 1 1024 Body 0xCE
  18. 18. @asgrim AMQP Headers ● content-type
  19. 19. @asgrim AMQP Headers ● content-type ● content-encoding
  20. 20. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id
  21. 21. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to
  22. 22. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to ● expiration
  23. 23. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to ● expiration ● priority
  24. 24. @asgrim AMQP Headers ● content-type ● content-encoding ● message-id ● correlation-id ● reply-to ● expiration ● priority ● headers
  25. 25. @asgrim Real world uses?
  26. 26. @asgrim Who is using it?
  27. 27. @asgrim SOA
  28. 28. @asgrim Why RabbitMQ?
  29. 29. @asgrim (you need Vagrant+VirtualBox already) Installing RabbitMQ
  30. 30. @asgrim vagrant up Copy folder from USB to your local machine. $ cd /path/to/the/folder/you/copied/from/usb/ $ vagrant box add asgrim/rmq-vm rmq-vm.box $ vagrant up --no-provision Test it in your browser... http://192.168.33.99:15672/
  31. 31. @asgrim Library with Composer composer require php-amqplib/php-amqplib
  32. 32. @asgrim The Management Console http://192.168.33.99:15672/
  33. 33. @asgrim Practical
  34. 34. @asgrim https://github.com/asgrim/rmq-tutorial
  35. 35. @asgrim Objective: Basic Queuing Producer Consumer test_queue 1 2 3 4 5
  36. 36. @asgrim Get vs Consume
  37. 37. @asgrim Exchanges: Fanout
  38. 38. @asgrim Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  39. 39. @asgrim A word on Temporary Queues test_exchangeProducer Messages go nowhere
  40. 40. @asgrim Exchanges: Direct
  41. 41. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  42. 42. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  43. 43. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  44. 44. @asgrim Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  45. 45. @asgrim Exchanges: Topic
  46. 46. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  47. 47. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  48. 48. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  49. 49. @asgrim Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  50. 50. @asgrim Message Acknowledgement
  51. 51. @asgrim Another Example
  52. 52. @asgrim
  53. 53. @asgrim Fetch message Logging Sequence ApplicationBrowser Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  54. 54. @asgrim Flexibility!
  55. 55. @asgrim Parallel Processing
  56. 56. @asgrim test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Producer 1 1 2 2 3 3 4 4 5 5 Consumer Consumer Consumer Consumer Consumer Consumer Consumer Parallel Processing
  57. 57. @asgrim RPC
  58. 58. @asgrim RPC example Producer Consumer test_queue 1 reply_to: amq.gen-Xa2 1’
  59. 59. @asgrim TTL
  60. 60. @asgrim TTL - per queue message Producer Consumer test_queue 1 2 3 4 5 } 10s
  61. 61. @asgrim TTL - per message Producer Consumer test_queue 1 2 3 4 5 5s 3s 7s 1s 9s
  62. 62. @asgrim TTL - queue Producer test_queue } 5s (if no consumers)
  63. 63. @asgrim Photo: © Rob Allen (akrabat) https://flic.kr/p/6wp9iz
  64. 64. @asgrim DLX
  65. 65. @asgrim DLX test_exchange amq.KfgPZ3PE Producer 1 dlx_exchange dlx_queue 1
  66. 66. @asgrim Scheduling / Delayed messages
  67. 67. @asgrim Shovel
  68. 68. @asgrim Federated Queues
  69. 69. @asgrim Federated Queues 1 Consumer WAN Rabbit Node #1 Rabbit Node #2 my_queue my_queue (federated)
  70. 70. @asgrim Priority
  71. 71. @asgrim Management HTTP API
  72. 72. @asgrim Infrastructure
  73. 73. @asgrim Problem: SPOF
  74. 74. @asgrim Solution 1: Clustering
  75. 75. @asgrim Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 5 RabbitMQ Node 6 Load Balance / Floating IP / Low TTL DNS etc.
  76. 76. @asgrim Everything Replicates (except queues…)
  77. 77. @asgrim Disk / RAM
  78. 78. @asgrim Configuration...
  79. 79. @asgrim /etc/rabbitmq/rabbitmq.config [ {rabbit, [ {loopback_users, []}, {vm_memory_high_watermark, 0.8} ]} ].
  80. 80. @asgrim /etc/rabbitmq/rabbitmq.config [{{{[{{[{{}}{][[[{[{{}[[}{[[{}[][}{}}}{}}{{,},]{ [[{rabbit, [{{}[[}{,,{}[][}{[][][{}{{{{}}}}[[}{{ {{}}{loopback_users, []},[][][]{}{}{}<}{[}[][][} [{{[{vm_memory_high_watermark, 0.8}]]{}{[[[]]{}] {{]}[{[{{}[[}{]]{}[][,{}[][}{[][][{}.[]}{]][][]} ]...{}[][,]{.}[][}{}[[[{}{][]}{}{}[}{}{}{]{}{}}[
  81. 81. @asgrim node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. Creating a cluster
  82. 82. @asgrim node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done. Creating a cluster
  83. 83. @asgrim Starting/Stopping Nodes
  84. 84. @asgrim node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done. Removing Nodes
  85. 85. @asgrim Solution 2: HA
  86. 86. @asgrim HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  87. 87. @asgrim BunnyPHP
  88. 88. @asgrim composer require bunny/bunny
  89. 89. @asgrim Challenge!
  90. 90. Any questions? https://joind.in/talk/be4e5 James Titcumb @asgrim

×