Scaling Symfony2 apps with RabbitMQ - Symfony UK Meetup

14,761 views

Published on

Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper

Video: https://www.youtube.com/watch?v=cha92Og9M5A

Published in: Technology
3 Comments
57 Likes
Statistics
Notes
No Downloads
Views
Total views
14,761
On SlideShare
0
From Embeds
0
Number of Embeds
1,669
Actions
Shares
0
Downloads
189
Comments
3
Likes
57
Embeds 0
No embeds

No notes for slide

Scaling Symfony2 apps with RabbitMQ - Symfony UK Meetup

  1. 1. Scaling Symfony2 apps with RabbitMQ
  2. 2. Kacper Gunia @cakper Software Engineer @SensioLabsUK Symfony Certified Developer PHPers Silesia @PHPersPL
  3. 3. Agenda ❖What is RabbitMQ! ❖How to communicate with it from PHP! ❖How to integrate it with Symfony! ❖Diving into details! ❖Demo
  4. 4. Why would I need Messaging?
  5. 5. Use Cases ❖ Offloading / Background jobs! ❖ Integration ! ❖ Scaling! ❖ Queueing! ❖ Scheduling! ❖ Events
  6. 6. Message-oriented Middleware “(…) allows distributed applications ! to communicate and exchange data ! by sending and receiving messages” http://docs.oracle.com/cd/E19316-01/820-6424/aeraq/index.html
  7. 7. What is inside a Rabbit?
  8. 8. http://madamtruffle.deviantart.com/art/What-­‐is-­‐inside-­‐a-­‐rabbit-­‐81036248
  9. 9. What is inside the RabbitMQ?
  10. 10. Message Broker
  11. 11. AMQP
  12. 12. Erlang
  13. 13. Interoperability ❖ PHP! ❖ Clojure! ❖ Erlang! ❖ Java! ❖ Perl! ❖ Python! ❖ Ruby ❖ C#! ❖ JavaScript! ❖ C/C++! ❖ Go! ❖ Lisp! ❖ Haskell! ❖ …
  14. 14. How does it work?
  15. 15. Broker Bindings Producer Exchange Queue Queue Consumer Consumer Consumer Consumer
  16. 16. Producer Producer
  17. 17. Exchange Exchange
  18. 18. Direct Exchange Exchange Queue
  19. 19. Fanout Exchange Exchange Queue Queue
  20. 20. Topic Exchange #.error #.warning, log.* *.mobile.* log.error log.sql.warning log.mobile.error
  21. 21. “.” - word separator
  22. 22. “#” - prefix/suffix wildcard
  23. 23. “*” - word wildcard
  24. 24. Bindings Bindings Exchange Queue Queue
  25. 25. Queue Queue
  26. 26. Consumer Consumer
  27. 27. How to feed Rabbit from PHP?
  28. 28. PHP libraries and tools ❖ php-amqplib! ❖ PECL AMQP ! ❖ amqphp! ❖ VorpalBunny! ❖ Thumper! ❖ CAMQP
  29. 29. Declare queue $connection = new AMQPConnection ('localhost', 5672, 'guest', 'guest'); $channel = $connection-­‐>channel(); ! $channel-­‐>queue_declare ('hello', false, false, false, false); ! /** Your stuff */ ! $channel-­‐>close(); $connection-­‐>close();
  30. 30. Producer $msg = new AMQPMessage('Hello World!'); $channel-­‐>basic_publish($msg, '', 'hello'); ! echo " [x] Sent 'Hello World!'n";
  31. 31. Consumer $callback = function($msg) { echo ' [x] Received ', $msg-­‐>body, "n"; }; ! $channel-­‐>basic_consume ('hello', '', false, true, false, false, $callback); ! while(count($channel-­‐>callbacks)) { $channel-­‐>wait(); }
  32. 32. RabbitMQ & Symfony https://secure.flickr.com/photos/8725928@N02/9657136424
  33. 33. Step 1
  34. 34. Install RabbitMQ Bundle composer require “oldsound/rabbitmq-­‐bundle 1.5.*”
  35. 35. Step 2
  36. 36. Enable bundle in Kernel public function registerBundles() { $bundles =[ (…) new OldSoundRabbitMqBundleOldSoundRabbitMqBundle() ]; }
  37. 37. Step 3
  38. 38. Say thanks to @old_sound :)
  39. 39. Step 4
  40. 40. Get rid off manual configuration old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: false
  41. 41. Get rid off manual configuration producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
  42. 42. Get rid off manual configuration producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
  43. 43. Step 5
  44. 44. Produce some data public function indexAction($name) { $this -­‐>get('old_sound_rabbit_mq.hello_world_producer') -­‐>publish($name); }
  45. 45. Step 6
  46. 46. Implement consumer class HelloWorldConsumer implements ConsumerInterface { public function execute(AMQPMessage $msg) { echo "Hello $msg-­‐>body!".PHP_EOL; } }
  47. 47. Step 7
  48. 48. Configure Service Container services: hello_world_service: class: CakperHelloWorldConsumer
  49. 49. Step 8
  50. 50. Run the Consumer ./app/console rabbitmq:consumer hello_world
  51. 51. Nailed it!
  52. 52. Diving into https://secure.flickr.com/photos/toms/159393358
  53. 53. Producers
  54. 54. Custom producer class producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} class: CakperHelloProducer
  55. 55. Custom producer class class HelloProducer extends Producer { public function publish($msgBody, …) { $msgBody = serialize($msgBody); ! parent::publish($msgBody, …); } }
  56. 56. Set content type function __construct() { $this-­‐>setContentType('application/json'); } ! public function publish($msgBody, …) { parent::publish(json_encode($msgBody), …); }
  57. 57. Consumers
  58. 58. Re-queue message public function execute(AMQPMessage $msg) { if ('cakper' === $msg-­‐>body) { return false; } ! echo "Hello $msg-­‐>body!".PHP_EOL; }
  59. 59. Idle timeouts consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service idle_timeout: 180
  60. 60. Limit number of messages ./app/console rabbitmq:consumer hello_world -­‐m 10
  61. 61. Quality of Service qos_options: prefetch_size: 0 prefetch_count: 0..65535 global: false/true
  62. 62. Exchanges
  63. 63. Exchange options exchange_options: name: ~ type: direct/fanout/topic durable: true/false
  64. 64. Queues
  65. 65. Queue options queue_options: name: ~ durable: true/false arguments: 'x-­‐message-­‐ttl': ['I', 20000] routing_keys: -­‐ 'logs.sql.#' -­‐ '*.error'
  66. 66. Purge queue ./app/console rabbitmq:purge -­‐-­‐no-­‐confirmation hello
  67. 67. Connection
  68. 68. Make it lazy! old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: true
  69. 69. Setup
  70. 70. Setup Fabric ./app/console rabbitmq:setup-­‐fabric ! producers: upload_picture: auto_setup_fabric: false consumers: upload_picture: auto_setup_fabric: false
  71. 71. Multiple consumers
  72. 72. Multiple consumers multiple_consumers: hello: connection: default exchange_options: {name: 'hello', type: direct} queues: hello-­‐vip: name: hello_vip callback: hello_vip_world_service routing_keys: -­‐ vip hello-­‐regular: name: hello_regular callback: hello_regular_world_service
  73. 73. Anonymous Consumers
  74. 74. Anonymous Consumer producers: hello: connection: default exchange_options: {name: 'hello', type: topic}
  75. 75. Anonymous Consumer anon_consumers: hello: connection: default exchange_options: {name: 'hello', type: topic} callback: hello_world_service
  76. 76. Anonymous Consumer ./app/console_dev rabbitmq:anon-­‐consumer -­‐r '#.vip' hello
  77. 77. Management console
  78. 78. Demo time
  79. 79. Summary ❖ RabbitMQ is fast! ❖ and reliable! ❖ also language agnostic! ❖ and easy to install and use! ❖ gives you flexibility! ❖ supports high-availability, clustering
  80. 80. Kacper Gunia Software Engineer Thanks! Symfony Certified Developer PHPers Silesia

×