Scaling Symfony2 apps with RabbitMQ - Symfony UK Meetup

  • 3,914 views
Uploaded on

Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper …

Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper

Video: https://www.youtube.com/watch?v=cha92Og9M5A

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
3,914
On Slideshare
0
From Embeds
0
Number of Embeds
10

Actions

Shares
Downloads
75
Comments
1
Likes
22

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Scaling Symfony2 apps with RabbitMQ
  • 2. Kacper Gunia @cakper Software Engineer @SensioLabsUK Symfony Certified Developer PHPers Silesia @PHPersPL
  • 3. Agenda ❖What is RabbitMQ! ❖How to communicate with it from PHP! ❖How to integrate it with Symfony! ❖Diving into details! ❖Demo
  • 4. Why would I need Messaging?
  • 5. Use Cases ❖ Offloading / Background jobs! ❖ Integration ! ❖ Scaling! ❖ Queueing! ❖ Scheduling! ❖ Events
  • 6. Message-oriented Middleware “(…) allows distributed applications ! to communicate and exchange data ! by sending and receiving messages” http://docs.oracle.com/cd/E19316-01/820-6424/aeraq/index.html
  • 7. What is inside a Rabbit?
  • 8. http://madamtruffle.deviantart.com/art/What-­‐is-­‐inside-­‐a-­‐rabbit-­‐81036248
  • 9. What is inside the RabbitMQ?
  • 10. Message Broker
  • 11. AMQP
  • 12. Erlang
  • 13. Interoperability ❖ PHP! ❖ Clojure! ❖ Erlang! ❖ Java! ❖ Perl! ❖ Python! ❖ Ruby ❖ C#! ❖ JavaScript! ❖ C/C++! ❖ Go! ❖ Lisp! ❖ Haskell! ❖ …
  • 14. How does it work?
  • 15. Broker Bindings Producer Exchange Queue Queue Consumer Consumer Consumer Consumer
  • 16. Producer Producer
  • 17. Exchange Exchange
  • 18. Direct Exchange Exchange Queue
  • 19. Fanout Exchange Exchange Queue Queue
  • 20. Topic Exchange #.error #.warning, log.* *.mobile.* log.error log.sql.warning log.mobile.error
  • 21. “.” - word separator
  • 22. “#” - prefix/suffix wildcard
  • 23. “*” - word wildcard
  • 24. Bindings Bindings Exchange Queue Queue
  • 25. Queue Queue
  • 26. Consumer Consumer
  • 27. How to feed Rabbit from PHP?
  • 28. PHP libraries and tools ❖ php-amqplib! ❖ PECL AMQP ! ❖ amqphp! ❖ VorpalBunny! ❖ Thumper! ❖ CAMQP
  • 29. Declare queue $connection = new AMQPConnection ('localhost', 5672, 'guest', 'guest'); $channel = $connection-­‐>channel(); ! $channel-­‐>queue_declare ('hello', false, false, false, false); ! /** Your stuff */ ! $channel-­‐>close(); $connection-­‐>close();
  • 30. Producer $msg = new AMQPMessage('Hello World!'); $channel-­‐>basic_publish($msg, '', 'hello'); ! echo " [x] Sent 'Hello World!'n";
  • 31. Consumer $callback = function($msg) { echo ' [x] Received ', $msg-­‐>body, "n"; }; ! $channel-­‐>basic_consume ('hello', '', false, true, false, false, $callback); ! while(count($channel-­‐>callbacks)) { $channel-­‐>wait(); }
  • 32. RabbitMQ & Symfony https://secure.flickr.com/photos/8725928@N02/9657136424
  • 33. Step 1
  • 34. Install RabbitMQ Bundle composer require “oldsound/rabbitmq-­‐bundle 1.5.*”
  • 35. Step 2
  • 36. Enable bundle in Kernel public function registerBundles() { $bundles =[ (…) new OldSoundRabbitMqBundleOldSoundRabbitMqBundle() ]; }
  • 37. Step 3
  • 38. Say thanks to @old_sound :)
  • 39. Step 4
  • 40. Get rid off manual configuration old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: false
  • 41. Get rid off manual configuration producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
  • 42. Get rid off manual configuration producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
  • 43. Step 5
  • 44. Produce some data public function indexAction($name) { $this -­‐>get('old_sound_rabbit_mq.hello_world_producer') -­‐>publish($name); }
  • 45. Step 6
  • 46. Implement consumer class HelloWorldConsumer implements ConsumerInterface { public function execute(AMQPMessage $msg) { echo "Hello $msg-­‐>body!".PHP_EOL; } }
  • 47. Step 7
  • 48. Configure Service Container services: hello_world_service: class: CakperHelloWorldConsumer
  • 49. Step 8
  • 50. Run the Consumer ./app/console rabbitmq:consumer hello_world
  • 51. Nailed it!
  • 52. Diving into https://secure.flickr.com/photos/toms/159393358
  • 53. Producers
  • 54. Custom producer class producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} class: CakperHelloProducer
  • 55. Custom producer class class HelloProducer extends Producer { public function publish($msgBody, …) { $msgBody = serialize($msgBody); ! parent::publish($msgBody, …); } }
  • 56. Set content type function __construct() { $this-­‐>setContentType('application/json'); } ! public function publish($msgBody, …) { parent::publish(json_encode($msgBody), …); }
  • 57. Consumers
  • 58. Re-queue message public function execute(AMQPMessage $msg) { if ('cakper' === $msg-­‐>body) { return false; } ! echo "Hello $msg-­‐>body!".PHP_EOL; }
  • 59. Idle timeouts consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service idle_timeout: 180
  • 60. Limit number of messages ./app/console rabbitmq:consumer hello_world -­‐m 10
  • 61. Quality of Service qos_options: prefetch_size: 0 prefetch_count: 0..65535 global: false/true
  • 62. Exchanges
  • 63. Exchange options exchange_options: name: ~ type: direct/fanout/topic durable: true/false
  • 64. Queues
  • 65. Queue options queue_options: name: ~ durable: true/false arguments: 'x-­‐message-­‐ttl': ['I', 20000] routing_keys: -­‐ 'logs.sql.#' -­‐ '*.error'
  • 66. Purge queue ./app/console rabbitmq:purge -­‐-­‐no-­‐confirmation hello
  • 67. Connection
  • 68. Make it lazy! old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: true
  • 69. Setup
  • 70. Setup Fabric ./app/console rabbitmq:setup-­‐fabric ! producers: upload_picture: auto_setup_fabric: false consumers: upload_picture: auto_setup_fabric: false
  • 71. Multiple consumers
  • 72. Multiple consumers multiple_consumers: hello: connection: default exchange_options: {name: 'hello', type: direct} queues: hello-­‐vip: name: hello_vip callback: hello_vip_world_service routing_keys: -­‐ vip hello-­‐regular: name: hello_regular callback: hello_regular_world_service
  • 73. Anonymous Consumers
  • 74. Anonymous Consumer producers: hello: connection: default exchange_options: {name: 'hello', type: topic}
  • 75. Anonymous Consumer anon_consumers: hello: connection: default exchange_options: {name: 'hello', type: topic} callback: hello_world_service
  • 76. Anonymous Consumer ./app/console_dev rabbitmq:anon-­‐consumer -­‐r '#.vip' hello
  • 77. Management console
  • 78. Demo time
  • 79. Summary ❖ RabbitMQ is fast! ❖ and reliable! ❖ also language agnostic! ❖ and easy to install and use! ❖ gives you flexibility! ❖ supports high-availability, clustering
  • 80. Kacper Gunia Software Engineer Thanks! Symfony Certified Developer PHPers Silesia