Scaling Symfony2 apps 
with RabbitMQ
Kacper Gunia @cakper 
Software Engineer @SensioLabsUK 
Symfony Certified Developer 
PHPers Silesia @PHPersPL
Agenda 
❖What is RabbitMQ! 
❖How to communicate with it from PHP! 
❖How to integrate it with Symfony! 
❖Diving into detail...
Why would I need Messaging?
Use Cases 
❖ Offloading / Background jobs! 
❖ Integration ! 
❖ Scaling! 
❖ Queueing! 
❖ Scheduling! 
❖ Events
Message-oriented Middleware 
“(…) allows distributed applications ! 
to communicate and exchange data ! 
by sending and re...
What is inside a Rabbit?
http://madamtruffle.deviantart.com/art/What-­‐is-­‐inside-­‐a-­‐rabbit-­‐81036248
What is inside the RabbitMQ?
Message Broker
AMQP
Erlang
Interoperability 
❖ PHP! 
❖ Clojure! 
❖ Erlang! 
❖ Java! 
❖ Perl! 
❖ Python! 
❖ Ruby 
❖ C#! 
❖ JavaScript! 
❖ C/C++! 
❖ Go...
How does it work?
Broker 
Bindings 
Producer Exchange 
Queue 
Queue 
Consumer 
Consumer 
Consumer 
Consumer
Producer 
Producer
Exchange 
Exchange
Direct Exchange 
Exchange Queue
Fanout Exchange 
Exchange 
Queue 
Queue
Topic Exchange 
#.error 
#.warning, log.* 
*.mobile.* 
log.error 
log.sql.warning 
log.mobile.error
“.” - word separator
“#” - prefix/suffix wildcard
“*” - word wildcard
Bindings 
Bindings 
Exchange 
Queue 
Queue
Queue 
Queue
Consumer 
Consumer
How to feed Rabbit from PHP?
PHP libraries and tools 
❖ php-amqplib! 
❖ PECL AMQP ! 
❖ amqphp! 
❖ VorpalBunny! 
❖ Thumper! 
❖ CAMQP
Declare queue 
$connection 
= 
new 
AMQPConnection 
('localhost', 
5672, 
'guest', 
'guest'); 
$channel 
= 
$connection-­‐...
Producer 
$msg 
= 
new 
AMQPMessage('Hello 
World!'); 
$channel-­‐>basic_publish($msg, 
'', 
'hello'); 
! 
echo 
" 
[x] 
S...
Consumer 
$callback 
= 
function($msg) 
{ 
echo 
' 
[x] 
Received 
', 
$msg-­‐>body, 
"n"; 
}; 
! 
$channel-­‐>basic_consu...
RabbitMQ & Symfony 
https://secure.flickr.com/photos/8725928@N02/9657136424
Step 1
Install RabbitMQ Bundle 
composer 
require 
“oldsound/rabbitmq-­‐bundle 
1.5.*”
Step 2
Enable bundle in Kernel 
public 
function 
registerBundles() 
{ 
$bundles 
=[ 
(…) 
new 
OldSoundRabbitMqBundleOldSoundRab...
Step 3
Say thanks to 
@old_sound :)
Step 4
Get rid off manual configuration 
old_sound_rabbit_mq: 
connections: 
default: 
host: 
'localhost' 
port: 
5672 
user: 
'g...
Get rid off manual configuration 
producers: 
hello_world: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
typ...
Get rid off manual configuration 
producers: 
hello_world: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
typ...
Step 5
Produce some data 
public 
function 
indexAction($name) 
{ 
$this 
-­‐>get('old_sound_rabbit_mq.hello_world_producer') 
-­...
Step 6
Implement consumer 
class 
HelloWorldConsumer 
implements 
ConsumerInterface 
{ 
public 
function 
execute(AMQPMessage 
$m...
Step 7
Configure Service Container 
services: 
hello_world_service: 
class: 
CakperHelloWorldConsumer
Step 8
Run the Consumer 
./app/console 
rabbitmq:consumer 
hello_world
Nailed it!
Diving into 
https://secure.flickr.com/photos/toms/159393358
Producers
Custom producer class 
producers: 
hello_world: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
type: 
direct}...
Custom producer class 
class 
HelloProducer 
extends 
Producer 
{ 
public 
function 
publish($msgBody, 
…) 
{ 
$msgBody 
=...
Set content type 
function 
__construct() 
{ 
$this-­‐>setContentType('application/json'); 
} 
! 
public 
function 
publis...
Consumers
Re-queue message 
public 
function 
execute(AMQPMessage 
$msg) 
{ 
if 
('cakper' 
=== 
$msg-­‐>body) 
{ 
return 
false; 
}...
Idle timeouts 
consumers: 
hello_world: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
type: 
direct} 
queue_...
Limit number of messages 
./app/console 
rabbitmq:consumer 
hello_world 
-­‐m 
10
Quality of Service 
qos_options: 
prefetch_size: 
0 
prefetch_count: 
0..65535 
global: 
false/true
Exchanges
Exchange options 
exchange_options: 
name: 
~ 
type: 
direct/fanout/topic 
durable: 
true/false
Queues
Queue options 
queue_options: 
name: 
~ 
durable: 
true/false 
arguments: 
'x-­‐message-­‐ttl': 
['I', 
20000] 
routing_ke...
Purge queue 
./app/console 
rabbitmq:purge 
-­‐-­‐no-­‐confirmation 
hello
Connection
Make it lazy! 
old_sound_rabbit_mq: 
connections: 
default: 
host: 
'localhost' 
port: 
5672 
user: 
'guest' 
password: 
'...
Setup
Setup Fabric 
./app/console 
rabbitmq:setup-­‐fabric 
! 
producers: 
upload_picture: 
auto_setup_fabric: 
false 
consumers...
Multiple consumers
Multiple consumers 
multiple_consumers: 
hello: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
type: 
direct}...
Anonymous Consumers
Anonymous Consumer 
producers: 
hello: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
type: 
topic}
Anonymous Consumer 
anon_consumers: 
hello: 
connection: 
default 
exchange_options: 
{name: 
'hello', 
type: 
topic} 
cal...
Anonymous Consumer 
./app/console_dev 
rabbitmq:anon-­‐consumer 
-­‐r 
'#.vip' 
hello
Management console
Demo time
Summary 
❖ RabbitMQ is fast! 
❖ and reliable! 
❖ also language agnostic! 
❖ and easy to install and use! 
❖ gives you flex...
Kacper Gunia 
Software Engineer 
Thanks! 
Symfony Certified Developer 
PHPers Silesia
Scaling Symfony2 apps with RabbitMQ - Symfony UK Meetup
Upcoming SlideShare
Loading in...5
×

Scaling Symfony2 apps with RabbitMQ - Symfony UK Meetup

7,889

Published on

Slides from my talk at Symfony UK Meetup. London, 20 Aug 2014. http://twitter.com/cakper

Video: https://www.youtube.com/watch?v=cha92Og9M5A

Published in: Technology
3 Comments
41 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,889
On Slideshare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
138
Comments
3
Likes
41
Embeds 0
No embeds

No notes for slide

Transcript of "Scaling Symfony2 apps with RabbitMQ - Symfony UK Meetup"

  1. 1. Scaling Symfony2 apps with RabbitMQ
  2. 2. Kacper Gunia @cakper Software Engineer @SensioLabsUK Symfony Certified Developer PHPers Silesia @PHPersPL
  3. 3. Agenda ❖What is RabbitMQ! ❖How to communicate with it from PHP! ❖How to integrate it with Symfony! ❖Diving into details! ❖Demo
  4. 4. Why would I need Messaging?
  5. 5. Use Cases ❖ Offloading / Background jobs! ❖ Integration ! ❖ Scaling! ❖ Queueing! ❖ Scheduling! ❖ Events
  6. 6. Message-oriented Middleware “(…) allows distributed applications ! to communicate and exchange data ! by sending and receiving messages” http://docs.oracle.com/cd/E19316-01/820-6424/aeraq/index.html
  7. 7. What is inside a Rabbit?
  8. 8. http://madamtruffle.deviantart.com/art/What-­‐is-­‐inside-­‐a-­‐rabbit-­‐81036248
  9. 9. What is inside the RabbitMQ?
  10. 10. Message Broker
  11. 11. AMQP
  12. 12. Erlang
  13. 13. Interoperability ❖ PHP! ❖ Clojure! ❖ Erlang! ❖ Java! ❖ Perl! ❖ Python! ❖ Ruby ❖ C#! ❖ JavaScript! ❖ C/C++! ❖ Go! ❖ Lisp! ❖ Haskell! ❖ …
  14. 14. How does it work?
  15. 15. Broker Bindings Producer Exchange Queue Queue Consumer Consumer Consumer Consumer
  16. 16. Producer Producer
  17. 17. Exchange Exchange
  18. 18. Direct Exchange Exchange Queue
  19. 19. Fanout Exchange Exchange Queue Queue
  20. 20. Topic Exchange #.error #.warning, log.* *.mobile.* log.error log.sql.warning log.mobile.error
  21. 21. “.” - word separator
  22. 22. “#” - prefix/suffix wildcard
  23. 23. “*” - word wildcard
  24. 24. Bindings Bindings Exchange Queue Queue
  25. 25. Queue Queue
  26. 26. Consumer Consumer
  27. 27. How to feed Rabbit from PHP?
  28. 28. PHP libraries and tools ❖ php-amqplib! ❖ PECL AMQP ! ❖ amqphp! ❖ VorpalBunny! ❖ Thumper! ❖ CAMQP
  29. 29. Declare queue $connection = new AMQPConnection ('localhost', 5672, 'guest', 'guest'); $channel = $connection-­‐>channel(); ! $channel-­‐>queue_declare ('hello', false, false, false, false); ! /** Your stuff */ ! $channel-­‐>close(); $connection-­‐>close();
  30. 30. Producer $msg = new AMQPMessage('Hello World!'); $channel-­‐>basic_publish($msg, '', 'hello'); ! echo " [x] Sent 'Hello World!'n";
  31. 31. Consumer $callback = function($msg) { echo ' [x] Received ', $msg-­‐>body, "n"; }; ! $channel-­‐>basic_consume ('hello', '', false, true, false, false, $callback); ! while(count($channel-­‐>callbacks)) { $channel-­‐>wait(); }
  32. 32. RabbitMQ & Symfony https://secure.flickr.com/photos/8725928@N02/9657136424
  33. 33. Step 1
  34. 34. Install RabbitMQ Bundle composer require “oldsound/rabbitmq-­‐bundle 1.5.*”
  35. 35. Step 2
  36. 36. Enable bundle in Kernel public function registerBundles() { $bundles =[ (…) new OldSoundRabbitMqBundleOldSoundRabbitMqBundle() ]; }
  37. 37. Step 3
  38. 38. Say thanks to @old_sound :)
  39. 39. Step 4
  40. 40. Get rid off manual configuration old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: false
  41. 41. Get rid off manual configuration producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
  42. 42. Get rid off manual configuration producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service
  43. 43. Step 5
  44. 44. Produce some data public function indexAction($name) { $this -­‐>get('old_sound_rabbit_mq.hello_world_producer') -­‐>publish($name); }
  45. 45. Step 6
  46. 46. Implement consumer class HelloWorldConsumer implements ConsumerInterface { public function execute(AMQPMessage $msg) { echo "Hello $msg-­‐>body!".PHP_EOL; } }
  47. 47. Step 7
  48. 48. Configure Service Container services: hello_world_service: class: CakperHelloWorldConsumer
  49. 49. Step 8
  50. 50. Run the Consumer ./app/console rabbitmq:consumer hello_world
  51. 51. Nailed it!
  52. 52. Diving into https://secure.flickr.com/photos/toms/159393358
  53. 53. Producers
  54. 54. Custom producer class producers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} class: CakperHelloProducer
  55. 55. Custom producer class class HelloProducer extends Producer { public function publish($msgBody, …) { $msgBody = serialize($msgBody); ! parent::publish($msgBody, …); } }
  56. 56. Set content type function __construct() { $this-­‐>setContentType('application/json'); } ! public function publish($msgBody, …) { parent::publish(json_encode($msgBody), …); }
  57. 57. Consumers
  58. 58. Re-queue message public function execute(AMQPMessage $msg) { if ('cakper' === $msg-­‐>body) { return false; } ! echo "Hello $msg-­‐>body!".PHP_EOL; }
  59. 59. Idle timeouts consumers: hello_world: connection: default exchange_options: {name: 'hello', type: direct} queue_options: {name: 'hello'} callback: hello_world_service idle_timeout: 180
  60. 60. Limit number of messages ./app/console rabbitmq:consumer hello_world -­‐m 10
  61. 61. Quality of Service qos_options: prefetch_size: 0 prefetch_count: 0..65535 global: false/true
  62. 62. Exchanges
  63. 63. Exchange options exchange_options: name: ~ type: direct/fanout/topic durable: true/false
  64. 64. Queues
  65. 65. Queue options queue_options: name: ~ durable: true/false arguments: 'x-­‐message-­‐ttl': ['I', 20000] routing_keys: -­‐ 'logs.sql.#' -­‐ '*.error'
  66. 66. Purge queue ./app/console rabbitmq:purge -­‐-­‐no-­‐confirmation hello
  67. 67. Connection
  68. 68. Make it lazy! old_sound_rabbit_mq: connections: default: host: 'localhost' port: 5672 user: 'guest' password: 'guest' vhost: '/' lazy: true
  69. 69. Setup
  70. 70. Setup Fabric ./app/console rabbitmq:setup-­‐fabric ! producers: upload_picture: auto_setup_fabric: false consumers: upload_picture: auto_setup_fabric: false
  71. 71. Multiple consumers
  72. 72. Multiple consumers multiple_consumers: hello: connection: default exchange_options: {name: 'hello', type: direct} queues: hello-­‐vip: name: hello_vip callback: hello_vip_world_service routing_keys: -­‐ vip hello-­‐regular: name: hello_regular callback: hello_regular_world_service
  73. 73. Anonymous Consumers
  74. 74. Anonymous Consumer producers: hello: connection: default exchange_options: {name: 'hello', type: topic}
  75. 75. Anonymous Consumer anon_consumers: hello: connection: default exchange_options: {name: 'hello', type: topic} callback: hello_world_service
  76. 76. Anonymous Consumer ./app/console_dev rabbitmq:anon-­‐consumer -­‐r '#.vip' hello
  77. 77. Management console
  78. 78. Demo time
  79. 79. Summary ❖ RabbitMQ is fast! ❖ and reliable! ❖ also language agnostic! ❖ and easy to install and use! ❖ gives you flexibility! ❖ supports high-availability, clustering
  80. 80. Kacper Gunia Software Engineer Thanks! Symfony Certified Developer PHPers Silesia
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×