Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Adding 1.21 Gigawatts 
to Applications 
with RabbitMQ 
James Titcumb 
PHPNW December 2014
Who is this guy? 
James Titcumb 
www.jamestitcumb.com 
www.protected.co.uk 
www.phphants.co.uk 
@asgrim
What is message 
queueing?
Separation of Concerns
Scaling with Rabbit 
Application RabbitMQ 
Background processing
Scaling with Rabbit 
Application RabbitMQ 
Background processing 
Background processing
Scaling with Rabbit 
Application RabbitMQ 
Background processing 
Background processing 
Background processing
Scaling with Rabbit 
Application RabbitMQ 
Background processing 
Background processing 
Background processing 
Background...
Scaling with Rabbit 
Application RabbitMQ 
Background processing 
Background processing 
Background processing 
Background...
Real world uses?
Why RabbitMQ?
Installing RabbitMQ 
(on precise64, other OSs may vary)
Using Apt 
● add apt repo 
○ deb http://www.rabbitmq.com/debian/ testing main 
● add signing key 
○ http://www.rabbitmq.co...
http://localhost:15672/
Basic Message Queuing
Objective: Basic Queuing 
test_queue 
1 2 3 4 5 
Producer Consumer
composer.json 
{ 
"require": { 
"videlalvaro/php-amqplib": "2.*" 
} 
} 
then composer install
Please wait, connecting... 
use PhpAmqpLibConnectionAMQPConnection; 
$connection = new AMQPConnection( 
'localhost', 
5672...
basic/producer.php 
use PhpAmqpLibMessageAMQPMessage; 
$channel->queue_declare( 
'test_queue', 
false, 
true, 
false, fals...
basic/consumer.php 
$channel->basic_consume( 
'test_queue', // Queue to consume 
'', // Consumer identifier 
false, 
true,...
What to expect...
Exchanges: Fanout
Objective: Fanout Exchange 
test_exchange 
amq.KfgPZ3PE 
amq.cK5Cp3FC 
Consumer 
Consumer 
Producer 
1 
1 
2 
2 
3 
3 
4 
...
fanout/producer.php 
use PhpAmqpLibMessageAMQPMessage; 
$channel->exchange_declare( 
'test_exchange', 
'fanout', 
false, f...
fanout/consumer.php 
$q = $channel->queue_declare( 
'', // Lets RabbitMQ pick a name for queue 
false, false, false, 
true...
What to expect...
A word on Temporary Queues 
Producer test_exchange 
Messages 
go nowhere
Exchanges: Direct
Objective: Direct Exchange 
test_direct 
BK = banana, apple 
BK = apple 
Consumer 
Consumer 
Producer 
BK = orange, banana...
Objective: Direct Exchange 
test_direct 
BK = banana, apple 
BK = apple 
Consumer 
Consumer 
Producer 
MESSAGE 
ROUTING KE...
Objective: Direct Exchange 
test_direct 
BK = banana, apple 
BK = apple 
Consumer 
Consumer 
Producer 
MESSAGE 
ROUTING KE...
Objective: Direct Exchange 
test_direct 
BK = banana, apple 
BK = apple 
Consumer 
Consumer 
Producer 
MESSAGE 
ROUTING KE...
direct/producer.php 
$channel->exchange_declare( 
'test_direct', 'fanout', false, false, false); 
$messageContent = 'my te...
direct/consumer.php 
$q = $channel->queue_declare('', false, false, false, true); 
$queue_name = $q[0]; 
$channel->exchang...
What to expect...
Exchanges: Topic
Objective: 
Topic Exchange 
test_topic 
BK = *.vegetable 
BK = # 
Consumer 
Consumer 
Producer 
BK = green.# 
Consumer 
BK...
Objective: 
Topic Exchange 
test_topic 
BK = *.vegetable 
BK = # 
Consumer 
Consumer 
Producer 
RED.VEGETABLE 
BK = green....
Objective: 
Topic Exchange 
test_topic 
BK = *.vegetable 
BK = # 
Consumer 
Consumer 
Producer 
GREEN.VEGETABLE 
BK = gree...
Objective: 
Topic Exchange 
test_topic 
BK = *.vegetable 
BK = # 
Consumer 
Consumer 
Producer 
GREEN.GRASS.LONG 
BK = gre...
Real World Example
Fetch message 
Logging Sequence 
Browser Application Log Server 
HTTP request 
JSON via AMQP 
Error! 
HTTP response 
Rabbi...
Flexibility!
RPC
TTL
DLX
Infrastructure
Problem: SPOF
Solution 1: Clustering
Clustering 
RabbitMQ 
Node 1 
RabbitMQ 
Node 3 
RabbitMQ 
Node 2 
RabbitMQ 
Node 4 
RabbitMQ 
Node 6 
RabbitMQ 
Node 5 
Lo...
Everything Replicates 
(except queues…)
RAM / Disk
Configuration...
Creating a cluster 
node1$ rabbitmqctl cluster_status 
Cluster status of node rabbit@node1 ... 
[{nodes,[{disc,[rabbit@nod...
Creating a cluster 
node2$ rabbitmqctl join_cluster --ram rabbit@node1 
node3$ rabbitmqctl join_cluster rabbit@node2 
node...
Starting/Stopping Nodes
Removing Nodes 
node1$ rabbitmqctl stop_app 
node2$ rabbitmqctl forget_cluster_node rabbit@node1 
node1$ rabbitmqctl reset...
Solution 2: HA
HA + Queue Mirroring 
RabbitMQ 
Node 1 
RabbitMQ 
Node 2 
Load Balance / Floating IP / Low TTL DNS etc.
Have a go yourself! 
https://github.com/asgrim/rmq-slides
Questions?
Thanks for watching! 
James Titcumb 
@asgrim
Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)
Upcoming SlideShare
Loading in …5
×

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

849 views

Published on

As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ. In this talk, we will take a look at the techniques you can use to vastly enhance inter-application communication, learn about the core concepts of RabbitMQ, cover how you can scale different parts of your application separately, and modernise your development using a message-oriented architecture

Published in: Technology
  • Be the first to comment

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

  1. 1. Adding 1.21 Gigawatts to Applications with RabbitMQ James Titcumb PHPNW December 2014
  2. 2. Who is this guy? James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim
  3. 3. What is message queueing?
  4. 4. Separation of Concerns
  5. 5. Scaling with Rabbit Application RabbitMQ Background processing
  6. 6. Scaling with Rabbit Application RabbitMQ Background processing Background processing
  7. 7. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing
  8. 8. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing Background processing
  9. 9. Scaling with Rabbit Application RabbitMQ Background processing Background processing Background processing Background processing Background processing
  10. 10. Real world uses?
  11. 11. Why RabbitMQ?
  12. 12. Installing RabbitMQ (on precise64, other OSs may vary)
  13. 13. Using Apt ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart
  14. 14. http://localhost:15672/
  15. 15. Basic Message Queuing
  16. 16. Objective: Basic Queuing test_queue 1 2 3 4 5 Producer Consumer
  17. 17. composer.json { "require": { "videlalvaro/php-amqplib": "2.*" } } then composer install
  18. 18. Please wait, connecting... use PhpAmqpLibConnectionAMQPConnection; $connection = new AMQPConnection( 'localhost', 5672, 'guest', 'guest', '/' ); $channel = $connection->channel();
  19. 19. basic/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->queue_declare( 'test_queue', false, true, false, false); $message = new AMQPMessage('my test message'); $channel->basic_publish($message, '', 'test_queue');
  20. 20. basic/consumer.php $channel->basic_consume( 'test_queue', // Queue to consume '', // Consumer identifier false, true, // No-ack means messages are "auto acknowledged" false, // Exclusive - no other consumers can use the queue false, function(AMQPMessage $message) { echo $message->body . "n"; } ); while (count($channel->callbacks)) { $channel->wait(); }
  21. 21. What to expect...
  22. 22. Exchanges: Fanout
  23. 23. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  24. 24. fanout/producer.php use PhpAmqpLibMessageAMQPMessage; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $message = new AMQPMessage('my test message #' . $id); $channel->basic_publish($message, 'test_exchange');
  25. 25. fanout/consumer.php $q = $channel->queue_declare( '', // Lets RabbitMQ pick a name for queue false, false, false, true // Delete this queue ); $queue_name = $q[0]; $channel->exchange_declare( 'test_exchange', 'fanout', false, false, false); $channel->queue_bind($queue_name, 'test_exchange');
  26. 26. What to expect...
  27. 27. A word on Temporary Queues Producer test_exchange Messages go nowhere
  28. 28. Exchanges: Direct
  29. 29. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  30. 30. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  31. 31. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  32. 32. Objective: Direct Exchange test_direct BK = banana, apple BK = apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  33. 33. direct/producer.php $channel->exchange_declare( 'test_direct', 'fanout', false, false, false); $messageContent = 'my test message, key=' . $routingKey; $message = new AMQPMessage($messageContent); $channel->basic_publish($message, 'test_direct', $routingKey);
  34. 34. direct/consumer.php $q = $channel->queue_declare('', false, false, false, true); $queue_name = $q[0]; $channel->exchange_declare( 'test_direct', 'direct', false, false, false); // Bind for each routing key we want (BINDING KEY) $channel->queue_bind($queue_name, 'test_direct', 'apple'); $channel->queue_bind($queue_name, 'test_direct', 'orange'); $channel->queue_bind($queue_name, 'test_direct', 'banana');
  35. 35. What to expect...
  36. 36. Exchanges: Topic
  37. 37. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  38. 38. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  39. 39. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  40. 40. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  41. 41. Real World Example
  42. 42. Fetch message Logging Sequence Browser Application Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  43. 43. Flexibility!
  44. 44. RPC
  45. 45. TTL
  46. 46. DLX
  47. 47. Infrastructure
  48. 48. Problem: SPOF
  49. 49. Solution 1: Clustering
  50. 50. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 6 RabbitMQ Node 5 Load Balance / Floating IP / Low TTL DNS etc.
  51. 51. Everything Replicates (except queues…)
  52. 52. RAM / Disk
  53. 53. Configuration...
  54. 54. Creating a cluster node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}] ...done. node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}] ...done.
  55. 55. Creating a cluster node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done.
  56. 56. Starting/Stopping Nodes
  57. 57. Removing Nodes node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done.
  58. 58. Solution 2: HA
  59. 59. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  60. 60. Have a go yourself! https://github.com/asgrim/rmq-slides
  61. 61. Questions?
  62. 62. Thanks for watching! James Titcumb @asgrim

×