1. Practical Message
Queueing Using
RabbitMQ
James Titcumb
PHPem
3rd July 2014

2. James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim
Who is this guy?

3. Who are you?
https://www.flickr.com/photos/akrabat/10168019755/

4. What is message
queueing?

5. Separation of Concerns

6. Scaling with Rabbit
RabbitMQApplication
Background processing

7. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing

8. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing

9. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing

10. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing
Background processing

11. ● Fast logging solution
Some real world uses...

12. ● Fast logging solution
● Background Processing
Some real world uses...

13. ● Fast logging solution
● Background Processing
○ Sending emails
Some real world uses...

14. ● Fast logging solution
● Background Processing
○ Sending emails
○ Sending SMS
Some real world uses...

15. ● Fast logging solution
● Background Processing
○ Sending emails
○ Sending SMS
○ Analytics, reporting
Some real world uses...

16. (on precise64, other OSs may vary)
Installing RabbitMQ

17. ● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
● apt-get update
● apt-get install rabbitmq-server
● rabbitmq-plugins enable rabbitmq_management
● sudo service rabbitmq-server restart
Using Apt

18. http://localhost:15672/

19. Basic Message Queuing

20. Objective: Basic Queuing
Producer Consumer
test_queue
1 2 3 4 5

21. composer.json
{
"require": {
"videlalvaro/php-amqplib": "2.*"
}
}
then composer install

22. Please wait, connecting...
use PhpAmqpLibConnectionAMQPConnection;
$connection = new AMQPConnection(
'localhost',
5672,
'guest',
'guest',
'/'
);
$channel = $connection->channel();

23. basic/producer.php
use PhpAmqpLibMessageAMQPMessage;
$channel->queue_declare(
'test_queue',
false,
true,
false, false);
$message = new AMQPMessage('my test message');
$channel->basic_publish($message, '', 'test_queue');

24. basic/consumer.php
$channel->basic_consume(
'test_queue', // Queue to consume
'', // Consumer identifier
false,
true, // No-ack means messages are "auto acknowledged"
false, // Exclusive - no other consumers can use the queue
false,
function(AMQPMessage $message) {
echo $message->body . "n";
}
);
while (count($channel->callbacks)) {
$channel->wait();
}

25. What to expect...

26. Exchanges: Fanout

27. Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5

28. fanout/producer.php
use PhpAmqpLibMessageAMQPMessage;
$channel->exchange_declare(
'test_exchange',
'fanout',
false, false, false);
$message = new AMQPMessage('my test message #' . $id);
$channel->basic_publish($message, 'test_exchange');

29. fanout/consumer.php
$q = $channel->queue_declare(
'', // Lets RabbitMQ pick a name for queue
false, false, false,
true // Delete this queue
);
$queue_name = $q[0];
$channel->exchange_declare(
'test_exchange', 'fanout', false, false, false);
$channel->queue_bind($queue_name, 'test_exchange');

30. What to expect...

31. A word on Temporary Queues
● Only the “current” flow
● Specific use cases
test_exchangeProducer
Messages
go nowhere

32. Exchanges: Direct

33. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
3
Message Routing Keys
1 = orange
2 = banana
3 = apple
2 3
BK = orange, banana,
apple
Consumer1 2 3

34. direct/producer.php
$channel->exchange_declare(
'test_direct', 'fanout', false, false, false);
$messageContent = 'my test message, key=' . $routingKey;
$message = new AMQPMessage($messageContent);
$channel->basic_publish($message, 'test_direct', $routingKey);

35. direct/consumer.php
$q = $channel->queue_declare('', false, false, false, true);
$queue_name = $q[0];
$channel->exchange_declare(
'test_direct', 'direct', false, false, false);
// Bind for each routing key we want (BINDING KEY)
$channel->queue_bind($queue_name, 'test_direct', 'apple');
$channel->queue_bind($queue_name, 'test_direct', 'orange');
$channel->queue_bind($queue_name, 'test_direct', 'banana');

36. What to expect...

37. Exchanges: Topic

38. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
1 2
Message Routing Keys
1 = red.vegetable
2 = green.vegetable
3 = green.fruit
4 = red.meat
5 = green.grass.long
1 2 3 4 5
BK = green.#
Consumer2 3 5
BK = *.grass.* / *.*.long
Consumer5

39. Real World Example

41. Fetch message
Logging Sequence
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

42. Flexibility!
● Temporary Queues
○ e.g. inspect “debug” messages

43. Flexibility!
● Temporary Queues
● Queues to log to DB

44. Flexibility!
● Temporary Queues
● Queues to log to DB
● Queues to email “alert/emergency”

45. Flexibility!
● Temporary Queues
● Queues to log to DB
● Queues to email “alert/emergency”
● Get creative with routing keys
○ RK = app.api.error … BK = #.api.error
○ RK = app.form.debug … BK = #.debug

46. Problem: SPOF

47. Solution 1: Clustering

48. Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 5
RabbitMQ
Node 6
Load Balance / Floating IP / Low TTL DNS etc.

49. Clustering
● Everything replicated, except queues

50. Clustering
● Everything replicated, except queues
● Types:
○ RAM
○ Disk

51. Clustering
● Everything replicated, except queues
● Types:
○ RAM
○ Disk
● Configuration:
○ CLI (rabbitmqctl)
○ Configuration files

52. Creating a cluster
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}]
...done.
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}]
...done.

53. node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.
Creating a cluster

54. node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.
Managing Nodes
● Stopping/starting nodes
● Removing nodes:

55. Solution 2: HA

56. HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.

57. https://github.com/asgrim/rmq-slides
Have a go yourself!

58. Questions?

59. James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim
Thanks for watching!