As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ. In this talk, we will take a look at the techniques you can use to vastly enhance inter-application communication, learn about the core concepts of RabbitMQ, cover how you can scale different parts of your application separately, and modernise your development using a message-oriented architecture

1. Adding 1.21 Gigawatts
to Applications
with RabbitMQ
James Titcumb
PHPNW December 2014

2. Who is this guy?
James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim

3. What is message
queueing?

4. Separation of Concerns

5. Scaling with Rabbit
Application RabbitMQ
Background processing

6. Scaling with Rabbit
Application RabbitMQ
Background processing
Background processing

7. Scaling with Rabbit
Application RabbitMQ
Background processing
Background processing
Background processing

8. Scaling with Rabbit
Application RabbitMQ
Background processing
Background processing
Background processing
Background processing

9. Scaling with Rabbit
Application RabbitMQ
Background processing
Background processing
Background processing
Background processing
Background processing

10. Real world uses?

11. Why RabbitMQ?

12. Installing RabbitMQ
(on precise64, other OSs may vary)

13. Using Apt
● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
● apt-get update
● apt-get install rabbitmq-server
● rabbitmq-plugins enable rabbitmq_management
● sudo service rabbitmq-server restart

14. http://localhost:15672/

15. Basic Message Queuing

16. Objective: Basic Queuing
test_queue
1 2 3 4 5
Producer Consumer

17. composer.json
{
"require": {
"videlalvaro/php-amqplib": "2.*"
}
}
then composer install

18. Please wait, connecting...
use PhpAmqpLibConnectionAMQPConnection;
$connection = new AMQPConnection(
'localhost',
5672,
'guest',
'guest',
'/'
);
$channel = $connection->channel();

19. basic/producer.php
use PhpAmqpLibMessageAMQPMessage;
$channel->queue_declare(
'test_queue',
false,
true,
false, false);
$message = new AMQPMessage('my test message');
$channel->basic_publish($message, '', 'test_queue');

20. basic/consumer.php
$channel->basic_consume(
'test_queue', // Queue to consume
'', // Consumer identifier
false,
true, // No-ack means messages are "auto acknowledged"
false, // Exclusive - no other consumers can use the queue
false,
function(AMQPMessage $message) {
echo $message->body . "n";
}
);
while (count($channel->callbacks)) {
$channel->wait();
}

21. What to expect...

22. Exchanges: Fanout

23. Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5

24. fanout/producer.php
use PhpAmqpLibMessageAMQPMessage;
$channel->exchange_declare(
'test_exchange',
'fanout',
false, false, false);
$message = new AMQPMessage('my test message #' . $id);
$channel->basic_publish($message, 'test_exchange');

25. fanout/consumer.php
$q = $channel->queue_declare(
'', // Lets RabbitMQ pick a name for queue
false, false, false,
true // Delete this queue
);
$queue_name = $q[0];
$channel->exchange_declare(
'test_exchange', 'fanout', false, false, false);
$channel->queue_bind($queue_name, 'test_exchange');

26. What to expect...

27. A word on Temporary Queues
Producer test_exchange
Messages
go nowhere

28. Exchanges: Direct

29. Objective: Direct Exchange
test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
BK = orange, banana,
apple
Consumer

30. Objective: Direct Exchange
test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= ORANGE
BK = orange, banana,
apple
Consumer

31. Objective: Direct Exchange
test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= BANANA
BK = orange, banana,
apple
Consumer

32. Objective: Direct Exchange
test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= APPLE
BK = orange, banana,
apple
Consumer

33. direct/producer.php
$channel->exchange_declare(
'test_direct', 'fanout', false, false, false);
$messageContent = 'my test message, key=' . $routingKey;
$message = new AMQPMessage($messageContent);
$channel->basic_publish($message, 'test_direct', $routingKey);

34. direct/consumer.php
$q = $channel->queue_declare('', false, false, false, true);
$queue_name = $q[0];
$channel->exchange_declare(
'test_direct', 'direct', false, false, false);
// Bind for each routing key we want (BINDING KEY)
$channel->queue_bind($queue_name, 'test_direct', 'apple');
$channel->queue_bind($queue_name, 'test_direct', 'orange');
$channel->queue_bind($queue_name, 'test_direct', 'banana');

35. What to expect...

36. Exchanges: Topic

37. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

38. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

39. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

40. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

41. Real World Example

43. Fetch message
Logging Sequence
Browser Application Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

44. Flexibility!

45. RPC

46. TTL

47. DLX

48. Infrastructure

49. Problem: SPOF

50. Solution 1: Clustering

51. Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 6
RabbitMQ
Node 5
Load Balance / Floating IP / Low TTL DNS etc.

52. Everything Replicates
(except queues…)

53. RAM / Disk

54. Configuration...

55. Creating a cluster
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}]
...done.
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}]
...done.

56. Creating a cluster
node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.

57. Starting/Stopping Nodes

58. Removing Nodes
node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.

59. Solution 2: HA

60. HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.

61. Have a go yourself!
https://github.com/asgrim/rmq-slides

62. Questions?

63. Thanks for watching!
James Titcumb
@asgrim