Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

Adding 1.21 Gigawatts
to Applications
with RabbitMQ
James Titcumb
PHPNW December 2014

Who is this guy?
James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim

Scaling with Rabbit
Application RabbitMQ
Background processing

Scaling with Rabbit

Installing RabbitMQ
(on precise64, other OSs may vary)

Using Apt
● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
● apt-get update
● apt-get install rabbitmq-server
● rabbitmq-plugins enable rabbitmq_management
● sudo service rabbitmq-server restart

Objective: Basic Queuing
test_queue
1 2 3 4 5
Producer Consumer

composer.json
{
"require": {
"videlalvaro/php-amqplib": "2.*"
}
}
then composer install

Please wait, connecting...
use PhpAmqpLibConnectionAMQPConnection;
$connection = new AMQPConnection(
'localhost',
5672,
'guest',
'guest',
'/'
);
$channel = $connection->channel();

basic/producer.php
use PhpAmqpLibMessageAMQPMessage;
$channel->queue_declare(
'test_queue',
false,
true,
false, false);
$message = new AMQPMessage('my test message');
$channel->basic_publish($message, '', 'test_queue');

basic/consumer.php
$channel->basic_consume(
'test_queue', // Queue to consume
'', // Consumer identifier
false,
true, // No-ack means messages are "auto acknowledged"
false, // Exclusive - no other consumers can use the queue
false,
function(AMQPMessage $message) {
echo $message->body . "n";
}
);
while (count($channel->callbacks)) {
$channel->wait();
}

Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5

fanout/producer.php
use PhpAmqpLibMessageAMQPMessage;
$channel->exchange_declare(
'test_exchange',
'fanout',
false, false, false);
$message = new AMQPMessage('my test message #' . $id);
$channel->basic_publish($message, 'test_exchange');

fanout/consumer.php
$q = $channel->queue_declare(
'', // Lets RabbitMQ pick a name for queue
false, false, false,
true // Delete this queue
);
$queue_name = $q[0];
'test_exchange', 'fanout', false, false, false);
$channel->queue_bind($queue_name, 'test_exchange');

A word on Temporary Queues
Producer test_exchange
Messages
go nowhere

Objective: Direct Exchange
test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
BK = orange, banana,
apple
Consumer

test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= ORANGE
apple
Consumer

test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= BANANA
apple
Consumer

test_direct
BK = banana, apple
BK = apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= APPLE
apple
Consumer

direct/producer.php
'test_direct', 'fanout', false, false, false);
$messageContent = 'my test message, key=' . $routingKey;
$message = new AMQPMessage($messageContent);
$channel->basic_publish($message, 'test_direct', $routingKey);

direct/consumer.php
$q = $channel->queue_declare('', false, false, false, true);
$queue_name = $q[0];
'test_direct', 'direct', false, false, false);
// Bind for each routing key we want (BINDING KEY)
$channel->queue_bind($queue_name, 'test_direct', 'apple');
$channel->queue_bind($queue_name, 'test_direct', 'orange');
$channel->queue_bind($queue_name, 'test_direct', 'banana');

Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#
Consumer
Consumer

Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green.#
Consumer
Consumer

Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = green.#
Consumer
Consumer

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

Fetch message
Logging Sequence
Browser Application Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 6
RabbitMQ
Node 5
Load Balance / Floating IP / Low TTL DNS etc.

Everything Replicates
(except queues…)

Creating a cluster
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
...done.
...done.

Creating a cluster
node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.

Removing Nodes
node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.

HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.

Have a go yourself!
https://github.com/asgrim/rmq-slides

Thanks for watching!
James Titcumb
@asgrim

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

Recommended

More Related Content

What's hot

What's hot(20)

Viewers also liked

Viewers also liked(20)

Similar to Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)

Similar to Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)(20)

More from James Titcumb

More from James Titcumb(20)

Recently uploaded

Recently uploaded(20)

Adding 1.21 Gigawatts to Applications with RabbitMQ (PHPNW Dec 2014 Meetup)