Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Practical Message
Queueing Using
RabbitMQ
James Titcumb
Nomad PHP EU December 2014
James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
@asgrim
Who is this guy?
What is message
queueing?
Separation of Concerns
Scaling with Rabbit
RabbitMQApplication
Background processing
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background proce...
Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background proce...
Real world uses?
Why RabbitMQ?
(on precise64, other OSs may vary)
Installing RabbitMQ
● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-sig...
http://localhost:15672/
http://localhost:15672/
http://localhost:15672/
http://localhost:15672/
http://localhost:15672/
Basic Message Queuing
Objective: Basic Queuing
Producer Consumer
test_queue
1 2 3 4 5
composer.json
1 {
2 "require": {
3 "videlalvaro/php-amqplib": "2.*"
4 }
5 }
then composer install
Please wait, connecting...
1 use PhpAmqpLibConnectionAMQPConnection;
2
3 $connection = new AMQPConnection(
4 'localhost',
...
basic/producer.php
1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->queue_declare(
4 'test_queue',
5 false,
6 true,
7 fal...
basic/consumer.php
1 $channel->basic_consume(
2 'test_queue', // Queue to consume
3 '', // Consumer identifier
4 false,
5 ...
What to expect...
Exchanges: Fanout
Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5
1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->exchange_declare(
4 'test_exchange',
5 'fanout',
6 false,
7 false,
8 fal...
fanout/consumer.php
1 $q = $channel->queue_declare(
2 '', // Lets RabbitMQ pick a name for queue
3 false, false, false,
4 ...
What to expect...
A word on Temporary Queues
test_exchangeProducer
Messages
go nowhere
Exchanges: Direct
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
BK = orange, banana,
apple...
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= ORAN...
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= BANA...
Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= APPL...
direct/producer.php
1 $channel->exchange_declare(
2 'test_direct', 'fanout', false, false, false);
3
4 $messageContent = '...
direct/consumer.php
1 $q = $channel->queue_declare('', false, false, false, true);
2 $queue_name = $q[0];
3 $channel->exch...
What to expect...
Exchanges: Topic
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK = *.grass...
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#
Consume...
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green.#
Consu...
Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = green.#
Cons...
Real World Example
Fetch message
Logging Sequence
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ
Flexibility!
Parallel Processing
Message
Acknowledgement
RPC
TTL
DLX
http://tryrabbitmq.com/
Infrastructure
Problem: SPOF
Solution 1: Clustering
Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 5
RabbitMQ
Node 6
Load Balance / ...
Everything Replicates
(except queues…)
RAM / Disk
Configuration...
/etc/rabbitmq/rabbitmq.config
[
{rabbit, [
{loopback_users, []},
{vm_memory_high_watermark, 0.8}
]}
].
Creating a cluster
node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]...
node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster...
Starting/Stopping Nodes
node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqct...
Solution 2: HA
HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.
https://github.com/asgrim/rmq-slides
Have a go yourself!
Questions?
https://joind.in/13093
James Titcumb
@asgrim
Thanks for watching!
Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014)
Upcoming SlideShare
Loading in …5
×

of

Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 1 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 2 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 3 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 4 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 5 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 6 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 7 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 8 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 9 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 10 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 11 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 12 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 13 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 14 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 15 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 16 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 17 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 18 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 19 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 20 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 21 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 22 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 23 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 24 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 25 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 26 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 27 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 28 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 29 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 30 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 31 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 32 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 33 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 34 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 35 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 36 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 37 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 38 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 39 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 40 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 41 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 42 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 43 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 44 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 45 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 46 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 47 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 48 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 49 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 50 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 51 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 52 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 53 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 54 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 55 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 56 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 57 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 58 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 59 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 60 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 61 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 62 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 63 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 64 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 65 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 66 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 67 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 68 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 69 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 70 Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014) Slide 71
Upcoming SlideShare
zeromq
Next
Download to read offline and view in fullscreen.

1 Like

Share

Download to read offline

Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014)

Download to read offline

RabbitMQ is a message broker – an application that allows communication between applications by way of a message queuing system. In this talk, we’ll set up a RabbitMQ instance, take an intermediate-level look into the technical features it provides and also how you can apply RabbitMQ in your applications to scale them efficiently.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Practical Message Queueing using RabbitMQ (Nomad PHP EU Dec 2014)

  1. 1. Practical Message Queueing Using RabbitMQ James Titcumb Nomad PHP EU December 2014
  2. 2. James Titcumb www.jamestitcumb.com www.protected.co.uk www.phphants.co.uk @asgrim Who is this guy?
  3. 3. What is message queueing?
  4. 4. Separation of Concerns
  5. 5. Scaling with Rabbit RabbitMQApplication Background processing
  6. 6. Scaling with Rabbit RabbitMQApplication Background processing Background processing
  7. 7. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing
  8. 8. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing
  9. 9. Scaling with Rabbit RabbitMQApplication Background processing Background processing Background processing Background processing Background processing
  10. 10. Real world uses?
  11. 11. Why RabbitMQ?
  12. 12. (on precise64, other OSs may vary) Installing RabbitMQ
  13. 13. ● add apt repo ○ deb http://www.rabbitmq.com/debian/ testing main ● add signing key ○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc ● apt-get update ● apt-get install rabbitmq-server ● rabbitmq-plugins enable rabbitmq_management ● sudo service rabbitmq-server restart Using Apt
  14. 14. http://localhost:15672/
  15. 15. http://localhost:15672/
  16. 16. http://localhost:15672/
  17. 17. http://localhost:15672/
  18. 18. http://localhost:15672/
  19. 19. Basic Message Queuing
  20. 20. Objective: Basic Queuing Producer Consumer test_queue 1 2 3 4 5
  21. 21. composer.json 1 { 2 "require": { 3 "videlalvaro/php-amqplib": "2.*" 4 } 5 } then composer install
  22. 22. Please wait, connecting... 1 use PhpAmqpLibConnectionAMQPConnection; 2 3 $connection = new AMQPConnection( 4 'localhost', 5 5672, 6 'guest', 7 'guest', 8 '/' 9 ); 10 $channel = $connection->channel();
  23. 23. basic/producer.php 1 use PhpAmqpLibMessageAMQPMessage; 2 3 $channel->queue_declare( 4 'test_queue', 5 false, 6 true, 7 false, 8 false); 9 10 $message = new AMQPMessage('my test message'); 11 $channel->basic_publish($message, '', 'test_queue');
  24. 24. basic/consumer.php 1 $channel->basic_consume( 2 'test_queue', // Queue to consume 3 '', // Consumer identifier 4 false, 5 true, // No-ack means messages are "auto acknowledged" 6 false, // Exclusive - no other consumers can use the queue 7 false, 8 function(AMQPMessage $message) { 9 echo $message->body . "n"; 10 } 11 ); 12 13 while (count($channel->callbacks)) { 14 $channel->wait(); 15 }
  25. 25. What to expect...
  26. 26. Exchanges: Fanout
  27. 27. Objective: Fanout Exchange test_exchange amq.KfgPZ3PE amq.cK5Cp3FC Consumer Consumer Producer 1 1 2 2 3 3 4 4 5 5
  28. 28. 1 use PhpAmqpLibMessageAMQPMessage; 2 3 $channel->exchange_declare( 4 'test_exchange', 5 'fanout', 6 false, 7 false, 8 false); 9 10 $message = new AMQPMessage('my test message #' . $id); 11 $channel->basic_publish($message, 'test_exchange'); fanout/producer.php
  29. 29. fanout/consumer.php 1 $q = $channel->queue_declare( 2 '', // Lets RabbitMQ pick a name for queue 3 false, false, false, 4 true // Delete this queue 5 ); 6 $queue_name = $q[0]; 7 8 $channel->exchange_declare( 9 'test_exchange', 'fanout', false, false, false); 10 $channel->queue_bind($queue_name, 'test_exchange');
  30. 30. What to expect...
  31. 31. A word on Temporary Queues test_exchangeProducer Messages go nowhere
  32. 32. Exchanges: Direct
  33. 33. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer BK = orange, banana, apple Consumer
  34. 34. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = ORANGE BK = orange, banana, apple Consumer
  35. 35. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = BANANA BK = orange, banana, apple Consumer
  36. 36. Objective: Direct Exchange test_direct BK = apple BK = banana, apple Consumer Consumer Producer MESSAGE ROUTING KEY = APPLE BK = orange, banana, apple Consumer
  37. 37. direct/producer.php 1 $channel->exchange_declare( 2 'test_direct', 'fanout', false, false, false); 3 4 $messageContent = 'my test message, key=' . $routingKey; 5 $message = new AMQPMessage($messageContent); 6 $channel->basic_publish( 7 $message, 8 'test_direct', 9 $routingKey 10 );
  38. 38. direct/consumer.php 1 $q = $channel->queue_declare('', false, false, false, true); 2 $queue_name = $q[0]; 3 $channel->exchange_declare( 4 'test_direct', 'direct', false, false, false); 5 6 // Bind for each routing key we want (BINDING KEY) 7 $channel->queue_bind($queue_name, 'test_direct', 'apple'); 8 $channel->queue_bind($queue_name, 'test_direct', 'orange'); 9 $channel->queue_bind($queue_name, 'test_direct', 'banana');
  39. 39. What to expect...
  40. 40. Exchanges: Topic
  41. 41. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  42. 42. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer RED.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  43. 43. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.VEGETABLE BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  44. 44. Objective: Topic Exchange test_topic BK = *.vegetable BK = # Consumer Consumer Producer GREEN.GRASS.LONG BK = green.# Consumer BK = *.grass.* / *.*.long Consumer
  45. 45. Real World Example
  46. 46. Fetch message Logging Sequence ApplicationBrowser Log Server HTTP request JSON via AMQP Error! HTTP response RabbitMQ
  47. 47. Flexibility!
  48. 48. Parallel Processing
  49. 49. Message Acknowledgement
  50. 50. RPC
  51. 51. TTL
  52. 52. DLX
  53. 53. http://tryrabbitmq.com/
  54. 54. Infrastructure
  55. 55. Problem: SPOF
  56. 56. Solution 1: Clustering
  57. 57. Clustering RabbitMQ Node 1 RabbitMQ Node 3 RabbitMQ Node 2 RabbitMQ Node 4 RabbitMQ Node 5 RabbitMQ Node 6 Load Balance / Floating IP / Low TTL DNS etc.
  58. 58. Everything Replicates (except queues…)
  59. 59. RAM / Disk
  60. 60. Configuration...
  61. 61. /etc/rabbitmq/rabbitmq.config [ {rabbit, [ {loopback_users, []}, {vm_memory_high_watermark, 0.8} ]} ].
  62. 62. Creating a cluster node1$ rabbitmqctl cluster_status Cluster status of node rabbit@node1 ... [{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}] ...done. node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node2]}]},{running_nodes,[rabbit@node2]}] ...done. node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3]}]},{running_nodes,[rabbit@node3]}] ...done.
  63. 63. node2$ rabbitmqctl join_cluster --ram rabbit@node1 node3$ rabbitmqctl join_cluster rabbit@node2 node3$ rabbitmqctl cluster_status Cluster status of node rabbit@node3 ... [{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}] ...done. Creating a cluster
  64. 64. Starting/Stopping Nodes
  65. 65. node1$ rabbitmqctl stop_app node2$ rabbitmqctl forget_cluster_node rabbit@node1 node1$ rabbitmqctl reset node1$ rabbitmqctl start_app node2$ rabbitmqctl cluster_status Cluster status of node rabbit@node2 ... [{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]}, {running_nodes,[rabbit@node2,rabbit@node3]}] ...done. Removing Nodes
  66. 66. Solution 2: HA
  67. 67. HA + Queue Mirroring RabbitMQ Node 1 RabbitMQ Node 2 Load Balance / Floating IP / Low TTL DNS etc.
  68. 68. https://github.com/asgrim/rmq-slides Have a go yourself!
  69. 69. Questions?
  70. 70. https://joind.in/13093 James Titcumb @asgrim Thanks for watching!
  • redlofa

    Apr. 24, 2017

RabbitMQ is a message broker – an application that allows communication between applications by way of a message queuing system. In this talk, we’ll set up a RabbitMQ instance, take an intermediate-level look into the technical features it provides and also how you can apply RabbitMQ in your applications to scale them efficiently.

Views

Total views

2,139

On Slideshare

0

From embeds

0

Number of embeds

25

Actions

Downloads

21

Shares

0

Comments

0

Likes

1

×