Dissecting the Rabbit:
RabbitMQ Internal Architecture
Alvaro Videla - RabbitMQ
Alvaro Videla
•
•
•
•
•

Developer Advocate at Pivotal / RabbitMQ!
Co-Author of RabbitMQ in Action!
Creator of the RabbitM...
About Me
Co-authored!
!

RabbitMQ in Action!
http://bit.ly/rabbitmq
Agenda
•

Intro to RabbitMQ

•

Dive into RabbitMQ Internals

•

A day in the life of a message

•

RabbitMQ message store...
What is RabbitMQ
RabbitMQ
RabbitMQ
RabbitMQ
• Multi Protocol Messaging Server
RabbitMQ
• Multi Protocol Messaging Server!
• Open Source (MPL)
RabbitMQ
• Multi Protocol Messaging Server!
• Open Source (MPL)!
• Polyglot
RabbitMQ
• Multi Protocol Messaging Server!
• Open Source (MPL)!
• Polyglot!
• Written in Erlang/OTP
Multi Protocol

http://bit.ly/rmq-protocols
Polyglot
Polyglot
Polyglot
• Java
Polyglot
• Java!
• node.js
Polyglot
• Java!
• node.js!
• Erlang
Polyglot
• Java!
• node.js!
• Erlang!
• PHP
Polyglot
• Java!
• node.js!
• Erlang!
• PHP!
• Ruby
Polyglot
• Java!
• node.js!
• Erlang!
• PHP!
• Ruby!
• .Net
Polyglot
• Java!
• node.js!
• Erlang!
• PHP!
• Ruby!
• .Net!
• Haskell
Polyglot

Even COBOL!!!11
Some users of RabbitMQ
Some users of RabbitMQ
•

Instagram
Some users of RabbitMQ
•
•

Instagram!
Indeed.com
Some users of RabbitMQ
•
•
•

Instagram!
Indeed.com!
MailboxApp
Some users of RabbitMQ
•
•
•
•

Instagram!
Indeed.com!
MailboxApp!
Mercado Libre
Some users of RabbitMQ
•
•
•
•
•

Instagram!
Indeed.com!
MailboxApp!
Mercado Libre!
NHS
Some users of RabbitMQ
•
•
•
•
•
•

Instagram!
Indeed.com!
MailboxApp!
Mercado Libre!
NHS!
Mozilla
http://www.rabbitmq.com/download.html

Unix - Mac - Windows
Messaging with RabbitMQ
A demo with the RabbitMQ Simulator

https://github.com/RabbitMQSimulator/RabbitMQSimulator
http://tryrabbitmq.com
RabbitMQ Simulator
RabbitMQ Internals
A day in the life of a message
A day in the life of a message
A day in the life of a message
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);!
$ch = $conn->channel();!
!

$ch...
What happens here?

$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);!
$ch = $conn->channel();
What happens here?
Erlang
Erlang App
•

Processes (probably thousands)

•

They communicate sending messages

•

Each process has a message queue (d...
Read More Here:
http://jlouisramblings.blogspot.ru/2013/01/how-erlang-does-scheduling.html
Erlang code structure
•

Modules

•

Functions

•

Function Arity

•

Arguments

M, F, A = Module, Function, Arguments
rabbit_client_sup.erl
-module(rabbit_client_sup).!
!

-behaviour(supervisor2).!
!

-export([start_link/1, start_link/2, st...
rabbit_client_sup.erl
start_link(Callback) ->!
supervisor2:start_link(?MODULE, Callback).!
!

start_link(SupName, Callback...
Supervision Trees
Supervision tree
•

Worker Processes

•

Supervisor Processes

•

Supervision tree as a hierarchical arrangement of proces...
rabbit_client_sup
init({M,F,A}) ->!
{ok, {{simple_one_for_one, 0, 1},!
[{client, {M,F,A}, temporary, !
! ! ! !
infinity, s...
Child Spec - restart strategies
•

one_for_one: only restart failing process

•

one_for_all: restart failing process and ...
Child Specification
child_spec() = {Id,StartFunc,Restart,Shutdown,Type,Modules}!
Id = term()!
StartFunc = {M,F,A}!
M = F = ...
rabbit_client_sup
init({M,F,A}) ->!
{ok, {{simple_one_for_one, 0, 1},!
[{client, {M,F,A}, temporary, !
! ! ! !
infinity, s...
Child Spec - restart
•

permanent: should always be restarted

•

temporary: should never be restarted

•

transient: shou...
Child Spec - shutdown
•

brutal_kill: child is terminated immediately.

•

timeout in seconds: supervisor waits for timeou...
Connection Supervision Tree
Connection Supervision Tree
Connection Supervision Tree
Connection Supervision Tree
Connection Supervision Tree
A day in the life of a message
A day in the life of a message
A day in the life of a message
Intermezzo: pattern matching
check_user_id_header(#'P_basic'{user_id = undefined}, _) ->!
ok;!
check_user_id_header(#'P_ba...
Read More Here:
http://videlalvaro.github.io/2013/09/rabbitmq-validating-user-ids-with-erlang-pattern-matching.html
A day in the life of a message
A day in the life of a message
A day in the life of a message
We have a list of queues
where the channel will
deliver the messages
A day in the life of a message
A day in the life of a message

If consumer ready
RabbitMQ Message Store
•

Made specifically for messaging

•

Keeps messages in memory and (sometimes) on disk

•

Bounded ...
RabbitMQ Message Store
•

Message written to disk when:
•

Message published as persistent (delivery_mode = 2)

•

Memory ...
Read More Here:
Read More Here:
Why a custom message store?
!

http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-a...
Read More Here:
Why a custom message store?
!

http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-a...
Read More Here:
Why a custom message store?
!

http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-a...
Credit Flow
Prevent processes from
overflowing each other’s mailboxes
Credit Specification

{InitialCredit, MoreCreditAfter}
Credit Specification

{200, 50}
Process A sends messages to B
Process will
•

Grant more credit

•

Block other processes

•

Delay granting credits
Read More Here:
http://videlalvaro.github.io/2013/09/rabbitmq-internals-credit-flowfor-erlang-processes.html
RabbitMQ Behaviours
RabbitMQ Behaviours
•

Add a common interface for different components
RabbitMQ Behaviours
•

Add a common interface for different components
•

Exchanges
RabbitMQ Behaviours
•

Add a common interface for different components
•

Exchanges

•

Queues
RabbitMQ Behaviours
•

Add a common interface for different components
•

Exchanges

•

Queues

•

Decorators
RabbitMQ Behaviours
•

Add a common interface for different components
•

Exchanges

•

Queues

•

Decorators

•

Authenti...
RabbitMQ Behaviours
•

Add a common interface for different components
•
•

Queues

•

Decorators

•
•

Exchanges

Authent...
rabbit_exchange_type
-module(rabbit_exchange_type).!
!

-callback description() -> [proplists:property()].!
!

-callback s...
rabbit_exchange_type
•

You can add your own exchange type via plugins
•

consistent hash exchange

•

random exchange

•
...
RabbitMQ Behaviours
•
•
•

rabbit_auth_backend
rabbit_msg_store_index
rabbit_backing_queue

(config)
(config)
(config)
RabbitMQ Behaviours
•
•
•

rabbit_auth_backend
rabbit_msg_store_index
rabbit_backing_queue

(config)
(config)
(config)

[{rab...
RabbitMQ Behaviours
•
•
•
•
•
•
•

rabbit_auth_mechanism
rabbit_exchange_decorator
rabbit_exchange_type
rabbit_mirror_queu...
RabbitMQ Behaviours
•
•
•
•
•
•
•

rabbit_auth_mechanism
rabbit_exchange_decorator
rabbit_exchange_type
rabbit_mirror_queu...
RabbitMQ Federation Plugin
•

Message replication across WANs

•

Uses an exchange decorator

•

Uses a queue decorator

•...
RabbitMQ Federation Plugin
rabbitmqctl set_parameter federation-upstream my-upstream !
'{"uri":"amqp://server-name","expir...
RabbitMQ Federation Plugin
rabbitmqctl set_parameter federation-upstream my-upstream !
'{"uri":"amqp://server-name","expir...
RabbitMQ Federation Plugin
-module(rabbit_federation_queue).!
!

-rabbit_boot_step({?MODULE,!
[{description, "federation q...
RabbitMQ Federation Plugin
-module(rabbit_federation_exchange).!
!

-rabbit_boot_step({?MODULE,!
[{description, "federatio...
Read More Here:
http://www.rabbitmq.com/federation.html
Read More Here:
Making sure the user provided the right behaviour:
!

http://videlalvaro.github.io/2013/09/rabbitmq-intern...
RabbitMQ uses Erlang’s features to
be robust, fault tolerant and very
extensible
Go grab the source code!

http://hg.rabbitmq.com/rabbitmq-server/
Questions?
Thanks
Alvaro Videla - @old_sound
Upcoming SlideShare
Loading in...5
×

Dissecting the rabbit: RabbitMQ Internal Architecture

13,030

Published on

Talk given at HighLoad++ in Moscow, 2013.

Introduces several Erlang patterns and techniques used by RabbitMQ

Published in: Technology, News & Politics
0 Comments
26 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
13,030
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
265
Comments
0
Likes
26
Embeds 0
No embeds

No notes for slide

Dissecting the rabbit: RabbitMQ Internal Architecture

  1. 1. Dissecting the Rabbit: RabbitMQ Internal Architecture Alvaro Videla - RabbitMQ
  2. 2. Alvaro Videla • • • • • Developer Advocate at Pivotal / RabbitMQ! Co-Author of RabbitMQ in Action! Creator of the RabbitMQ Simulator! Blogs about RabbitMQ Internals: http://videlalvaro.github.io/internals.html! @old_sound | alvaro@rabbitmq.com
 github.com/videlalvaro

  3. 3. About Me Co-authored! ! RabbitMQ in Action! http://bit.ly/rabbitmq
  4. 4. Agenda • Intro to RabbitMQ • Dive into RabbitMQ Internals • A day in the life of a message • RabbitMQ message store • RabbitMQ behaviours and extensibility
  5. 5. What is RabbitMQ
  6. 6. RabbitMQ
  7. 7. RabbitMQ
  8. 8. RabbitMQ • Multi Protocol Messaging Server
  9. 9. RabbitMQ • Multi Protocol Messaging Server! • Open Source (MPL)
  10. 10. RabbitMQ • Multi Protocol Messaging Server! • Open Source (MPL)! • Polyglot
  11. 11. RabbitMQ • Multi Protocol Messaging Server! • Open Source (MPL)! • Polyglot! • Written in Erlang/OTP
  12. 12. Multi Protocol http://bit.ly/rmq-protocols
  13. 13. Polyglot
  14. 14. Polyglot
  15. 15. Polyglot • Java
  16. 16. Polyglot • Java! • node.js
  17. 17. Polyglot • Java! • node.js! • Erlang
  18. 18. Polyglot • Java! • node.js! • Erlang! • PHP
  19. 19. Polyglot • Java! • node.js! • Erlang! • PHP! • Ruby
  20. 20. Polyglot • Java! • node.js! • Erlang! • PHP! • Ruby! • .Net
  21. 21. Polyglot • Java! • node.js! • Erlang! • PHP! • Ruby! • .Net! • Haskell
  22. 22. Polyglot Even COBOL!!!11
  23. 23. Some users of RabbitMQ
  24. 24. Some users of RabbitMQ • Instagram
  25. 25. Some users of RabbitMQ • • Instagram! Indeed.com
  26. 26. Some users of RabbitMQ • • • Instagram! Indeed.com! MailboxApp
  27. 27. Some users of RabbitMQ • • • • Instagram! Indeed.com! MailboxApp! Mercado Libre
  28. 28. Some users of RabbitMQ • • • • • Instagram! Indeed.com! MailboxApp! Mercado Libre! NHS
  29. 29. Some users of RabbitMQ • • • • • • Instagram! Indeed.com! MailboxApp! Mercado Libre! NHS! Mozilla
  30. 30. http://www.rabbitmq.com/download.html Unix - Mac - Windows
  31. 31. Messaging with RabbitMQ A demo with the RabbitMQ Simulator https://github.com/RabbitMQSimulator/RabbitMQSimulator
  32. 32. http://tryrabbitmq.com
  33. 33. RabbitMQ Simulator
  34. 34. RabbitMQ Internals
  35. 35. A day in the life of a message
  36. 36. A day in the life of a message
  37. 37. A day in the life of a message $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);! $ch = $conn->channel();! ! $ch->queue_declare($queue, false, true, false, false);! ! $ch->exchange_declare($exchange, 'direct', false, true, false);! ! $ch->queue_bind($queue, $exchange);! ! $msg_body = implode(' ', array_slice($argv, 1));! $msg = new AMQPMessage($msg_body, array('delivery_mode' => 2));! ! $ch->basic_publish($msg, $exchange);
  38. 38. What happens here? $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);! $ch = $conn->channel();
  39. 39. What happens here?
  40. 40. Erlang
  41. 41. Erlang App • Processes (probably thousands) • They communicate sending messages • Each process has a message queue (don’t confuse with RabbitMQ queues) • Virtual Machine has preemptive scheduler
  42. 42. Read More Here: http://jlouisramblings.blogspot.ru/2013/01/how-erlang-does-scheduling.html
  43. 43. Erlang code structure • Modules • Functions • Function Arity • Arguments M, F, A = Module, Function, Arguments
  44. 44. rabbit_client_sup.erl -module(rabbit_client_sup).! ! -behaviour(supervisor2).! ! -export([start_link/1, start_link/2, start_link_worker/2]).! ! -export([init/1]).! ! -include("rabbit.hrl").
  45. 45. rabbit_client_sup.erl start_link(Callback) ->! supervisor2:start_link(?MODULE, Callback).! ! start_link(SupName, Callback) ->! supervisor2:start_link(SupName, ?MODULE, Callback).! ! start_link_worker(SupName, Callback) ->! supervisor2:start_link(SupName, ?MODULE, {Callback, worker}).! ! init({M,F,A}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, infinity, supervisor, [M]}]}};! init({{M,F,A}, worker}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, ?MAX_WAIT, worker, [M]}]}}.
  46. 46. Supervision Trees
  47. 47. Supervision tree • Worker Processes • Supervisor Processes • Supervision tree as a hierarchical arrangement of processes http://www.erlang.org/doc/man/supervisor.html
  48. 48. rabbit_client_sup init({M,F,A}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, ! ! ! ! ! infinity, supervisor, [M]}]}};
  49. 49. Child Spec - restart strategies • one_for_one: only restart failing process • one_for_all: restart failing process and all siblings • simple_one_for_one: simplified version of one_for_one • MaxR: maximum restarts allowed • MaxT: in MaxT seconds
  50. 50. Child Specification child_spec() = {Id,StartFunc,Restart,Shutdown,Type,Modules}! Id = term()! StartFunc = {M,F,A}! M = F = atom()! A = [term()]! Restart = permanent | transient | temporary! Shutdown = brutal_kill | int()>0 | infinity! Type = worker | supervisor! Modules = [Module] | dynamic! Module = atom()
  51. 51. rabbit_client_sup init({M,F,A}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, ! ! ! ! ! infinity, supervisor, [M]}]}};
  52. 52. Child Spec - restart • permanent: should always be restarted • temporary: should never be restarted • transient: should only be restarted if terminated abnormally
  53. 53. Child Spec - shutdown • brutal_kill: child is terminated immediately. • timeout in seconds: supervisor waits for timeout before terminating children. • infinity: give enough timeout to children to shutdown its own supervision tree.
  54. 54. Connection Supervision Tree
  55. 55. Connection Supervision Tree
  56. 56. Connection Supervision Tree
  57. 57. Connection Supervision Tree
  58. 58. Connection Supervision Tree
  59. 59. A day in the life of a message
  60. 60. A day in the life of a message
  61. 61. A day in the life of a message
  62. 62. Intermezzo: pattern matching check_user_id_header(#'P_basic'{user_id = undefined}, _) ->! ok;! check_user_id_header(#'P_basic'{user_id = Username},! #ch{user = #user{username = Username}}) ->! ok;! check_user_id_header(#'P_basic'{user_id = Claimed},! #ch{user = #user{username = Actual,! tags = Tags}}) ->! case lists:member(impersonator, Tags) of! true -> ok;! false -> precondition_failed(! "user_id property set to '~s' but authenticated user was "! "'~s'", [Claimed, Actual])! end. http://videlalvaro.github.io/2013/09/rabbitmq-validating-user-ids-with-erlang-pattern-matching.html
  63. 63. Read More Here: http://videlalvaro.github.io/2013/09/rabbitmq-validating-user-ids-with-erlang-pattern-matching.html
  64. 64. A day in the life of a message
  65. 65. A day in the life of a message
  66. 66. A day in the life of a message We have a list of queues where the channel will deliver the messages
  67. 67. A day in the life of a message
  68. 68. A day in the life of a message If consumer ready
  69. 69. RabbitMQ Message Store • Made specifically for messaging • Keeps messages in memory and (sometimes) on disk • Bounded by disk size • per node message store (transient, persistent) • per queue “queue index”
  70. 70. RabbitMQ Message Store • Message written to disk when: • Message published as persistent (delivery_mode = 2) • Memory pressure
  71. 71. Read More Here:
  72. 72. Read More Here: Why a custom message store? ! http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/
  73. 73. Read More Here: Why a custom message store? ! http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/ How the message store compacts files ! http://hg.rabbitmq.com/rabbitmq-server/file/56d190fd4ea3/src/rabbit_msg_store.erl#l181
  74. 74. Read More Here: Why a custom message store? ! http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/ How the message store compacts files ! http://hg.rabbitmq.com/rabbitmq-server/file/56d190fd4ea3/src/rabbit_msg_store.erl#l181 How the message store reacts to memory pressure ! http://hg.rabbitmq.com/rabbitmq-server/file/56d190fd4ea3/src/rabbit_variable_queue.erl#l35
  75. 75. Credit Flow
  76. 76. Prevent processes from overflowing each other’s mailboxes
  77. 77. Credit Specification {InitialCredit, MoreCreditAfter}
  78. 78. Credit Specification {200, 50}
  79. 79. Process A sends messages to B
  80. 80. Process will • Grant more credit • Block other processes • Delay granting credits
  81. 81. Read More Here: http://videlalvaro.github.io/2013/09/rabbitmq-internals-credit-flowfor-erlang-processes.html
  82. 82. RabbitMQ Behaviours
  83. 83. RabbitMQ Behaviours • Add a common interface for different components
  84. 84. RabbitMQ Behaviours • Add a common interface for different components • Exchanges
  85. 85. RabbitMQ Behaviours • Add a common interface for different components • Exchanges • Queues
  86. 86. RabbitMQ Behaviours • Add a common interface for different components • Exchanges • Queues • Decorators
  87. 87. RabbitMQ Behaviours • Add a common interface for different components • Exchanges • Queues • Decorators • Authentication methods
  88. 88. RabbitMQ Behaviours • Add a common interface for different components • • Queues • Decorators • • Exchanges Authentication methods Add extensibility
  89. 89. rabbit_exchange_type -module(rabbit_exchange_type).! ! -callback description() -> [proplists:property()].! ! -callback serialise_events() -> boolean().! ! -callback route(rabbit_types:exchange(), rabbit_types:delivery()) ->! rabbit_router:match_result().! ! -callback validate(rabbit_types:exchange()) -> 'ok'.! ! -callback validate_binding(rabbit_types:exchange(), rabbit_types:binding()) ->! rabbit_types:ok_or_error({'binding_invalid', string(), [any()]}).
  90. 90. rabbit_exchange_type • You can add your own exchange type via plugins • consistent hash exchange • random exchange • recent history exchange • riak exchange
  91. 91. RabbitMQ Behaviours • • • rabbit_auth_backend rabbit_msg_store_index rabbit_backing_queue (config) (config) (config)
  92. 92. RabbitMQ Behaviours • • • rabbit_auth_backend rabbit_msg_store_index rabbit_backing_queue (config) (config) (config) [{rabbit,! [{auth_backends, [rabbit_auth_backend_http, ! rabbit_auth_backend_internal]}]! }].
  93. 93. RabbitMQ Behaviours • • • • • • • rabbit_auth_mechanism rabbit_exchange_decorator rabbit_exchange_type rabbit_mirror_queue_mode rabbit_policy_validator rabbit_queue_decorator rabbit_runtime_parameter (registry) (registry) (registry) (registry) (registry) (registry) (registry)
  94. 94. RabbitMQ Behaviours • • • • • • • rabbit_auth_mechanism rabbit_exchange_decorator rabbit_exchange_type rabbit_mirror_queue_mode rabbit_policy_validator rabbit_queue_decorator rabbit_runtime_parameter (registry) (registry) (registry) (registry) (registry) (registry) (registry) -rabbit_boot_step({?MODULE,! [{description, "exchange type direct"},! {mfa, {rabbit_registry, register,! [exchange, <<"direct">>, ?MODULE]}},! {requires, rabbit_registry},! {enables, kernel_ready}]}).
  95. 95. RabbitMQ Federation Plugin • Message replication across WANs • Uses an exchange decorator • Uses a queue decorator • Uses parameters • Uses policies
  96. 96. RabbitMQ Federation Plugin rabbitmqctl set_parameter federation-upstream my-upstream ! '{"uri":"amqp://server-name","expires":3600000}'
  97. 97. RabbitMQ Federation Plugin rabbitmqctl set_parameter federation-upstream my-upstream ! '{"uri":"amqp://server-name","expires":3600000}' rabbitmqctl set_policy federate-me "^amq." '{"federation-upstreamset":"all"}'
  98. 98. RabbitMQ Federation Plugin -module(rabbit_federation_queue).! ! -rabbit_boot_step({?MODULE,! [{description, "federation queue decorator"},! {mfa, {rabbit_registry, register,! [queue_decorator, <<"federation">>, ?MODULE]}},! {requires, rabbit_registry},! {enables, recovery}]}).! ! -behaviour(rabbit_queue_decorator).
  99. 99. RabbitMQ Federation Plugin -module(rabbit_federation_exchange).! ! -rabbit_boot_step({?MODULE,! [{description, "federation exchange decorator"},! {mfa, {rabbit_registry, register,! [exchange_decorator, <<"federation">>, ?MODULE]}},! {requires, rabbit_registry},! {enables, recovery}]}).! ! -behaviour(rabbit_exchange_decorator).
  100. 100. Read More Here: http://www.rabbitmq.com/federation.html
  101. 101. Read More Here: Making sure the user provided the right behaviour: ! http://videlalvaro.github.io/2013/09/rabbitmq-internals-validating-erlang-behaviours.html How RabbitMQ prevents arbitrary code execution: ! http://videlalvaro.github.io/2013/09/rabbitmq-sanitzing-user-input-in-erlang.html RabbitMQ Boot Step System: ! https://github.com/videlalvaro/rabbit-internals/blob/master/rabbit_boot_process.md
  102. 102. RabbitMQ uses Erlang’s features to be robust, fault tolerant and very extensible
  103. 103. Go grab the source code! http://hg.rabbitmq.com/rabbitmq-server/
  104. 104. Questions?
  105. 105. Thanks Alvaro Videla - @old_sound
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×