Dissecting the rabbit: RabbitMQ Internal Architecture

  • 10,358 views
Uploaded on

Talk given at HighLoad++ in Moscow, 2013. …

Talk given at HighLoad++ in Moscow, 2013.

Introduces several Erlang patterns and techniques used by RabbitMQ

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
10,358
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
167
Comments
0
Likes
13

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Dissecting the Rabbit: RabbitMQ Internal Architecture Alvaro Videla - RabbitMQ
  • 2. Alvaro Videla • • • • • Developer Advocate at Pivotal / RabbitMQ! Co-Author of RabbitMQ in Action! Creator of the RabbitMQ Simulator! Blogs about RabbitMQ Internals: http://videlalvaro.github.io/internals.html! @old_sound | alvaro@rabbitmq.com
 github.com/videlalvaro

  • 3. About Me Co-authored! ! RabbitMQ in Action! http://bit.ly/rabbitmq
  • 4. Agenda • Intro to RabbitMQ • Dive into RabbitMQ Internals • A day in the life of a message • RabbitMQ message store • RabbitMQ behaviours and extensibility
  • 5. What is RabbitMQ
  • 6. RabbitMQ
  • 7. RabbitMQ
  • 8. RabbitMQ • Multi Protocol Messaging Server
  • 9. RabbitMQ • Multi Protocol Messaging Server! • Open Source (MPL)
  • 10. RabbitMQ • Multi Protocol Messaging Server! • Open Source (MPL)! • Polyglot
  • 11. RabbitMQ • Multi Protocol Messaging Server! • Open Source (MPL)! • Polyglot! • Written in Erlang/OTP
  • 12. Multi Protocol http://bit.ly/rmq-protocols
  • 13. Polyglot
  • 14. Polyglot
  • 15. Polyglot • Java
  • 16. Polyglot • Java! • node.js
  • 17. Polyglot • Java! • node.js! • Erlang
  • 18. Polyglot • Java! • node.js! • Erlang! • PHP
  • 19. Polyglot • Java! • node.js! • Erlang! • PHP! • Ruby
  • 20. Polyglot • Java! • node.js! • Erlang! • PHP! • Ruby! • .Net
  • 21. Polyglot • Java! • node.js! • Erlang! • PHP! • Ruby! • .Net! • Haskell
  • 22. Polyglot Even COBOL!!!11
  • 23. Some users of RabbitMQ
  • 24. Some users of RabbitMQ • Instagram
  • 25. Some users of RabbitMQ • • Instagram! Indeed.com
  • 26. Some users of RabbitMQ • • • Instagram! Indeed.com! MailboxApp
  • 27. Some users of RabbitMQ • • • • Instagram! Indeed.com! MailboxApp! Mercado Libre
  • 28. Some users of RabbitMQ • • • • • Instagram! Indeed.com! MailboxApp! Mercado Libre! NHS
  • 29. Some users of RabbitMQ • • • • • • Instagram! Indeed.com! MailboxApp! Mercado Libre! NHS! Mozilla
  • 30. http://www.rabbitmq.com/download.html Unix - Mac - Windows
  • 31. Messaging with RabbitMQ A demo with the RabbitMQ Simulator https://github.com/RabbitMQSimulator/RabbitMQSimulator
  • 32. http://tryrabbitmq.com
  • 33. RabbitMQ Simulator
  • 34. RabbitMQ Internals
  • 35. A day in the life of a message
  • 36. A day in the life of a message
  • 37. A day in the life of a message $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);! $ch = $conn->channel();! ! $ch->queue_declare($queue, false, true, false, false);! ! $ch->exchange_declare($exchange, 'direct', false, true, false);! ! $ch->queue_bind($queue, $exchange);! ! $msg_body = implode(' ', array_slice($argv, 1));! $msg = new AMQPMessage($msg_body, array('delivery_mode' => 2));! ! $ch->basic_publish($msg, $exchange);
  • 38. What happens here? $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);! $ch = $conn->channel();
  • 39. What happens here?
  • 40. Erlang
  • 41. Erlang App • Processes (probably thousands) • They communicate sending messages • Each process has a message queue (don’t confuse with RabbitMQ queues) • Virtual Machine has preemptive scheduler
  • 42. Read More Here: http://jlouisramblings.blogspot.ru/2013/01/how-erlang-does-scheduling.html
  • 43. Erlang code structure • Modules • Functions • Function Arity • Arguments M, F, A = Module, Function, Arguments
  • 44. rabbit_client_sup.erl -module(rabbit_client_sup).! ! -behaviour(supervisor2).! ! -export([start_link/1, start_link/2, start_link_worker/2]).! ! -export([init/1]).! ! -include("rabbit.hrl").
  • 45. rabbit_client_sup.erl start_link(Callback) ->! supervisor2:start_link(?MODULE, Callback).! ! start_link(SupName, Callback) ->! supervisor2:start_link(SupName, ?MODULE, Callback).! ! start_link_worker(SupName, Callback) ->! supervisor2:start_link(SupName, ?MODULE, {Callback, worker}).! ! init({M,F,A}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, infinity, supervisor, [M]}]}};! init({{M,F,A}, worker}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, ?MAX_WAIT, worker, [M]}]}}.
  • 46. Supervision Trees
  • 47. Supervision tree • Worker Processes • Supervisor Processes • Supervision tree as a hierarchical arrangement of processes http://www.erlang.org/doc/man/supervisor.html
  • 48. rabbit_client_sup init({M,F,A}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, ! ! ! ! ! infinity, supervisor, [M]}]}};
  • 49. Child Spec - restart strategies • one_for_one: only restart failing process • one_for_all: restart failing process and all siblings • simple_one_for_one: simplified version of one_for_one • MaxR: maximum restarts allowed • MaxT: in MaxT seconds
  • 50. Child Specification child_spec() = {Id,StartFunc,Restart,Shutdown,Type,Modules}! Id = term()! StartFunc = {M,F,A}! M = F = atom()! A = [term()]! Restart = permanent | transient | temporary! Shutdown = brutal_kill | int()>0 | infinity! Type = worker | supervisor! Modules = [Module] | dynamic! Module = atom()
  • 51. rabbit_client_sup init({M,F,A}) ->! {ok, {{simple_one_for_one, 0, 1},! [{client, {M,F,A}, temporary, ! ! ! ! ! infinity, supervisor, [M]}]}};
  • 52. Child Spec - restart • permanent: should always be restarted • temporary: should never be restarted • transient: should only be restarted if terminated abnormally
  • 53. Child Spec - shutdown • brutal_kill: child is terminated immediately. • timeout in seconds: supervisor waits for timeout before terminating children. • infinity: give enough timeout to children to shutdown its own supervision tree.
  • 54. Connection Supervision Tree
  • 55. Connection Supervision Tree
  • 56. Connection Supervision Tree
  • 57. Connection Supervision Tree
  • 58. Connection Supervision Tree
  • 59. A day in the life of a message
  • 60. A day in the life of a message
  • 61. A day in the life of a message
  • 62. Intermezzo: pattern matching check_user_id_header(#'P_basic'{user_id = undefined}, _) ->! ok;! check_user_id_header(#'P_basic'{user_id = Username},! #ch{user = #user{username = Username}}) ->! ok;! check_user_id_header(#'P_basic'{user_id = Claimed},! #ch{user = #user{username = Actual,! tags = Tags}}) ->! case lists:member(impersonator, Tags) of! true -> ok;! false -> precondition_failed(! "user_id property set to '~s' but authenticated user was "! "'~s'", [Claimed, Actual])! end. http://videlalvaro.github.io/2013/09/rabbitmq-validating-user-ids-with-erlang-pattern-matching.html
  • 63. Read More Here: http://videlalvaro.github.io/2013/09/rabbitmq-validating-user-ids-with-erlang-pattern-matching.html
  • 64. A day in the life of a message
  • 65. A day in the life of a message
  • 66. A day in the life of a message We have a list of queues where the channel will deliver the messages
  • 67. A day in the life of a message
  • 68. A day in the life of a message If consumer ready
  • 69. RabbitMQ Message Store • Made specifically for messaging • Keeps messages in memory and (sometimes) on disk • Bounded by disk size • per node message store (transient, persistent) • per queue “queue index”
  • 70. RabbitMQ Message Store • Message written to disk when: • Message published as persistent (delivery_mode = 2) • Memory pressure
  • 71. Read More Here:
  • 72. Read More Here: Why a custom message store? ! http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/
  • 73. Read More Here: Why a custom message store? ! http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/ How the message store compacts files ! http://hg.rabbitmq.com/rabbitmq-server/file/56d190fd4ea3/src/rabbit_msg_store.erl#l181
  • 74. Read More Here: Why a custom message store? ! http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/ How the message store compacts files ! http://hg.rabbitmq.com/rabbitmq-server/file/56d190fd4ea3/src/rabbit_msg_store.erl#l181 How the message store reacts to memory pressure ! http://hg.rabbitmq.com/rabbitmq-server/file/56d190fd4ea3/src/rabbit_variable_queue.erl#l35
  • 75. Credit Flow
  • 76. Prevent processes from overflowing each other’s mailboxes
  • 77. Credit Specification {InitialCredit, MoreCreditAfter}
  • 78. Credit Specification {200, 50}
  • 79. Process A sends messages to B
  • 80. Process will • Grant more credit • Block other processes • Delay granting credits
  • 81. Read More Here: http://videlalvaro.github.io/2013/09/rabbitmq-internals-credit-flowfor-erlang-processes.html
  • 82. RabbitMQ Behaviours
  • 83. RabbitMQ Behaviours • Add a common interface for different components
  • 84. RabbitMQ Behaviours • Add a common interface for different components • Exchanges
  • 85. RabbitMQ Behaviours • Add a common interface for different components • Exchanges • Queues
  • 86. RabbitMQ Behaviours • Add a common interface for different components • Exchanges • Queues • Decorators
  • 87. RabbitMQ Behaviours • Add a common interface for different components • Exchanges • Queues • Decorators • Authentication methods
  • 88. RabbitMQ Behaviours • Add a common interface for different components • • Queues • Decorators • • Exchanges Authentication methods Add extensibility
  • 89. rabbit_exchange_type -module(rabbit_exchange_type).! ! -callback description() -> [proplists:property()].! ! -callback serialise_events() -> boolean().! ! -callback route(rabbit_types:exchange(), rabbit_types:delivery()) ->! rabbit_router:match_result().! ! -callback validate(rabbit_types:exchange()) -> 'ok'.! ! -callback validate_binding(rabbit_types:exchange(), rabbit_types:binding()) ->! rabbit_types:ok_or_error({'binding_invalid', string(), [any()]}).
  • 90. rabbit_exchange_type • You can add your own exchange type via plugins • consistent hash exchange • random exchange • recent history exchange • riak exchange
  • 91. RabbitMQ Behaviours • • • rabbit_auth_backend rabbit_msg_store_index rabbit_backing_queue (config) (config) (config)
  • 92. RabbitMQ Behaviours • • • rabbit_auth_backend rabbit_msg_store_index rabbit_backing_queue (config) (config) (config) [{rabbit,! [{auth_backends, [rabbit_auth_backend_http, ! rabbit_auth_backend_internal]}]! }].
  • 93. RabbitMQ Behaviours • • • • • • • rabbit_auth_mechanism rabbit_exchange_decorator rabbit_exchange_type rabbit_mirror_queue_mode rabbit_policy_validator rabbit_queue_decorator rabbit_runtime_parameter (registry) (registry) (registry) (registry) (registry) (registry) (registry)
  • 94. RabbitMQ Behaviours • • • • • • • rabbit_auth_mechanism rabbit_exchange_decorator rabbit_exchange_type rabbit_mirror_queue_mode rabbit_policy_validator rabbit_queue_decorator rabbit_runtime_parameter (registry) (registry) (registry) (registry) (registry) (registry) (registry) -rabbit_boot_step({?MODULE,! [{description, "exchange type direct"},! {mfa, {rabbit_registry, register,! [exchange, <<"direct">>, ?MODULE]}},! {requires, rabbit_registry},! {enables, kernel_ready}]}).
  • 95. RabbitMQ Federation Plugin • Message replication across WANs • Uses an exchange decorator • Uses a queue decorator • Uses parameters • Uses policies
  • 96. RabbitMQ Federation Plugin rabbitmqctl set_parameter federation-upstream my-upstream ! '{"uri":"amqp://server-name","expires":3600000}'
  • 97. RabbitMQ Federation Plugin rabbitmqctl set_parameter federation-upstream my-upstream ! '{"uri":"amqp://server-name","expires":3600000}' rabbitmqctl set_policy federate-me "^amq." '{"federation-upstreamset":"all"}'
  • 98. RabbitMQ Federation Plugin -module(rabbit_federation_queue).! ! -rabbit_boot_step({?MODULE,! [{description, "federation queue decorator"},! {mfa, {rabbit_registry, register,! [queue_decorator, <<"federation">>, ?MODULE]}},! {requires, rabbit_registry},! {enables, recovery}]}).! ! -behaviour(rabbit_queue_decorator).
  • 99. RabbitMQ Federation Plugin -module(rabbit_federation_exchange).! ! -rabbit_boot_step({?MODULE,! [{description, "federation exchange decorator"},! {mfa, {rabbit_registry, register,! [exchange_decorator, <<"federation">>, ?MODULE]}},! {requires, rabbit_registry},! {enables, recovery}]}).! ! -behaviour(rabbit_exchange_decorator).
  • 100. Read More Here: http://www.rabbitmq.com/federation.html
  • 101. Read More Here: Making sure the user provided the right behaviour: ! http://videlalvaro.github.io/2013/09/rabbitmq-internals-validating-erlang-behaviours.html How RabbitMQ prevents arbitrary code execution: ! http://videlalvaro.github.io/2013/09/rabbitmq-sanitzing-user-input-in-erlang.html RabbitMQ Boot Step System: ! https://github.com/videlalvaro/rabbit-internals/blob/master/rabbit_boot_process.md
  • 102. RabbitMQ uses Erlang’s features to be robust, fault tolerant and very extensible
  • 103. Go grab the source code! http://hg.rabbitmq.com/rabbitmq-server/
  • 104. Questions?
  • 105. Thanks Alvaro Videla - @old_sound