This document contains a summary of a presentation about using RabbitMQ for messaging. It discusses RabbitMQ's features like multi-protocol support, polyglot clients, and large users. It then describes challenges like distributing an application across servers. RabbitMQ solutions to these include federation to replicate across servers, sharding queues for scaling, and federating queues for load balancing consumers. The presentation argues RabbitMQ provides tools to ingest, distribute, and scale messaging applications globally.

2. Alvaro Videla
• Developer Advocate at Pivotal / RabbitMQ!
• Co-Author of RabbitMQ in Action!
• Creator of the RabbitMQ Simulator!
• Blogs about RabbitMQ Internals: http://videlalvaro.github.io/internals.html!
• @old_sound — alvaro@rabbitmq.com — github.com/videlalvaro

3. About Me
Co-authored!
!
RabbitMQ in Action!
http://bit.ly/rabbitmq

4. About this Talk
• Exploratory Talk
• A ‘what could be done’ talk instead of ‘this is how you do it’

5. Agenda
• Intro to RabbitMQ
• The Problem
• Solution Proposal
• Improvements

6. https://twitter.com/spacemanaki/status/514590885523505153

7. What is RabbitMQ

8. RabbitMQ

9. RabbitMQ

10. RabbitMQ
• Multi Protocol Messaging Server

11. RabbitMQ
• Multi Protocol Messaging Server!
• Open Source (MPL)

12. RabbitMQ
• Multi Protocol Messaging Server!
• Open Source (MPL)!
• Polyglot

13. RabbitMQ
• Multi Protocol Messaging Server!
• Open Source (MPL)!
• Polyglot!
• Written in Erlang/OTP

14. Multi Protocol
http://bit.ly/rmq-protocols

15. Community Plugins
http://www.rabbitmq.com/community-plugins.html

16. Polyglot

17. Polyglot

18. Polyglot
• Java

19. Polyglot
• Java!
• node.js

20. Polyglot
• Java!
• node.js!
• Erlang

21. Polyglot
• Java!
• node.js!
• Erlang!
• PHP

22. Polyglot
• Java!
• node.js!
• Erlang!
• PHP!
• Ruby

23. Polyglot
• Java!
• node.js!
• Erlang!
• PHP!
• Ruby!
• .Net

24. Polyglot
• Java!
• node.js!
• Erlang!
• PHP!
• Ruby!
• .Net!
• Haskell!
• Python

25. Polyglot
Even COBOL!!!11

26. Some users of RabbitMQ

27. Some users of RabbitMQ
• Instagram

28. Some users of RabbitMQ
• Instagram!
• Indeed.com

29. Some users of RabbitMQ
• Instagram!
• Indeed.com!
• Telefonica

30. Some users of RabbitMQ
• Instagram!
• Indeed.com!
• Telefonica!
• Mercado Libre

31. Some users of RabbitMQ
• Instagram!
• Indeed.com!
• Telefonica!
• Mercado Libre!
• NHS

32. Some users of RabbitMQ
• Instagram!
• Indeed.com!
• Telefonica!
• Mercado Libre!
• NHS!
• Mozilla

33. The New York Times on RabbitMQ
This architecture - Fabrik - has dozens of RabbitMQ instances
spread across 6 AWS zones in Oregon and Dublin.
Upon launch today, the system autoscaled to ~500,000 users.
Connection times remained flat at ~200ms.
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2014-January/032943.html

34. http://www.rabbitmq.com/download.html
Unix - Mac - Windows

35. Messaging with RabbitMQ
A demo with the RabbitMQ Simulator
https://github.com/RabbitMQSimulator/RabbitMQSimulator

36. http://tryrabbitmq.com

37. RabbitMQ Simulator

38. The Problem

39. Distributed Application
App
App
App
App

40. Distributed Application
App
App
App
App

41. {ok, Connection} =
Data Producer
Obtain a Channel
amqp_connection:start(#amqp_params_network{host = "localhost"}),
!
{ok, Channel} = amqp_connection:open_channel(Connection),

42. Data Producer
Declare an Exchange
amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"events">>,
type = <<"direct">>}),

43. Data Producer
amqp_channel:cast(Channel,
Publish a message
#'basic.publish'{
exchange = <<"events">>},
#amqp_msg{props = #'P_basic'{delivery_mode = 2},
payload = <<“Hello Federation">>}),

44. Data Consumer
Obtain a Channel
{ok, Connection} =
amqp_connection:start(#amqp_params_network{host = "localhost"}),
!
{ok, Channel} = amqp_connection:open_channel(Connection),

45. Data Consumer
Declare Queue and bind it
amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"events">>,
type = <<"direct">>}),
!
#'queue.declare_ok'{queue = Queue} =
amqp_channel:call(Channel, #'queue.declare'{exclusive = true}),
!
amqp_channel:call(Channel, #'queue.bind'{exchange = <<"events">>,
queue = Queue}),

46. amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue,
no_ack = true}, self()),
!
receive
#'basic.consume_ok'{} -> ok
end,
!
loop(Channel).
Data Consumer
Start a consumer

47. loop(Channel) ->
receive
{#'basic.deliver'{}, #amqp_msg{payload = Body}} ->
io:format(" [x] ~p~n", [Body]),
loop(Channel)
end.
Data Consumer
Process messages

48. Ad-hoc solution

49. A process that replicates data
to the remote server

50. Possible issues

51. Possible issues
• Remote server is offline

52. Possible issues
• Remote server is offline
• Prevent unbounded local buffers

53. Possible issues
• Remote server is offline
• Prevent unbounded local buffers
• Prevent message loss

54. Possible issues
• Remote server is offline
• Prevent unbounded local buffers
• Prevent message loss
• Prevent unnecessary message replication

55. Possible issues
• Remote server is offline
• Prevent unbounded local buffers
• Prevent message loss
• Prevent unnecessary message replication
• No need for those messages on remote server

56. Possible issues
• Remote server is offline
• Prevent unbounded local buffers
• Prevent message loss
• Prevent unnecessary message replication
• No need for those messages on remote server
• Messages that became stale

57. Can we do better?

58. RabbitMQ Federation

59. RabbitMQ Federation

60. RabbitMQ Federation
• Supports replication across different administrative domains

61. RabbitMQ Federation
• Supports replication across different administrative domains
• Supports mix of Erlang and RabbitMQ versions

62. RabbitMQ Federation
• Supports replication across different administrative domains
• Supports mix of Erlang and RabbitMQ versions
• Supports Network Partitions

63. RabbitMQ Federation
• Supports replication across different administrative domains
• Supports mix of Erlang and RabbitMQ versions
• Supports Network Partitions
• Specificity - not everything has to be federated

64. RabbitMQ Federation

65. RabbitMQ Federation

66. RabbitMQ Federation

67. RabbitMQ Federation
• It’s a RabbitMQ Plugin

68. RabbitMQ Federation
• It’s a RabbitMQ Plugin
• Internally uses Queues and Exchanges Decorators

69. RabbitMQ Federation
• It’s a RabbitMQ Plugin
• Internally uses Queues and Exchanges Decorators
• Managed using Parameters and Policies

70. Enabling the Plugin
rabbitmq-plugins enable rabbitmq_federation

71. Enabling the Plugin
rabbitmq-plugins enable rabbitmq_federation
rabbitmq-plugins enable rabbitmq_federation_management

72. Federating an Exchange
rabbitmqctl set_parameter federation-upstream my-upstream
‘{“uri":"amqp://server-name","expires":3600000}'

73. Federating an Exchange
rabbitmqctl set_parameter federation-upstream my-upstream
‘{“uri":"amqp://server-name","expires":3600000}'
!
rabbitmqctl set_policy --apply-to exchanges federate-me "^amq."
'{"federation-upstream-set":"all"}'

74. Federating an Exchange

75. Configuring Federation

76. Config Options
rabbitmqctl set_parameter federation-upstream
name ‘json-object’

77. Config Options
rabbitmqctl set_parameter federation-upstream
name ‘json-object’
!
json-object: {
‘uri’: ‘amqp://server-name/’,
‘prefetch-count’: 1000,
‘reconnect-delay’: 1,
‘ack-mode’: on-confirm
}
http://www.rabbitmq.com/federation-reference.html

78. Prevent unbound buffers
expires: N // ms.
message-ttl: N // ms.

79. Prevent message forwarding
max-hops: N

80. Speed vs No Message Loss
ack-mode: on-confirm
ack-mode: on-publish
ack-mode: no-ack

81. AMQP URI:
amqp://user:pass@host:10000/vhost
http://www.rabbitmq.com/uri-spec.html

82. Config can be applied via
• CLI using rabbitmqctl
• HTTP API
• RabbitMQ Management Interface

83. RabbitMQ Federation

84. Some Queueing Theory
http://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/

85. RabbitMQ BasicQos Simulator

86. Prevent Unbound Buffers
λ = mean arrival time
μ = mean service rate
if λ > μ what happens?
https://www.rabbitmq.com/blog/2014/01/23/preventing-unbounded-buffers-with-rabbitmq/

87. Prevent Unbound Buffers
λ = mean arrival time
μ = mean service rate
if λ > μ what happens?
Queue length goes to infinity over time.
https://www.rabbitmq.com/blog/2014/01/23/preventing-unbounded-buffers-with-rabbitmq/

88. Recommended Reading
Performance Modeling and
Design of Computer Systems:
Queueing Theory in Action

89. Scaling the Setup

90. The Problem

91. The Problem
• Queues contents live in the node where the Queue was declared

92. The Problem
• Queues contents live in the node where the Queue was declared
• A cluster can access the queue from every connected node

93. The Problem
• Queues contents live in the node where the Queue was declared
• A cluster can access the queue from every connected node
• Queues are an Erlang process (tied to one core)

94. The Problem
• Queues contents live in the node where the Queue was declared
• A cluster can access the queue from every connected node
• Queues are an Erlang process (tied to one core)
• Adding more nodes doesn’t really help

95. Enter Sharded Queues

96. Enter Sharded Queues

97. Pieces of the Puzzle
• modulo hash exchange (consistent hash works as well)
• good ol’ queues

98. Sharded Queues

99. Sharded Queues

100. Sharded Queues

101. Sharded Queues

102. Sharded Queues
• Declare Queues with name: nodename.queuename.index

103. Sharded Queues
• Declare Queues with name: nodename.queuename.index
• Bind the queues to a consistent hash exchange

104. Sharded Queues
• Declare Queues with name: nodename.queuename.index
• Bind the queues to a partitioner exchange
• Transparent to the consumer (virtual queue name)

105. We need more scale!

106. Federated Queues

107. Federated Queues
• Load-balance messages across federated queues
• Only moves messages when needed

108. Federating a Queue
rabbitmqctl set_parameter federation-upstream my-upstream
‘{“uri":"amqp://server-name","expires":3600000}'

109. Federating a Queue
rabbitmqctl set_parameter federation-upstream my-upstream
‘{“uri":"amqp://server-name","expires":3600000}'
!
rabbitmqctl set_policy --apply-to queues federate-me "^images."
'{"federation-upstream-set":"all"}'

110. With RabbitMQ we can

111. With RabbitMQ we can
• Ingest data using various protocols: AMQP, MQTT and STOMP

112. With RabbitMQ we can
• Ingest data using various protocols: AMQP, MQTT and STOMP
• Distribute that data globally using Federation

113. With RabbitMQ we can
• Ingest data using various protocols: AMQP, MQTT and STOMP
• Distribute that data globally using Federation
• Scale up using Sharding

114. With RabbitMQ we can
• Ingest data using various protocols: AMQP, MQTT and STOMP
• Distribute that data globally using Federation
• Scale up using Sharding
• Load balance consumers with Federated Queues

115. Credits
world map: wikipedia.org
federation diagrams: rabbitmq.com

116. Questions?

117. Thanks
Alvaro Videla - @old_sound