The document discusses clustering RabbitMQ using Consul, outlining the operational principles and features of RabbitMQ clusters and the benefits of using Consul for service discovery and health checking. Key topics include the startup sequences for clustering nodes, enhancements provided by the autocluster plugin, and deployment configurations. The presentation is aimed at demonstrating the integration of RabbitMQ with Consul for improved clustering management.

1. Clustering RabbitMQ using Consul
Valentin Fadeev
DevOps Engineer @ Student.com
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 1 / 10

2. Clustering in RabbitMQ
A RabbitMQ broker is a logical grouping of one or several Erlang nodes,
each running the RabbitMQ application and sharing users, virtual hosts,
queues, exchanges, bindings, and runtime parameters. Sometimes we refer
to the collection of nodes as a cluster1.
1
https://www.rabbitmq.com/clustering.html
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 2 / 10

3. Autocluster Plugin1
automated clustering
declarative configuration
pluggable backends
1
https://github.com/rabbitmq/rabbitmq-autocluster
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 3 / 10

4. Startup Sequence v1
start with a random delay
discover registered nodes
choose a node to cluster with and join
register with the backend
begin updating own health check
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 4 / 10

5. Startup Sequence v2
acquire startup lock
discover registered nodes
choose a node to cluster with and join
register with the backend
begin updating own health check
release startup lock
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 5 / 10

6. Why Consul?
DNS-based service discovery
decentralised health checking
configuration store
synchronisation primitives
event propagation
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 6 / 10

7. Locking in Consul1
create a session associated with a node
begin refreshing session TTL
try to acquire the session for a key in KV store
if lock is held by someone else watch for changes
acquire session
release session
1
https://www.consul.io/docs/guides/leader-election.html
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 7 / 10

8. Plugin Enhancements
use long node names
allow health checks in warning status
use start up locking instead of randomized delay
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 8 / 10

9. Deployment
Consul Server
Consul ServerConsul Server
RabbitMQ Consul Client
RabbitMQ Consul Client
RabbitMQ Consul Client
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 9 / 10

10. DNS Configuration
Docker Bridge
BIND
ConsulApp
Host network
Container network
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 10 / 10