Valentin Fadeev - Clustering RabbitMQ using Consul. Demo links - https://github.com/ValFadeev/consul-demo https://github.com/ValFadeev/consul-container

1. Clustering RabbitMQ using Consul
Valentin Fadeev
DevOps Engineer @ Student.com
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 1 / 10

2. Clustering in RabbitMQ
A RabbitMQ broker is a logical grouping of one or several Erlang nodes,
each running the RabbitMQ application and sharing users, virtual hosts,
queues, exchanges, bindings, and runtime parameters. Sometimes we refer
to the collection of nodes as a cluster1.
1
https://www.rabbitmq.com/clustering.html
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 2 / 10

3. Autocluster Plugin1
automated clustering
declarative configuration
pluggable backends
1
https://github.com/rabbitmq/rabbitmq-autocluster
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 3 / 10

4. Startup Sequence v1
start with a random delay
discover registered nodes
choose a node to cluster with and join
register with the backend
begin updating own health check
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 4 / 10

5. Startup Sequence v2
acquire startup lock
discover registered nodes
choose a node to cluster with and join
register with the backend
begin updating own health check
release startup lock
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 5 / 10

6. Why Consul?
DNS-based service discovery
decentralised health checking
configuration store
synchronisation primitives
event propagation
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 6 / 10

7. Locking in Consul1
create a session associated with a node
begin refreshing session TTL
try to acquire the session for a key in KV store
if lock is held by someone else watch for changes
acquire session
release session
1
https://www.consul.io/docs/guides/leader-election.html
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 7 / 10

8. Plugin Enhancements
use long node names
allow health checks in warning status
use start up locking instead of randomized delay
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 8 / 10

9. Deployment
Consul Server
Consul ServerConsul Server
RabbitMQ Consul Client
RabbitMQ Consul Client
RabbitMQ Consul Client
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 9 / 10

10. DNS Configuration
Docker Bridge
BIND
ConsulApp
Host network
Container network
@ValFadeev HashiCorp Meetup, London 20 Jun 2017 10 / 10