Michael Klishin from Pivotal gave an update on RabbitMQ. He discussed the state of the 3.7.x releases which focused on deployment automation, operator friendliness, and stability. Future plans include improved scalability, reliability, and simplifying upgrades. Key projects include implementing quorum queues using Raft for improved mirroring, adding OAuth 2.0 support, developing a new schema storage system, and allowing mixed version clusters. The 3.8 release will include quorum queues, OAuth 2.0, and initial mixed version cluster support.

1. MONDAY
12TH NOV
CodeNode
•London•
ORGANISERS

2. © Copyright 2018
Pivotal Software, Inc. All rights Reserved. Version 1.0
November 2018
An update from the RabbitMQ
team

3. Who am I?
Michael Klishin
! Senior Engineer at Pivotal
! RabbitMQ contributor since 2010
! @michaelklishin on GitHub and most other
places

4. Cover w/ Image
Agenda
■ The state of RabbitMQ 3.7.x
■ Future directions
■ What’s coming in 3.8 and beyond

5. The state of 3.7.x
An update from the RabbitMQ team

6. RabbitMQ 3.7.0
3.7.0 release themes
! Deployment automation friendliness
! Operator friendliness
! Stability
! Setting up stage for larger changes

7. RabbitMQ 3.7.0
Operator friendliness
! Distributed stats storage
! Per-vhost limits
! Operator policies
! New logging subsystem
! Proxy protocol
Deployment automation friendliness
! Peer discovery subsystem
! New, easy to generate configuration format
! Extensible command line tools

8. RabbitMQ 3.7.x
What have we been up to in 3.7.x?
! Best practices
! Documentation
! Non-breaking refinements
! Ecosystem improvements

9. RabbitMQ 3.7.x
Documentation
! Upgrades guide
! Monitoring guide
! Production checklist updates
! Runtime tuning guide
Guidance & best practices
! Blue/green deployment upgrades
! Workload-specific runtime tuning

10. RabbitMQ 3.7.x
Non-breaking refinements
! Erlang 21 support
! OAuth 2 support
! New metrics
! New CLI commands
! Greater adoption of the new config format in
plugins
! Usability improvements
! Kubernetes deployment example
! Improved default runtime flags*
! LDAP
Ecosystem Improvements
! Client libraries (primarily Java, .NET, Python)
! Erlang packaging

11. Future directions
An update from the RabbitMQ team

12. Future directions
General directions
! Improved reliability and operator friendliness
! Improved scalability
! Simplify upgrades
! Continue improving and expanding the ecosystem
! Move away from home grown distsys algorithms 🤦 "
! Improved correctness
! Repay technical debt

13. What problems are we addressing?
Scalability
! Mirroring uses a ring topology which is linear
! Excessive bandwidth usage
! No DC awareness
! The core is not protocol-agnostic which leads
to overhead
Reliability
! Recovery from failures is not always
predictable (or easy to reason about)
! Fails to pass certain resiliency tests in some
scenarios
! Queue sync after failure can take too long and
cause a thundering herd
! Schema data store (Mnesia) is overly
opinionated and isn’t a great fit
! Lack of unified WAL/oplog makes backups
hard

14. Future directions
Currently in progress
! Quorum queues (mirrored queues 2.0) based
on Raft
! OAuth 2.0 support
! Mixed version clusters
! Mnevis: a new schema data store
! Protocol-agnostic core

15. Quorum queues
An update from the RabbitMQ team

16. How does Raft-based replication compare?
Improved scalability
! No ring topology, parallel replication
! More reasonable bandwidth usage*
! Opens the door to cross-DC replication
Improved reliability
! Recovery from failures is predictable and well
defined
! Queue sync after failure transfers as little data
as possible
! Passes more resiliency tests
! Opens the door to a unified WAL/oplog to
simplify backups

17. What is Raft?
An update from the RabbitMQ team

18. What is Raft?
! A group of algorithms for reaching consensus in a distributed system
! Implementer-focused
! Proven
! Multiple implementations
! Industry use
! TLA+ specification

19. Raft: an oversimplified explanation
Raft
Follower
Leader
Follower

20. Raft: an oversimplified explanation
Raft
Follower
Leader
Follower
Commit index = 0
Enqueue
Append entry
Append entry
Log = []
Log = []
Log = []

21. Raft: an oversimplified explanation
Raft
Follower
Leader
Follower
Commit index = 1
Success
Success
Success
Log = [{1, Enqueue}]
Log = [{1, Enqueue}]
Log = [{1, Enqueue}]

22. Quorum queues
The State of Quorum Queues
! Reasonably polished
! Throughput beats mirrored queues
! Passes a [slightly modified] Jepsen test
! Available in RabbitMQ 3.8.0-beta.1 today

23. Quorum queues
Implementing Quorum Queues
! Powered by our own from scratch Raft implementation
! github.com/rabbitmq/ra
! Heavily geared towards but not specific to RabbitMQ
! Deviates from Raft a little bit to reduce network and disk I/O
! Does not sacrifice correctness
! Includes a peer unavailability detection library
! Optimized for throughput, adapts I/O for latency or throughput based on load

25. Quorum queues
The State of Quorum Queues
! Has limitations
! Doesn’t support some features, e.g. TTL won’t be supported
! Memory management is still a hard problem
! As any new major feature, will take time to mature (you can help by giving it a try!)

26. Queue types
An update from the RabbitMQ team

27. One queue type
to rule them all?
What are the chances that all
these features can work efficiently
or be easy to reason about when
they are combined?
! Durable
! Auto-delete
! Mirrored
! Lazy
! With TTL
! With message TTL
! With length limit
! With dead-lettering
! With failure recovery settings
! With exclusive consumers
! Affected by plugins
! …

28. One queue type
per workload?
Or is there a better way?
! Durable queues with consensus replication
! Transient queues, auto-delete with TTL and/or
length limit
! “Infinite queues” (durable, aggressive paging to
disk)
! Lower latency in-memory queues

29. OAuth 2.0 Support
An update from the RabbitMQ team

30. OAuth 2.0 support
How does it work?
! Implemented as a plugin, rabbitmq_auth_backend_oauth2
! OAuth 2.0/JWT token scopes that follow naming conventions are translated to RabbitMQ
permissions
! Clients can use any OAuth 2.0 authorization code flow
! Management UI will use the authorization code flow
! Officially supported clients will simplify token renewal

31. OAuth 2.0 support
The State of OAuth 2.0 Support
! Plugin is done, open source and currently targets RabbitMQ 3.8
! Will ship in 3.8.0 Milestone 2
! Management UI needs work
! Targets UAA and AD

32. Next gen schema storage
An update from the RabbitMQ team

33. How does next gen schema storage compare?
Improved scalability
! Mnesia does reasonably well here (for schema
storage)
! Opens the door to DC awareness
! No long-entrenched Mnesia limitations
! Higher rate of development iteration
Improved reliability
! Recovery from failures that works for
RabbitMQ users (and maintainers)
! Nodes no longer have to erase themselves in
order to re-sync
! Can integrate with the unified WAL/oplog

34. Next gen schema storage
The State of Mnevis
! Currently an area of active research
! Mnesia will be used in a node-local way
! Mnesia is quite extensible
! Raft is not necessarily a great fit for transaction log propagation due to [relatively] high
latency

35. Mixed version clusters
An update from the RabbitMQ team

36. Mixed version clusters
How do we get there?
! Relaxing overly conservative restrictions
! Capability testing instead of version testing
! Feature flags
! Extensive internal refactoring
! Somewhat funny APIs to work around Erlang record limitations

37. Mixed version clusters
The State of Mixed Version Clusters
! Feature flag experiment with promising results so far
! Meant to simplify upgrades
! Long running mixed clusters is not a goal
! Cannot guarantee safety for every possible breaking change
! Needs more feedback

38. RabbitMQ 3.8
An update from the RabbitMQ team

39. RabbitMQ 3.8
What’s coming in 3.8?
! Quorum queues
! OAuth 2.0
! Initial mixed version cluster support
! Minor usability improvements

40. RabbitMQ 3.9
An update from the RabbitMQ team

41. RabbitMQ 3.9
What’s coming in 3.9?
! Protocol-agnostic core
! 🤔

42. RabbitMQ 4.0
An update from the RabbitMQ team

43. RabbitMQ 4.0
What’s coming in 4.0?
! Next generation schema databas
! Transport beam? Food replicators? Tricorders? 🤔
! All the breaking changes

44. Thank you!
An update from the RabbitMQ team

45. Transforming How The World Builds Software
© Copyright 2018 Pivotal Software, Inc. All rights Reserved.