Presented at https://www.meetup.com/Architecting-for-Innovation/events/266313278

1. ARCHITECTURE
UNIFICATION
The fight for a greater consensus

2. 2
THE HISTORY OF INFRASTRUCTURE
Single
node
Master
Cluster
Slave
Slave
Slave
Multicluster
Cluster Cluster Cluster

3. Consensus
Lamport’s landmark 1984 paper Time,
Clocks and the Ordering of Events in a
Distributed System provided the first
solution to the replicated database
problem and—more generally—the
distributed state machine problem.
Improper consensus algorithms and
implementations can cause split-brain,
data-loss, slowdown and inaccessibility
scenarios

4. Popular distributed systems by
consensus type
Zab Raft
■ etcd
■ RethinkDB
■ Consul
■ TiKV
■ Redis (RedisRaft)
■ MongoDB
■ …

5. Popular distributed systems by
consensus type
Paxos Other
■ Mencius (variant of Paxos used by
Galera, which is used for open-
source clustering in, e.g., MariaDB)
■ Bucardo (multimaster, multislave,
can be used for Postgres)
■ …
■ Ceph
■ Neo4j
■ Cassandra
■ AWS Elastic Container Service
■ Google Spanner
■ Windows Server Failover Clustering
■ Postgres
■ …

6. Tradeoffs
Overhead
understandability number of RTTs
strong vs weak
leadership
quorum size
multimaster /
multislave

9. https://twitter.com/apachekafka/status/1261786929101565952

10. Datacentre
■ Postgres
■ Redis
■ MySQL
■ RabbitMQ
■ Kafka
■ Kubernetes
■ Mesos
https://landscape.cncf.io

12. SOLVING
CONSENSUS FOR
EVERYONE?
(is that even possible?)

13. Approach
Language
C++ (with C bindings)
Java, Rust
Go
Distributed System
ZooKeeper
etcd
Consul

14. Contribute to everyone? - Current
■ Open mailing-list thread with Apache Mesos, to decouple their ZooKeeper
– Some follow-ups on Twitter and LinkedIn…
■ Presented at Sydney Kubernetes Meetup

15. Contribute to everyone? - Future
Mesos ecosystem (&
Mesos itself)
Kubernetes (offered
at the Kubernetes
meetup @ Google;
they are interested)
Kubernetes
ecosystem
Every other system
on the CNCF website
(and beyond!)
More languages
More distributed
systems (i.e., more
than just etcd,
consul, and
ZooKeeper)

16. QUESTIONS?
samuel@offscale.io

17. Appendix
■ C89 tools for boostrapping installations
■ Independent package-managers à la rvm and nvm; but for Kafka, MongoDB, and dozens more
■ Support for:
– Windows
– Linux (various distributions)
– macOS
– FreeBSD
– OpenBSD
– Solaris → OpenSolaris → OpenIndiana
■ Useful inside—and outside—of Docker, Packer, Vagrantfile, Ansible, Chef, Puppet