This document discusses building a distributed and robust Phoenix API application backed by CockroachDB. It introduces CockroachDB and how it enables consistency, availability, and partition tolerance using consensus replication with Raft. The document also demonstrates CockroachDB passing the Jensen tests to prove its consistency under failures. Finally, it shows code for a sample Phoenix API application using CockroachDB with Postgrex and authentication.

1. APPLYING CAP TO BUILD DISTRIBUTED
ROBUST PHOENIX API APP
PHOENIX API APP BACKED BY
COCKROACHDB

2. 2
Who Are we
yeongsheng@odd-e.com

3. AGENDA
‣ YOUR WEAKEST LINK
‣ #NEWSQL + D.A.S.H
‣ COCKROACHDB
‣ WHEN RUBBER HITS THE ROAD
‣ SHOW ME THE CODE
‣ Q&A

4. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
YOUR WEAKEST LINK
▸ Traditional RDBMS
▸ NoSQL
▸ NewSQL
▸ YugaqByteDB
▸ TiDB
▸ CockroachDB

5. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
D.A.S.H
▸ Beyond 12-Factor-App
▸ 4 properties of CloudNative Ready DB
DISPOSABILITY
API SYMMETRY
SHARED NOTHING
HORIZONTAL SCALING
D :
A :
S :
H :

6. WHY?
COCKROACHDB

7. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP

8. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
ENABLES GLOBAL, SCALABLE, RESILIENT SERVICES
▸ Make life easier for humans. Low-touch and
highly automated distributed DB for
operators yet simple to reason about for
developers.
▸ Industry-leading consistency, even on
massively scaled deployments. Enabling
distributed transactions, yet removing the
pain of eventual consistency issues.
▸ Always-on database that accepts reads and
writes on all nodes without generating
conflicts.
▸ Flexible deployment in any environment,
without tying you to any platform or vendor.
▸ Support familiar tools for working with
relational data (i.e., SQL).

9. WHAT GIVES?
CONSISTENCY

10. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
PERILS OF INCONSISTENCIES
▸ Emin Gün Sirer, an Associate Professor at Cornell University, wrote a
blog post blaming eventually consistent data stores for the lost
bitcoins. He mentions MongoDB, Cassandra and Riak among the
NoSQL solutions that are vulnerable to banking thefts because:

11. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
CONSENSUS REPLICATION
▸ Raft
▸ Commit when quorum
has written data (nodes
> = 3)
▸ 1 consensus group per
data range
ution
replication protocol (Raft)
transactions
t, repair, rebalance
Gossip /
Raft
SQL API
Distributed, transactional KV
Node 1
Node 2
Node 3

12. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
CONSISTENCY<-CONSENSUS<-MULTI-ACTIVE AVAILABILITY
▸ “CONSISTENCY” - ACID semantics & CAP theorem. Data should be
anomaly-free.
▸ “CLUSTER” - Single logical DB, multiple nodes, joined together to
form a uniform consistent cluster.
▸ “RANGE” - All data (tables, indices, etc) as a giant sorted map of KV
pairs; 64 MB in size per range; Auto split and balanced across nodes.
▸ “LEASEHOLDER” - Ranges replicated across nodes & 1 replica of
range holds “range lease”; coordinates all reads/writes to that range.

13. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
CONSISTENCY<-CONSENSUS<-MULTI-ACTIVE AVAILABILITY
▸ “RAFTLEADER” - 1 replica for each range is “leader” for writes; Leader
coordinates all writes to raft group (followers); Quorum consensus for
writes. Leaseholder usually also Raftleader. When a write doesn't achieve
consensus, forward progress halts to maintain consistency within the cluster.
Ensures ACID semantics for multi-tables data operations are consistent.
▸ “REPLICATION” - Synchronous distribution of copies of data that are
ensured to be consistent.
▸ “MULTI-ACTIVE AVAILABILITY” - Each node in cluster can handle reads/
writes for a subset of stored data (range and quorum consensus concepts).
Symmetrical -> works nicely with load balancers over SQL API (PostgreSQL
wire format).

14. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
@cockroachdb
Put “cherry” Leader
apricot
banana
blueberry
cherry
grape
Follower
apricot
banana
blueberry
grape
Follower
apricot
banana
blueberry
grape
Consensus Replication
Put “cherry” Leader
apricot
banana
blueberry
cherry
grape
Follower
apricot
banana
blueberry
grape
Follower
apricot
banana
blueberry
grape
Put “cherry”
Consensus Replication
RAFT
CONSENSUS
PROTOCOL

15. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
@cockroach
Write committed when
2 out of 3 nodes have written data
Follower
apricot
banana
blueberry
cherry
grape
Follower
apricot
banana
blueberry
grape
Consensus Replication
Put “cherry” Leader
apricot
banana
blueberry
cherry
grape
Put “cherry”

16. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
@cockroachdb
Follower
apricot
banana
blueberry
cherry
grape
Follower
apricot
banana
blueberry
grape
Consensus Replication
Put “cherry” Leader
apricot
banana
blueberry
cherry
grape
Put “cherry”
AckAck
Follower
apricot
banana
blueberry
cherry
grape
Follower
apricot
banana
blueberry
grape
Consensus Replication
Put “cherry” Leader
apricot
banana
blueberry
cherry
grape
Put “cherry”
cherry
Ack

17. CAN IT BE
TRUSTED?
WHEN RUBBER HITS
THE ROAD

18. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
COCKROACHDB PASSES JENSEN TESTING
▸ CockroachDB Beta passes Jepsen
testing
▸ Jepsen tests lessons 2+ years on...
▸ Register, Bank, Monotonic & Sets
▸ G2, Sequential, Comments & Nemesis
modes (clock skews, network failures)

19. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP

20. DEMO

21. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
▸ https://github.com/yeongsheng-tan/lunchbox_api.git
▸ Branches
▸ master
▸ cockroachdb_postgrex_cdb
▸ cockroachdb_postgrex_cdb_with_api_jwt_auth
▸ cockroachdb_postgrex_cdb_api_jwt_auth_e2e_tests

22. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
WHAT IS NEEDED?
▸ Production code -> hex
package: postgrex_cdb (yet to
support correlated sub-
queries)
▸ Test code -> hex package:
ecto_replay_sandbox
▸ No prod code changes!!!
▸ Test code changes.

23. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP

24. APPLYING CAP TO BUILD DISTRIBUTED ROBUST PHOENIX API APP
CREDITS
▸ Nate Stewart
▸ Ben Darnell
▸ Aphyr Jepsen
▸ Diana Hsieh
▸ Emin Gün Sirer