This talk is about CAP theorem since its inception in 90s, and the PACELC theorem which followed it, and the present.

1. Pass Elk
CAP Theorem since 90s and
Beyond!
@rprabhu
randomsurfer

8. ● Latency
○ Being closer to user
● Availability
○ Elastic infrastructure
○ Resilience to external factors
○ Failover
● Performance:
○ Moore’s law
○ Scale-out
○ Scale up
● Security
Why distribute?

9. CAPTheorem
● Started as a principle from Eric
Brewer in 98.
● Later a conjecture.
● Wasn’t even for “distributed”
systems.
● Proven in 2002!
“Harvest, Yield, and Scalable Tolerant
Systems” =>

13. Fallacies of distributed computing

14. ● Single node - Not a distributed
system
● Multiple nodes - all unavailable
during network partition: not a
distributed system
● No network partitions - Unicorns
will be seen too if this is true
CA system

15. ● “Total or Partial Loss of Network”
○ Slow Network
● Partitions are common
○ The Network is Reliable
■ “Five racks going wonky (40-80 machines seeing 50 percent packet
loss).”
○ Correlated vs Independent failures
● What is not a partition:
○ Failed nodes
○ Degraded nodes: c.f. Availability
Partitions

16. ● Constraints imposed when a single node data store needs to be distributed
over asynchronous network
● “In presence of partitions, a system has to choose between either C
(consistency) or A (Availability)”: Partition Decision
○ Consistency: Single-copy consistency aka Linearizability
○ Availability: Response from a healty node according to specifications.
● “In absence of partitions? - whatever the system is meant to be”
○ A.C.I.D consistency?
○ PACELC
● ACID and CAP
CAP theorem in tldr (kinda..)

18. https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html

19. https://jepsen.io/consistency

20. PACELC: CAP <-> ELC
● What’s missing in CAP: Latency!
● “In case of network partitioning (P) in a distributed computer system, one
has to choose between availability (A) and consistency (C) (as per the CAP
theorem), but else (E), even when the system is running normally in the
absence of partitions, one has to choose between latency (L) and
consistency (C).”
● Compromises a partition-safe data store has to make when there are no
partitions.

21. Systems at Yelp
● Zookeeper: PC + EC (sync) / EL (default)
● Cassandra (and Dynamo-like): AP + EL/EC (tunable)
○ R+W < or > N
● Elasticsearch: AP + EL
● PNUTS/Sherpa: PC + EL
● MySQL: CP + EC (multi in causal) / EL (async)
● What about CP + EC systems?
○ VoltDB
○ HBase / BigTable
○ Megastore

22. Beyond CAP
● CAP is reductionist.
○ PACELC
● Intelligent clients
● Linearizability is not a requirement in most of cases.
○ Availability is more important
○ Highly Available Transactions
○ Probabilistic Bounded Staleness
● Partition Management

23. Intelligent clients
● Conflict resolution
○ LWW is prone to data loss
● Write to master, read from master for a while
○ Stickiness - “Dirty” Cookie
● Multi-master but segmented
● State/Session management by clients
○ Thick API pattern
● Google Docs under partition

24. Partition Decision

25. Managing Partitions
● Co-ordination Free Systems
○ Coordinate only where required
○ AWS Aurora Paper
● Commutative Data Types (CRDT)
○ Commutative data types: counters, sets
○ Logical monotonicity
● Mergeable Persistent Data Structures
● Version Vectors

26. Further Reading
● https://www.the-paper-trail.org/page/cap-faq/ <= Read this for sure!
● https://aphyr.com/posts/313-strong-consistency-models => Good tree of
consistency
● Proof of CAP theorem by Gilbert and Lynch
● https://www.researchgate.net/publication/220476881_CAP_Twelve_years_l
ater_How_the_Rules_have_Changed
● https://queue.acm.org/detail.cfm?id=2582994
● https://codahale.com/you-cant-sacrifice-partition-tolerance/
● Highly Available Transactions: Virtues and Limitations
● Harvest, Yield, and Scalable Tolerant Systems
● http://dataintensive.net/

27. Further Reading
● Clarifications On The CAP Theorem And Data-Related Errors
● Replicated Data Consistency Explained Through Baseball
● Critique of CAP Theorem
● Linearizability: A Correctness Condition for Concurrent Objects
● Brewer's conjecture and the feasibility of consistent, available,
partition-tolerant web services
● https://people.eecs.berkeley.edu/~brewer/PODC2000.pdf
● http://www.bailis.org/blog/linearizability-versus-serializability/
● https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-
cp-or-ap.html
● Impossibility of Distributed Consensus with One Faulty Process

28. Credits
● https://pixabay.com/en/pass-win-succeed-winner-success-1714374/
● https://pdfs.semanticscholar.org/5015/8bc1a8a67295ab7bce0550886a9859000dc2.pdf
● https://github.com/danluu/post-mortems
● https://blog.github.com/2018-10-30-oct21-post-incident-analysis/
● https://www.datacenterknowledge.com/archives/2015/08/19/lightning-strikes-google-data-center-disrupts-cloud-services
● https://dl.acm.org/citation.cfm?id=564601
● https://pixabay.com/en/unicorn-fly-by-side-wings-horse-3418003/
● https://people.eecs.berkeley.edu/~brewer/PODC2000.pdf
● https://en.wikipedia.org
● https://ieeexplore.ieee.org/document/6133253
● https://webapp.lifesizecloud.com
● https://aphyr.com/posts/313-strong-consistency-models
● https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html
● https://yourlogicalfallacyis.com/
● https://codahale.com/you-cant-sacrifice-partition-tolerance/
● https://www.fastcompany.com/1825005/how-one-second-could-cost-amazon-16-billion-sales
● http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-and-yahoos-little.html