SlideShare a Scribd company logo
1 of 39
Download to read offline
A Critique of the CAP Theorem
Martin Kleppmann
Papers We Love San Diego
Daniel Norman – August 4th, 2016
@DreamingInCode
PWLSD Presents:
GREETINGS PROFESSOR BREWER.
SHALL WE PLAY A GAME?
Love to. How about Global Total Order?
CAP TheoremCAP Theorem
You’ve probably heard it before:
Consistency, Availability, Partition
tolerance:
“pick any two”
Seth Gilbert Nancy Lynch
Meet Our Players:
Eric Brewer Martin Kleppmann
Chronology of CAP
2000: Brewer publicizes his conjecture in various talks and papers – “The CAP Principle”.
2002: Gilbert and Lynch formally proved Brewer’s conjecture, and CAP Theorem was born.
1970s, 80s, and 90s: Absolutely[1] Nothing[2] happened[3] Nothing[4] to[5] see[6] here[7] folks.
[1] Paul R Johnson and Robert H Thomas. RFC 677: The maintenance of duplicate databases. Network Working Group, January 1975. URL https://tools.ietf.org/html/rfc677.
[2] Jim N Gray, Raymond A Lorie, Gianfranco R Putzolu, and Irving L Traiger. Granularity of locks and degrees of consistency in a shared data base. In G M Nijssen, editor,
Modelling in Data Base Management Systems: Proceedings of the IFIP Working Conference on Modelling in Data Base Management Systems, pages 364–394. Elsevier/North
Holland, 1976.
[3] Leslie Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Transactions on Computers, 28(9):690–691, September 1979.
doi:10.1109/TC.1979.1675439.
[4] Bruce G Lindsay, Patricia Griffiths Selinger, C Galtieri, Jim N Gray, Raymond A Lorie, Thomas G Price, Gianfranco R Putzolu, Irving L Traiger, and Bradford W Wade. Notes on
distributed databases. Technical Report RJ2571(33471), IBM Research, July 1979.
[5] Bowen Alpern and Fred B Schneider. Defining liveness. Information Processing Letters, 21(4):181–185, October 1985. doi:10.1016/0020-0190(85)90056-0.
[6] Susan B Davidson, Hector Garcia-Molina, and Dale Skeen. Consistency in partitioned networks. ACM Computing Surveys, 17(3):341–370, September 1985.
doi:10.1145/5505.5508.
[7] Linearizability: A Correctness Condition for Concurrent Objects. M P. Herlihy and J M. Wing. ACM Transactions, Vol. 12, No. 3, July 1990.
“Consistency”
“Consistency”
“Consistency”
“I’m totally pro-choice.” (Fox News, October 31, 1999)
“I’m pro-life.” (CPAC, February 10, 2011)
“I wanted to do this for myself. I had to do it for myself.”
(Time, August 18, 2015)
“I don’t want it for myself. I don’t need it for myself.” (ABC
News, November 20, 2015)
“I think the institution of marriage should be between a
man and a woman.” (The Advocate, February 15, 2000)
“If two people dig each other, they dig each other.”
(Trump University “Trump Blog,” December 22, 2005)
“I’m against gay marriage.” (Fox News, April 14, 2011)
What is “Consistency”
● Brewer defines consistency as one-copy-serializability (1SR)
● Gilbert and Lynch define consistency as linearizability
Linearizability
Kyle Kingsbury 2014 – https://aphyr.com/posts/313-strong-consistency-models
Kyle Kingsbury 2014 – https://aphyr.com/posts/313-strong-consistency-models
Put Simply:
Linearizability can be viewed as a special case of strict
serializability where transactions are restricted to consist
of a single operation applied to a single object.[1]
[1] Linearizability: A Correctness Condition for Concurrent Objects. By M P. Herlihy and J M. Wing. ACM Transactions on Programming
Languages and Systems, Vol. 12, No. 3, July 1990.
Linearizability vs Serializability
Meaningfully different,
But close enough for our purposes
From Martin Kleppmann’s “Sequential Consistency versus Linearizability”
From Martin Kleppmann’s “Sequential Consistency versus Linearizability”
I ALREADY F#$KING
READ IT!
(Eventual consistency)
(Terminal inconsistency)
“Availability”
Gilbert & Lynch defined availability differently.
Property of algorithm, or observed metric?
Brewer:
“availability is obviously continuous from 0 to 100 percent”
Gylbert & Lynch:
“For a distributed system to be continuously available, every request received by a non-failing node in the
system must result in a response”
“Availability”
The Server is DOWN! – What is “UP” anyway?
● Is it a server that’s fails to respond in 1000ms?
● What if it responds 5 minutes later?
● What if it responds, but with invalid data?
● What if it responds but the response is not received?
● What if your request packet is dropped, but all others are fine?
● What is the sound of one hand clapping?
“Availability”
Hush you! I’ll get around to responding.
“Availability”
If everybody fails, that’s availability.
WUT?
“Availability”
● Gilbert & Lynch definition – Contradictory and counter-intuitive
● Brewer’s definition is ok
● Nonsensical to call an algorithm “Available”
“Partition Tolerance”
A Network partition is:
“a communication failure in which the network is split into disjoint
sub-networks, with no communication possible across sub-networks”
“Partition Tolerance”
Most networks are Fairloss links
A Fairloss link is a link where the probability the message you send is
delivered is non-zero.
(and the probability of delivery is less than 100%)
“Partition Tolerance”
A meditation:
What fraction of messages must go undelivered before it’s a “Partition”?
CA AP CP CCCP
Using Gilbert & Lynch definitions,
essentially only two options:
CA - Avoid the gulag dear comrade, coordinate carefully with your party official.
AP - You’re on your own, capitalist swine!
CP - Not really any different from CA
CCCP - Purge all your data and start over every few years
PROVE IT
From “Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web
Services” Gylbert & Lynch 2002
PROVE IT
G&L offer Irrefutable proof that:
The operation of their algorithm is non-linearizable only if the partition lasts
forever.
But:
Partitions don’t last forever!
Sometimes faults aren’t detected!
Maybe you don’t even want linearizability!
GREETINGS PROFESSOR BREWER
CAP IS A STRANGE GAME.
THE ONLY WINNING MOVE IS NOT
TO PLAY.
HELLO
HOW ABOUT A NICE GAME OF CHESS?
In my humble opinion...
There’s a simpler way to say it:
● Some consistency models require total event ordering
● Any up-to-date list can only exist in a single point in space
● We are not optimistic about FTL information transfer
Be patient, or be self-sufficient. No whining.
Kleppmann proposes a “Delay sensitivity framework”
In a nutshell: Travel takes time, how patient can you be?
So What’s better?
“Delay sensitivity framework”
● Networks latency is lumpy / uncertain
● Very few scenarios entail actually reliable networks - packet loss is common
● Understanding lower bounds for different consistency models
So What’s better?
Proposed Terminology
● Availability - Empirical measurements only!
● Delay Sensitive - How patient can we be for a given operation?
● Network Faults - All kinds of weird shit happens on networks.
● Fault Tolerance - Under what specific failure modes can we guarantee our invariants?
● Consistency - Not just one, but many different consistency models to choose from
So What’s better?
Strong vs Weak
Consistency models galore:
Consistency models are a continuum.
“Strong” and “Weak” are conversational terms, not formal.
“Weak” does not mean unsafe.
It’s all about selecting which invariants you want.
Kyle Kingsbury 2014 – https://aphyr.com/posts/313-strong-consistency-models
In Conclusion:
● Irrefutable truths mayy be predicated on
misleading definitions
● consistency is subjective.
● CAP has outlived it’s usefulness
● We can do better!
● Let’s reason about latency
● Lets fix our terminology
● Rigor is good, CS needs moar of it
Thank you!
Papers We Love San Diego
A Critique of the CAP Theorem

More Related Content

Similar to PWLSD.1 A critique of the cap theorem - Martin Kleppmann

Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...Codemotion
 
Continuous Automated Testing - Cast conference workshop august 2014
Continuous Automated Testing - Cast conference workshop august 2014Continuous Automated Testing - Cast conference workshop august 2014
Continuous Automated Testing - Cast conference workshop august 2014Noah Sussman
 
Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...
Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...
Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...Health IT Conference – iHT2
 
A time series of networks. Is everything OK? Are there anomalies?
A time series of networks. Is everything OK? Are there anomalies?A time series of networks. Is everything OK? Are there anomalies?
A time series of networks. Is everything OK? Are there anomalies?CSIRO
 
2nd Spinoza workshop: Looking at the Long Tail - introductory slides
2nd Spinoza workshop: Looking at the Long Tail - introductory slides2nd Spinoza workshop: Looking at the Long Tail - introductory slides
2nd Spinoza workshop: Looking at the Long Tail - introductory slidesFilip Ilievski
 
A Digital Conversation: The Next Web
A Digital Conversation: The Next Web A Digital Conversation: The Next Web
A Digital Conversation: The Next Web Reading Room
 
We ain’t seen nothing yet - making sense of a massively connected world
We ain’t seen nothing yet - making sense of a massively connected worldWe ain’t seen nothing yet - making sense of a massively connected world
We ain’t seen nothing yet - making sense of a massively connected worldMartin Spindler
 
Large Components in the Rearview Mirror
Large Components in the Rearview MirrorLarge Components in the Rearview Mirror
Large Components in the Rearview MirrorMichelle Brush
 
Hushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for EchoHushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for EchoDeja vu Security
 
Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...
Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...
Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...Sri Ambati
 
Anomalous Networks
Anomalous NetworksAnomalous Networks
Anomalous NetworksCSIRO
 
The Two Generals Problem
The Two Generals ProblemThe Two Generals Problem
The Two Generals ProblemVoltDB
 
You online: Identity, Privacy, and the Self
You online: Identity, Privacy, and the SelfYou online: Identity, Privacy, and the Self
You online: Identity, Privacy, and the SelfAbhay Agarwal
 
The Next Moore's Law: Netness v6x
The Next Moore's Law: Netness v6xThe Next Moore's Law: Netness v6x
The Next Moore's Law: Netness v6xSheldon Renan
 
Importance Of College Education Essay Essay O
Importance Of College Education Essay  Essay OImportance Of College Education Essay  Essay O
Importance Of College Education Essay Essay OKelly Lipiec
 
Architecting a Post Mortem - Velocity 2018 San Jose Tutorial
Architecting a Post Mortem - Velocity 2018 San Jose TutorialArchitecting a Post Mortem - Velocity 2018 San Jose Tutorial
Architecting a Post Mortem - Velocity 2018 San Jose TutorialWill Gallego
 
The Art of Speaking Data.
The Art of Speaking Data.The Art of Speaking Data.
The Art of Speaking Data.David Wellman
 

Similar to PWLSD.1 A critique of the cap theorem - Martin Kleppmann (20)

Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...
 
Black ops 2012
Black ops 2012Black ops 2012
Black ops 2012
 
Continuous Automated Testing - Cast conference workshop august 2014
Continuous Automated Testing - Cast conference workshop august 2014Continuous Automated Testing - Cast conference workshop august 2014
Continuous Automated Testing - Cast conference workshop august 2014
 
Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...
Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...
Health IT Summit Austin 2013 - Case Study "Enabling Data Interoperability thr...
 
A time series of networks. Is everything OK? Are there anomalies?
A time series of networks. Is everything OK? Are there anomalies?A time series of networks. Is everything OK? Are there anomalies?
A time series of networks. Is everything OK? Are there anomalies?
 
2nd Spinoza workshop: Looking at the Long Tail - introductory slides
2nd Spinoza workshop: Looking at the Long Tail - introductory slides2nd Spinoza workshop: Looking at the Long Tail - introductory slides
2nd Spinoza workshop: Looking at the Long Tail - introductory slides
 
A Digital Conversation: The Next Web
A Digital Conversation: The Next Web A Digital Conversation: The Next Web
A Digital Conversation: The Next Web
 
Dec2018 istanbul-2
Dec2018 istanbul-2Dec2018 istanbul-2
Dec2018 istanbul-2
 
We ain’t seen nothing yet - making sense of a massively connected world
We ain’t seen nothing yet - making sense of a massively connected worldWe ain’t seen nothing yet - making sense of a massively connected world
We ain’t seen nothing yet - making sense of a massively connected world
 
Large Components in the Rearview Mirror
Large Components in the Rearview MirrorLarge Components in the Rearview Mirror
Large Components in the Rearview Mirror
 
Hushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for EchoHushcon 2016 Keynote: Test for Echo
Hushcon 2016 Keynote: Test for Echo
 
Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...
Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...
Explaining Black-Box Machine Learning Predictions - Sameer Singh, Assistant P...
 
Anomalous Networks
Anomalous NetworksAnomalous Networks
Anomalous Networks
 
The Two Generals Problem
The Two Generals ProblemThe Two Generals Problem
The Two Generals Problem
 
You online: Identity, Privacy, and the Self
You online: Identity, Privacy, and the SelfYou online: Identity, Privacy, and the Self
You online: Identity, Privacy, and the Self
 
The Next Moore's Law: Netness v6x
The Next Moore's Law: Netness v6xThe Next Moore's Law: Netness v6x
The Next Moore's Law: Netness v6x
 
Logic - Week 1
Logic - Week 1Logic - Week 1
Logic - Week 1
 
Importance Of College Education Essay Essay O
Importance Of College Education Essay  Essay OImportance Of College Education Essay  Essay O
Importance Of College Education Essay Essay O
 
Architecting a Post Mortem - Velocity 2018 San Jose Tutorial
Architecting a Post Mortem - Velocity 2018 San Jose TutorialArchitecting a Post Mortem - Velocity 2018 San Jose Tutorial
Architecting a Post Mortem - Velocity 2018 San Jose Tutorial
 
The Art of Speaking Data.
The Art of Speaking Data.The Art of Speaking Data.
The Art of Speaking Data.
 

Recently uploaded

Business Analyzopedia - Your Pocket Gita for Business Analysis
Business Analyzopedia - Your Pocket Gita for Business AnalysisBusiness Analyzopedia - Your Pocket Gita for Business Analysis
Business Analyzopedia - Your Pocket Gita for Business AnalysisDEEPRAJ PATHAK
 
OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024OpenMetadata
 
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive ReviewRevolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Reviewjw364beach
 
Effort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software ProjectsEffort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software ProjectsDEEPRAJ PATHAK
 
ManageIQ - Sprint 234 Review - Slide Deck
ManageIQ - Sprint 234 Review - Slide DeckManageIQ - Sprint 234 Review - Slide Deck
ManageIQ - Sprint 234 Review - Slide DeckManageIQ
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?Alexandre Beguel
 
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...OnePlan Solutions
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
logical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptxlogical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptxRemote DBA Services
 
Reliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfReliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfRalf Gommers
 
SPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptx
SPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptxSPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptx
SPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptxVersion 1 Analytics
 
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...Piyovi
 
Preparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple ArchitecturesPreparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple ArchitecturesAke Koomsin
 
Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.OnePlan Solutions
 
Chapter -5 Agile Testing types and its examples.pptx
Chapter -5 Agile Testing types and its examples.pptxChapter -5 Agile Testing types and its examples.pptx
Chapter -5 Agile Testing types and its examples.pptxManishaPatil932723
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfkalichargn70th171
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
The State of the Green IT at the beginning of 2024
The State of the Green IT at the beginning of 2024The State of the Green IT at the beginning of 2024
The State of the Green IT at the beginning of 2024Artur Skowroński
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxAS Design & AST.
 

Recently uploaded (20)

Business Analyzopedia - Your Pocket Gita for Business Analysis
Business Analyzopedia - Your Pocket Gita for Business AnalysisBusiness Analyzopedia - Your Pocket Gita for Business Analysis
Business Analyzopedia - Your Pocket Gita for Business Analysis
 
OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024
 
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive ReviewRevolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
 
Effort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software ProjectsEffort Estimation Techniques used in Software Projects
Effort Estimation Techniques used in Software Projects
 
ManageIQ - Sprint 234 Review - Slide Deck
ManageIQ - Sprint 234 Review - Slide DeckManageIQ - Sprint 234 Review - Slide Deck
ManageIQ - Sprint 234 Review - Slide Deck
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?
 
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
logical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptxlogical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptx
 
Reliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfReliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdf
 
SPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptx
SPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptxSPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptx
SPSS Statistics - Encrypting a Syntax File in IBM SPSS Statistics.pptx
 
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
 
Preparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple ArchitecturesPreparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple Architectures
 
Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.
 
Chapter -5 Agile Testing types and its examples.pptx
Chapter -5 Agile Testing types and its examples.pptxChapter -5 Agile Testing types and its examples.pptx
Chapter -5 Agile Testing types and its examples.pptx
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
The State of the Green IT at the beginning of 2024
The State of the Green IT at the beginning of 2024The State of the Green IT at the beginning of 2024
The State of the Green IT at the beginning of 2024
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptx
 

PWLSD.1 A critique of the cap theorem - Martin Kleppmann

  • 1. A Critique of the CAP Theorem Martin Kleppmann Papers We Love San Diego Daniel Norman – August 4th, 2016 @DreamingInCode
  • 3. GREETINGS PROFESSOR BREWER. SHALL WE PLAY A GAME? Love to. How about Global Total Order?
  • 5. You’ve probably heard it before: Consistency, Availability, Partition tolerance: “pick any two”
  • 6. Seth Gilbert Nancy Lynch Meet Our Players: Eric Brewer Martin Kleppmann
  • 7. Chronology of CAP 2000: Brewer publicizes his conjecture in various talks and papers – “The CAP Principle”. 2002: Gilbert and Lynch formally proved Brewer’s conjecture, and CAP Theorem was born. 1970s, 80s, and 90s: Absolutely[1] Nothing[2] happened[3] Nothing[4] to[5] see[6] here[7] folks. [1] Paul R Johnson and Robert H Thomas. RFC 677: The maintenance of duplicate databases. Network Working Group, January 1975. URL https://tools.ietf.org/html/rfc677. [2] Jim N Gray, Raymond A Lorie, Gianfranco R Putzolu, and Irving L Traiger. Granularity of locks and degrees of consistency in a shared data base. In G M Nijssen, editor, Modelling in Data Base Management Systems: Proceedings of the IFIP Working Conference on Modelling in Data Base Management Systems, pages 364–394. Elsevier/North Holland, 1976. [3] Leslie Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Transactions on Computers, 28(9):690–691, September 1979. doi:10.1109/TC.1979.1675439. [4] Bruce G Lindsay, Patricia Griffiths Selinger, C Galtieri, Jim N Gray, Raymond A Lorie, Thomas G Price, Gianfranco R Putzolu, Irving L Traiger, and Bradford W Wade. Notes on distributed databases. Technical Report RJ2571(33471), IBM Research, July 1979. [5] Bowen Alpern and Fred B Schneider. Defining liveness. Information Processing Letters, 21(4):181–185, October 1985. doi:10.1016/0020-0190(85)90056-0. [6] Susan B Davidson, Hector Garcia-Molina, and Dale Skeen. Consistency in partitioned networks. ACM Computing Surveys, 17(3):341–370, September 1985. doi:10.1145/5505.5508. [7] Linearizability: A Correctness Condition for Concurrent Objects. M P. Herlihy and J M. Wing. ACM Transactions, Vol. 12, No. 3, July 1990.
  • 10. “Consistency” “I’m totally pro-choice.” (Fox News, October 31, 1999) “I’m pro-life.” (CPAC, February 10, 2011) “I wanted to do this for myself. I had to do it for myself.” (Time, August 18, 2015) “I don’t want it for myself. I don’t need it for myself.” (ABC News, November 20, 2015) “I think the institution of marriage should be between a man and a woman.” (The Advocate, February 15, 2000) “If two people dig each other, they dig each other.” (Trump University “Trump Blog,” December 22, 2005) “I’m against gay marriage.” (Fox News, April 14, 2011)
  • 11. What is “Consistency” ● Brewer defines consistency as one-copy-serializability (1SR) ● Gilbert and Lynch define consistency as linearizability
  • 12. Linearizability Kyle Kingsbury 2014 – https://aphyr.com/posts/313-strong-consistency-models
  • 13. Kyle Kingsbury 2014 – https://aphyr.com/posts/313-strong-consistency-models
  • 14. Put Simply: Linearizability can be viewed as a special case of strict serializability where transactions are restricted to consist of a single operation applied to a single object.[1] [1] Linearizability: A Correctness Condition for Concurrent Objects. By M P. Herlihy and J M. Wing. ACM Transactions on Programming Languages and Systems, Vol. 12, No. 3, July 1990.
  • 15. Linearizability vs Serializability Meaningfully different, But close enough for our purposes
  • 16. From Martin Kleppmann’s “Sequential Consistency versus Linearizability”
  • 17. From Martin Kleppmann’s “Sequential Consistency versus Linearizability”
  • 18. I ALREADY F#$KING READ IT! (Eventual consistency)
  • 20. “Availability” Gilbert & Lynch defined availability differently. Property of algorithm, or observed metric? Brewer: “availability is obviously continuous from 0 to 100 percent” Gylbert & Lynch: “For a distributed system to be continuously available, every request received by a non-failing node in the system must result in a response”
  • 21. “Availability” The Server is DOWN! – What is “UP” anyway? ● Is it a server that’s fails to respond in 1000ms? ● What if it responds 5 minutes later? ● What if it responds, but with invalid data? ● What if it responds but the response is not received? ● What if your request packet is dropped, but all others are fine? ● What is the sound of one hand clapping?
  • 22. “Availability” Hush you! I’ll get around to responding.
  • 23. “Availability” If everybody fails, that’s availability. WUT?
  • 24. “Availability” ● Gilbert & Lynch definition – Contradictory and counter-intuitive ● Brewer’s definition is ok ● Nonsensical to call an algorithm “Available”
  • 25. “Partition Tolerance” A Network partition is: “a communication failure in which the network is split into disjoint sub-networks, with no communication possible across sub-networks”
  • 26. “Partition Tolerance” Most networks are Fairloss links A Fairloss link is a link where the probability the message you send is delivered is non-zero. (and the probability of delivery is less than 100%)
  • 27. “Partition Tolerance” A meditation: What fraction of messages must go undelivered before it’s a “Partition”?
  • 28. CA AP CP CCCP Using Gilbert & Lynch definitions, essentially only two options: CA - Avoid the gulag dear comrade, coordinate carefully with your party official. AP - You’re on your own, capitalist swine! CP - Not really any different from CA CCCP - Purge all your data and start over every few years
  • 29. PROVE IT From “Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services” Gylbert & Lynch 2002
  • 30. PROVE IT G&L offer Irrefutable proof that: The operation of their algorithm is non-linearizable only if the partition lasts forever. But: Partitions don’t last forever! Sometimes faults aren’t detected! Maybe you don’t even want linearizability!
  • 31. GREETINGS PROFESSOR BREWER CAP IS A STRANGE GAME. THE ONLY WINNING MOVE IS NOT TO PLAY. HELLO HOW ABOUT A NICE GAME OF CHESS?
  • 32. In my humble opinion... There’s a simpler way to say it: ● Some consistency models require total event ordering ● Any up-to-date list can only exist in a single point in space ● We are not optimistic about FTL information transfer Be patient, or be self-sufficient. No whining.
  • 33. Kleppmann proposes a “Delay sensitivity framework” In a nutshell: Travel takes time, how patient can you be? So What’s better?
  • 34. “Delay sensitivity framework” ● Networks latency is lumpy / uncertain ● Very few scenarios entail actually reliable networks - packet loss is common ● Understanding lower bounds for different consistency models So What’s better?
  • 35. Proposed Terminology ● Availability - Empirical measurements only! ● Delay Sensitive - How patient can we be for a given operation? ● Network Faults - All kinds of weird shit happens on networks. ● Fault Tolerance - Under what specific failure modes can we guarantee our invariants? ● Consistency - Not just one, but many different consistency models to choose from So What’s better?
  • 36. Strong vs Weak Consistency models galore: Consistency models are a continuum. “Strong” and “Weak” are conversational terms, not formal. “Weak” does not mean unsafe. It’s all about selecting which invariants you want. Kyle Kingsbury 2014 – https://aphyr.com/posts/313-strong-consistency-models
  • 37. In Conclusion: ● Irrefutable truths mayy be predicated on misleading definitions ● consistency is subjective. ● CAP has outlived it’s usefulness ● We can do better! ● Let’s reason about latency ● Lets fix our terminology ● Rigor is good, CS needs moar of it
  • 38. Thank you! Papers We Love San Diego
  • 39. A Critique of the CAP Theorem