EXPLOITING LOOPHOLES
IN CAP
MichaelT. Nygard	

Cognitect, Inc.
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese an...
Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread o...
About CAP	

!
a.k.a. Brewer’s Conjecture	

!
a.k.a. Theorem that Shipped

1,000 Launches
“Brewer's conjecture and the
feasibility of consistent, available,
partition-tolerant web services.”	

!
Seth Gilbert and ...
Consistency
Availability
Partition-Tolerance
Consistency
Availability
Partition-Tolerance
ChooseTwo
BEWARE BAD LOGIC
⊭¬C → A
C ∩ P → ¬A
A ∩ P → ¬C
CAP	

Gödel’s IncompletenessTheorem	

Heisenberg’s Uncertainty Principle
CAP
GIT
HUP
They’re a drag!
The Network
Asynchronous	

Message-passing	

Network
Consistency
n2
→ write 12
← ack
→ read
← 12
→ read
← 12
→ write 20
← ack
n2
write 12
read
read
write 20
n11
write 12
read
read
write 20
n17
write 12
write 9
read
write 20
×
Atomic	

Linearizable
Availability
n2
→ write 12
← ack
→ read
← 12
→ read
→ write 20
Partitioning
G1 = {n1, n2, n3, n4, n5,
n6, n7, n16, n17,
n18, n19, n20}
G2 = {n8, n9, n10, n11,
n12, n13, n14,
n15}
G1 = {n1, n2, n3, n4, n5,
n6, n7, n16, n17,
n18, n19, n20}
G2 = {n8, n9, n10, n11,
n12, n13, n14,
n15}
Theorem
Shared atomic object
Network divided into {G1, G2}
All messages between G1 and G2 are lost
Asynchronous message-passing ne...
Suppose algorithm A meets all 3 of C,A, & P.
v0
time
write	

v1
G1
α1
v0
time
read	

←v0
G2
α2
v0
time
write	

v1
G1
read	

←v0
G2
α = α1 + α2
loophole	

noun	

!
1.A way of escaping a difficulty, especially an
omission or ambiguity in the wording of a
contract or l...
Loophole 1
HQ9+
H Prints “Hello,World!”
Q Prints source text
9 Prints lyrics to 99
bottles+ Increments the register
Distributed HQ9+
H Prints “Hello,World!”
Q Prints source text
9 Prints lyrics to 99 bottles
+
Increments the distributed
register
Loophole 2
Write Once,
ImmutableThereafter
v0
time
G1
G2
read	

←v0
read	

←v0
read	

←v0
read	

←v0
read	

←v0
read	

←v0
read	

←v0
A = v0
time
read A	

←v0
G1
read B	

←w0
G2
new	

C, X
read B	

←w0
read C	

←X
read A	

←v0
B = w0
A = v0
time
read A	

←v0
G1
read B	

←w0
G2
new	

C, X
read B	

←w0
read C	

←X
read A	

←v0
B = w0
A bit of trickery?
Loophole 3
An older definition
of consistency
The data base consists of entities which are
related in certain ways. These relationships
are best thought of as assertion...
Examples of such assertions are:	

“Names is an index for
Telephone_numbers.”	

“The value of Count_of_X gives the
number ...
The data base is said to be consistent if it
satisfies all its assertions. In some cases,
the data base must become tempora...
Consistency is a predicate C on entities and
their values.The predicate is generally not
known to the system but is embodi...
Can this kind of consistency
be maintained in a
distributed system?
V = v0
time
G1
G2
readV	

←v0
read X	

←x1
readV	

←v0
write 

V, v1
write	

X, x1
read X	

←x1
X = x0
write	

X, x1
write...
C
R
D
T
Commutative
Replicated
Data
Type
Loophole 4
Partition A: <Ca, Ga, a1, a2, …, an>
C Consistency predicate over a
G Subset of nodes in network
a Value of variable i
Partition A: <Ca, Ga, a1, a2, …, an>
C Consistency predicate over a
G Subset of nodes in network
a Value of variable i
Partition A: <Ca, Ga, a1, a2, …, an>
Partition B: <Cb, Gb, b1, b2, …, bm>
Ga Gb
WAN
LOHRs	

 LOHRs
Loophole 5
Bounded Consistency
Core Nebula
RDBMS
Memcached
Item Display	

A & P	

Heavy caching
Bid History	

C & P	

Strong consistency
Loophole 6
Stop building
distributed systems
Loophole 7
Get a better network!
Asynchronous message passing
!
That’s UDP!
Semi-synchronous network
Lost messages are detected after time t	

(by a missed acknowledgement)
“Delayed-t Consistency”
A partial ordering P orders all writes, and all reads
with respect to writes.	

!
The value of eve...
“Delayed-t Consistency”
“Eventual Consistency”
Loophole 7
×
Loophole 8
Use the Force
Relativistic Quantum FieldTheory
GPS
Loophole 9
Redefine availability
Normal
Operation
Partition
Detected
Query Available Available
Alter Available
Not
available
ASYMMETRY OFTIME
Send
Request
Send
Request
100 ms 200 ms
ASYMMETRY OFTIME
Send
Request
100 ms 200 ms 300 ms 400 ms 500 ms 600 ms
ASYMMETRY OFTIME
Send
Request
100 ms 200 ms 300 ms 400 ms 500 ms 600 ms
Time
Out
ASYMMETRY OFTIME
9000 100 200 300 400 500 600 700 800
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Time Elapsed (ms)
ConfidenceofResponsebeforeTi...
Send
Request
100 ms 200 ms 300 ms 400 ms 500 ms 600 ms
Time
Out
Response
Arrives
ASYMMETRY OFTIME
To the observer, there is no
difference between “too
slow” and “not there”.
P A C E L C
P
A
C
E
L
C
Partition?
Availability
Consistency
L
C
Yes
vs
A
C
Partition?
Availability
Consistency
Latency
Consistency
Yes No
vs vs
L
C
Loophole 10
OBSERVABLE CONSISTENCY
Porky Pig’s Window Shade
If Porky Pig is looking at the window
shade, it will be down.	

If he is looking away from the wi...
FIRST DIMENSION
X1 = {looking, not looking}
SECOND DIMENSION
X1 = {looking, not looking}
X2 = {shade open, shade closed}
FORBIDDEN STATES
X1 = {looking, not looking}
X2 = {shade open, shade closed}
Back to “consistency” as a
predicate over the state space
time
t11 t12
Back to CAP
None of these make
CAP “untrue”
Some of them operate under
different assumptions.
Some of them are totally
impractical.
Some of them are in
production today.
Finally, I’ll close with this bit of
code:
QHH9Q+++
mtnygard@cognitect.com
@mtnygard
MichaelT. Nygard	

Cognitect, Inc.
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/cap-
loopholes
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Exploiting Loopholes in CAP
Upcoming SlideShare
Loading in...5
×

Exploiting Loopholes in CAP

134

Published on

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1p8xJSS.

Michael Nygard discusses several loopholes in the CAP theorem that can be used to engineer practical, real-world systems with desirable features. Filmed at qconlondon.com.

Michael Nygard has written and co-authored several books, including "Release It!", "Beautiful Architecture", "97 Things Every Software Architect Should Know" and "Java Developer’s Reference". An expert in highly-available, highly-scalable commerce systems, Michael has been professional programmer and architect for over 15 years.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
134
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Exploiting Loopholes in CAP

  1. 1. EXPLOITING LOOPHOLES IN CAP MichaelT. Nygard Cognitect, Inc.
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /cap-loopholes
  3. 3. Presented at QCon London www.qconlondon.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  4. 4. About CAP ! a.k.a. Brewer’s Conjecture ! a.k.a. Theorem that Shipped
 1,000 Launches
  5. 5. “Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services.” ! Seth Gilbert and Nancy Lynch. 
 SIGACT News 33, 2 (June 2002), 51-59. DOI=10.1145/564585.564601 
 http://doi.acm.org/10.1145/564585.564601
  6. 6. Consistency Availability Partition-Tolerance
  7. 7. Consistency Availability Partition-Tolerance ChooseTwo
  8. 8. BEWARE BAD LOGIC ⊭¬C → A C ∩ P → ¬A A ∩ P → ¬C
  9. 9. CAP Gödel’s IncompletenessTheorem Heisenberg’s Uncertainty Principle
  10. 10. CAP GIT HUP
  11. 11. They’re a drag!
  12. 12. The Network
  13. 13. Asynchronous Message-passing Network
  14. 14. Consistency
  15. 15. n2 → write 12 ← ack → read ← 12 → read ← 12 → write 20 ← ack
  16. 16. n2 write 12 read read write 20 n11 write 12 read read write 20 n17 write 12 write 9 read write 20 ×
  17. 17. Atomic Linearizable
  18. 18. Availability
  19. 19. n2 → write 12 ← ack → read ← 12 → read → write 20
  20. 20. Partitioning
  21. 21. G1 = {n1, n2, n3, n4, n5, n6, n7, n16, n17, n18, n19, n20} G2 = {n8, n9, n10, n11, n12, n13, n14, n15}
  22. 22. G1 = {n1, n2, n3, n4, n5, n6, n7, n16, n17, n18, n19, n20} G2 = {n8, n9, n10, n11, n12, n13, n14, n15}
  23. 23. Theorem
  24. 24. Shared atomic object Network divided into {G1, G2} All messages between G1 and G2 are lost Asynchronous message-passing network
  25. 25. Suppose algorithm A meets all 3 of C,A, & P.
  26. 26. v0 time write v1 G1 α1
  27. 27. v0 time read ←v0 G2 α2
  28. 28. v0 time write v1 G1 read ←v0 G2 α = α1 + α2
  29. 29. loophole noun ! 1.A way of escaping a difficulty, especially an omission or ambiguity in the wording of a contract or law that provides a means of evading compliance. ! 2.A small hole or slit in a wall, especially one through which small arms may be fired.
  30. 30. Loophole 1
  31. 31. HQ9+
  32. 32. H Prints “Hello,World!” Q Prints source text 9 Prints lyrics to 99 bottles+ Increments the register
  33. 33. Distributed HQ9+
  34. 34. H Prints “Hello,World!” Q Prints source text 9 Prints lyrics to 99 bottles + Increments the distributed register
  35. 35. Loophole 2
  36. 36. Write Once, ImmutableThereafter
  37. 37. v0 time G1 G2 read ←v0 read ←v0 read ←v0 read ←v0 read ←v0 read ←v0 read ←v0
  38. 38. A = v0 time read A ←v0 G1 read B ←w0 G2 new C, X read B ←w0 read C ←X read A ←v0 B = w0
  39. 39. A = v0 time read A ←v0 G1 read B ←w0 G2 new C, X read B ←w0 read C ←X read A ←v0 B = w0
  40. 40. A bit of trickery?
  41. 41. Loophole 3
  42. 42. An older definition of consistency
  43. 43. The data base consists of entities which are related in certain ways. These relationships are best thought of as assertions about the data.
  44. 44. Examples of such assertions are: “Names is an index for Telephone_numbers.” “The value of Count_of_X gives the number of employees in department X.”
  45. 45. The data base is said to be consistent if it satisfies all its assertions. In some cases, the data base must become temporarily inconsistent in order to transform it to a new consistent state. From "Granularity of Locks and Degrees of Consistency in a 
 Shared Data Base",
 J.N. Gray, R.A. Lorie, G.R. Putzolu, I.L.Traiger, 1976 From "Granularity of Locks and Degrees of Consistency in a 
 Shared Data Base",
 J.N. Gray, R.A. Lorie, G.R. Putzolu, I.L.Traiger, 1976
  46. 46. Consistency is a predicate C on entities and their values.The predicate is generally not known to the system but is embodied in the structure of the transactions. From "Transactions and Consistency in Distributed Database Systems",
 I.L.Traiger, J.N. Gray, C.A. Galtieri, and B.G. Lindsay, 1982
  47. 47. Can this kind of consistency be maintained in a distributed system?
  48. 48. V = v0 time G1 G2 readV ←v0 read X ←x1 readV ←v0 write 
 V, v1 write X, x1 read X ←x1 X = x0 write X, x1 write 
 V, v1
  49. 49. C R D T Commutative Replicated Data Type
  50. 50. Loophole 4
  51. 51. Partition A: <Ca, Ga, a1, a2, …, an> C Consistency predicate over a G Subset of nodes in network a Value of variable i
  52. 52. Partition A: <Ca, Ga, a1, a2, …, an> C Consistency predicate over a G Subset of nodes in network a Value of variable i
  53. 53. Partition A: <Ca, Ga, a1, a2, …, an> Partition B: <Cb, Gb, b1, b2, …, bm>
  54. 54. Ga Gb WAN LOHRs LOHRs
  55. 55. Loophole 5
  56. 56. Bounded Consistency
  57. 57. Core Nebula
  58. 58. RDBMS Memcached
  59. 59. Item Display A & P Heavy caching Bid History C & P Strong consistency
  60. 60. Loophole 6
  61. 61. Stop building distributed systems
  62. 62. Loophole 7
  63. 63. Get a better network!
  64. 64. Asynchronous message passing ! That’s UDP!
  65. 65. Semi-synchronous network Lost messages are detected after time t (by a missed acknowledgement)
  66. 66. “Delayed-t Consistency” A partial ordering P orders all writes, and all reads with respect to writes. ! The value of every read is the one written by the previous write, where “previous” is under P. ! The order in P is consistent with the order of read and write requests at each node. ! If all messages are delivered and an operation θ
  67. 67. “Delayed-t Consistency”
  68. 68. “Eventual Consistency”
  69. 69. Loophole 7 ×
  70. 70. Loophole 8 Use the Force
  71. 71. Relativistic Quantum FieldTheory
  72. 72. GPS
  73. 73. Loophole 9
  74. 74. Redefine availability
  75. 75. Normal Operation Partition Detected Query Available Available Alter Available Not available
  76. 76. ASYMMETRY OFTIME Send Request
  77. 77. Send Request 100 ms 200 ms ASYMMETRY OFTIME
  78. 78. Send Request 100 ms 200 ms 300 ms 400 ms 500 ms 600 ms ASYMMETRY OFTIME
  79. 79. Send Request 100 ms 200 ms 300 ms 400 ms 500 ms 600 ms Time Out ASYMMETRY OFTIME
  80. 80. 9000 100 200 300 400 500 600 700 800 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Time Elapsed (ms) ConfidenceofResponsebeforeTimeout
  81. 81. Send Request 100 ms 200 ms 300 ms 400 ms 500 ms 600 ms Time Out Response Arrives ASYMMETRY OFTIME
  82. 82. To the observer, there is no difference between “too slow” and “not there”.
  83. 83. P A C E L C
  84. 84. P A C E L C
  85. 85. Partition? Availability Consistency L C Yes vs A C
  86. 86. Partition? Availability Consistency Latency Consistency Yes No vs vs L C
  87. 87. Loophole 10
  88. 88. OBSERVABLE CONSISTENCY
  89. 89. Porky Pig’s Window Shade If Porky Pig is looking at the window shade, it will be down. If he is looking away from the window shade, it will be up.
  90. 90. FIRST DIMENSION X1 = {looking, not looking}
  91. 91. SECOND DIMENSION X1 = {looking, not looking} X2 = {shade open, shade closed}
  92. 92. FORBIDDEN STATES X1 = {looking, not looking} X2 = {shade open, shade closed}
  93. 93. Back to “consistency” as a predicate over the state space
  94. 94. time t11 t12
  95. 95. Back to CAP
  96. 96. None of these make CAP “untrue” Some of them operate under different assumptions.
  97. 97. Some of them are totally impractical. Some of them are in production today.
  98. 98. Finally, I’ll close with this bit of code: QHH9Q+++
  99. 99. mtnygard@cognitect.com @mtnygard MichaelT. Nygard Cognitect, Inc.
  100. 100. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/cap- loopholes

×