consistency without borders
Peter Alvaro, Peter Bailis,
Neil Conway, Joseph M. Hellerstein
UC Berkeley
The transaction concept
	
  DEBIT_CREDIT:	
  	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  BEGIN_TRANSACTION;	
  	
  
	
  	...
The transaction concept
	
  DEBIT_CREDIT:	
  	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  BEGIN_TRANSACTION;	
  	
  
	
  	...
An application-level contract
Write	
   Read	
  
Application
Opaque
store
Transactions
Pervasive distribution
Pervasive distribution
partial failure
Pervasive distribution
partial failure
asynchrony
Pervasive distribution
CAP
partial failure
asynchrony
Research on consistency
Write	
   Read	
  
Application
Opaque
store
Consistency
models
R1(X=1)	
  R2(X=1)	
  W1(X=2)	
  W2...
Research on consistency
Write	
   Read	
  
Application
Opaque
store
Consistency
models
Assert:
balance > 0
causal?
PRAM?
d...
Meanwhile, in industry…
Application
Opaque
store
Custom solutions
Write	
   Read	
  
Meanwhile, in industry…
Application
Opaque
store
Custom solutions
Assert:
balance > 0
(prayer)
Write	
   Read	
  
Distributed consistency:
staying relevant
•  Is this an important problem?
•  Is academia disconnected from reality?
•  OK...
Case study: a graph
T1
T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Partitioned, for scalability
T1
T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Replicated, for availability
T1
T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
T1
T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Problem: deadlock detection
Task: Identify strongly-connected
components
Waits-for graph
T1
T2
T4
T10
T3
T6
T5
T9
T7
T8
T1...
Problem: garbage collection
Task: Identify nodes not reachable
from Root. Root	
  
Refers-to graph
T1
T2
T4
T10
T3
T6
T5
T...
T1
T2
T4
T10
T3
T6
T5
T9
T7
T8
T11
T12
T13
T14
Correctness
Deadlock detection
•  Safety: No false positives-
•  Liveness: ...
Consistency at the extremes
Storage
Object
Flow
Language
Application
Linearizable
key-value store?
	
  
	
  Custom solutio...
Consistency at the extremes
Storage
Object
Flow
Language
Application
Linearizable
key-value store?
	
  
	
  Custom solutio...
Consistency across the stack
Object-level consistency
Capture semantics of data structures that
•  allow greater concurrency
•  maintain guarantees (e....
Insert	
   Read	
  
Convergent
data structure
(e.g., Set CRDT)
Object-level consistency
Insert	
   Read	
  
Commutativity
...
Application
Convergent
data structures
Object-level consistency
?	
   ?	
  
GC Assert:
No live nodes are reclaimed
Assert:...
Flow-level consistency	
  
Storage
Object
Flow
Language
Application
Flow-level consistency	
  
Capture semantics of data in motion
•  Asynchronous dataflow model
•  component properties à sy...
Flow-level consistency
Order-insensitivity (confluence)
output	
  set	
  =	
  f(input	
  set)	
  	
  	
  
{	
  	
  	
  	
  ...
Flow-level consistency
Confluence is compositional
output	
  set	
  =	
  f	
  Ÿ	
  g(input	
  set)	
  	
  	
  
Flow-level consistency
Confluence is compositional
output	
  set	
  =	
  f	
  Ÿ	
  g(input	
  set)	
  	
  	
  
Graph
store
Memory
allocator
Transitive
closure
Garbage
collector
Confluent Not
Confluent
Confluent
Graph
store
Transactio...
Graph
store
Memory
allocator
Transitive
closure
Garbage
collector
Confluent Not
Confluent
Confluent
Graph
store
Transactio...
Language-level consistency	
  
DSLs for distributed programming?
•  Capture consistency concerns in the
type system
	
  
	...
Language-level consistency	
  
CALM Theorem:
Monotonic à confluent
Conservative, syntactic test for confluence
	
  
Language-level consistency
Deadlock detector
Garbage collector
Language-level consistency
Deadlock detector
Garbage collector
nonmonotonic	
  
Storage
Object
Flow
Language
Application
correct
reusable
efficient
intuitive
Where we’ve been;
where we’re headed
Storage
Object
Flow
Language
Application
correct
reusable
efficient
intuitive
Where we’ve been;
where we’re headed
Storage
Object
Flow
Language
Application
correct
reusable
efficient
intuitive
Where we’ve been;
where we’re headed
Remember
•  Consistency is an application-level property
•  Correctness and performance are compatible
•  Meet programmers...
Queries?
Upcoming SlideShare
Loading in …5
×

consistency without borders

1,128 views

Published on

talk at SoCC'13

Published in: Technology, Business
1 Comment
0 Likes
Statistics
Notes
  • updated to clean up animations
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total views
1,128
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
13
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • 2 things happened: academia ran down the stack, creating new I/O-level models with no accompanying intuitive contract for apps.Industry ran up the stack, developing best practices for app-level consistency without support (or guarantees!) from infrastructure
  • Recover common ground with programmers that was lost with the abandonment of xacts
  • To guard against failures, we replicate.NB: asynchrony => replicas might not agree
  • Very similar looking criteria (1 safe 1 live). Takes some work, even on a single site. But hard in our scenario: disorder => replica disagreement, partial failure => missing partitions
  • recall “the developer’s conundrum.” essentially 3 choices: 1) SC, meaningful at app level,
  • Ie, reorderability, batchability, tolerance to duplication / retryNow programmer must map from application invariants to object API (with richer semantics than read/write).
  • All API calls to add() commute; no CC necessaryApproach: adjacency lists as setsHowever, not sufficient to synchronize GC.
  • However, not sufficient to synchronize GC.Perhaps more importantly, not *compositional* -- what guarantees does my app – pieced together from many convergent objects – give?To reason compositionally, need guarantees about what comes OUT of my objects, and how it transits the app.
  • We are interested in the properties of component *outputs* rather than just internal state. Hence we are interested in a different property: confluence.A confluent module behaves like a function from sets (of inputs) to sets (of outputs)
  • Confluence is compositional: Composing confluent components yields a confluent dataflow
  • All of these components are confluent! Composing confluent components yields a confluent dataflowBut annotations are burdensome
  • M – a semantic property of code – implies confluenceAn appropriately constrained language provides a conservative syntactic test for M.
  • M – a semantic property of code – implies confluenceAn appropriately constrained language provides a conservative syntactic test for M.
  • Also note that a data-centric language give us the dataflow graph automatically, via dependencies (across LOC, modules, processes, nodes, etc)
  • Consistency models and mechanisms are only useful if they help programmers write correct applications.
  • consistency without borders

    1. 1. consistency without borders Peter Alvaro, Peter Bailis, Neil Conway, Joseph M. Hellerstein UC Berkeley
    2. 2. The transaction concept  DEBIT_CREDIT:                        BEGIN_TRANSACTION;                        GET  MESSAGE;                        EXTRACT  ACCOUT_NUMBER,  DELTA,  TELLER,  BRANCH                                FROM  MESSAGE;                        FIND  ACCOUNT(ACCOUT_NUMBER)  IN  DATA  BASE;                        IF  NOT_FOUND    |  ACCOUNT_BALANCE  +  DELTA  <  0  THEN                                  PUT  NEGATIVE  RESPONSE;                        ELSE  DO;                                  ACCOUNT_BALANCE  =  ACCOUNT_BALANCE  +  DELTA;                                  POST  HISTORY  RECORD  ON  ACCOUNT  (DELTA);                                  CASH_DRAWER(TELLER)  =  CASH_DRAWER(TELLER)  +  DELTA;                                  BRANCH_BALANCE(BRANCH)  =  BRANCH_BALANCE(BRANCH)  +  DELTA;                                  PUT  MESSAGE  ('NEW  BALANCE  ='  ACCOUNT_BALANCE);                                  END;                        COMMIT;    
    3. 3. The transaction concept  DEBIT_CREDIT:                        BEGIN_TRANSACTION;                        GET  MESSAGE;                        EXTRACT  ACCOUT_NUMBER,  DELTA,  TELLER,  BRANCH                                FROM  MESSAGE;                        FIND  ACCOUNT(ACCOUT_NUMBER)  IN  DATA  BASE;                        IF  NOT_FOUND    |  ACCOUNT_BALANCE  +  DELTA  <  0  THEN                                  PUT  NEGATIVE  RESPONSE;                        ELSE  DO;                                  ACCOUNT_BALANCE  =  ACCOUNT_BALANCE  +  DELTA;                                  POST  HISTORY  RECORD  ON  ACCOUNT  (DELTA);                                  CASH_DRAWER(TELLER)  =  CASH_DRAWER(TELLER)  +  DELTA;                                  BRANCH_BALANCE(BRANCH)  =  BRANCH_BALANCE(BRANCH)  +  DELTA;                                  PUT  MESSAGE  ('NEW  BALANCE  ='  ACCOUNT_BALANCE);                                  END;                        COMMIT;    
    4. 4. An application-level contract Write   Read   Application Opaque store Transactions
    5. 5. Pervasive distribution
    6. 6. Pervasive distribution partial failure
    7. 7. Pervasive distribution partial failure asynchrony
    8. 8. Pervasive distribution CAP partial failure asynchrony
    9. 9. Research on consistency Write   Read   Application Opaque store Consistency models R1(X=1)  R2(X=1)  W1(X=2)  W2(X=0)   W1(X=1)  W1(Y=2)  R2(Y=2)  R2(X=0)  
    10. 10. Research on consistency Write   Read   Application Opaque store Consistency models Assert: balance > 0 causal? PRAM? delta? fork/join? red/blue? release? SC? (translation) R1(X=1)  R2(X=1)  W1(X=2)  W2(X=0)   W1(X=1)  W1(Y=2)  R2(Y=2)  R2(X=0)  
    11. 11. Meanwhile, in industry… Application Opaque store Custom solutions Write   Read  
    12. 12. Meanwhile, in industry… Application Opaque store Custom solutions Assert: balance > 0 (prayer) Write   Read  
    13. 13. Distributed consistency: staying relevant •  Is this an important problem? •  Is academia disconnected from reality? •  OK, what now? Goal: help programmers write correct applications. Today: some promising approaches
    14. 14. Case study: a graph T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14
    15. 15. Partitioned, for scalability T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14
    16. 16. Replicated, for availability T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14
    17. 17. Problem: deadlock detection Task: Identify strongly-connected components Waits-for graph T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14
    18. 18. Problem: garbage collection Task: Identify nodes not reachable from Root. Root   Refers-to graph T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14
    19. 19. T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 Correctness Deadlock detection •  Safety: No false positives- •  Liveness: Identify all deadlocks Garbage collection •  Safety: Never GC live memory! •  Liveness: GC all orphaned memory T1 T2 T4 T10 T3 T6 T5 T9 T7 T8 T11 T12 T13 T14 ParLLon   Root  
    20. 20. Consistency at the extremes Storage Object Flow Language Application Linearizable key-value store?    Custom solutions?
    21. 21. Consistency at the extremes Storage Object Flow Language Application Linearizable key-value store?    Custom solutions? Efficient Correct
    22. 22. Consistency across the stack
    23. 23. Object-level consistency Capture semantics of data structures that •  allow greater concurrency •  maintain guarantees (e.g. convergence) Storage Object Flow Language Application
    24. 24. Insert   Read   Convergent data structure (e.g., Set CRDT) Object-level consistency Insert   Read   Commutativity Associativity Idempotence Reordering Batching Retry/duplication Tolerant to
    25. 25. Application Convergent data structures Object-level consistency ?   ?   GC Assert: No live nodes are reclaimed Assert: Graph replicas converge
    26. 26. Flow-level consistency   Storage Object Flow Language Application
    27. 27. Flow-level consistency   Capture semantics of data in motion •  Asynchronous dataflow model •  component properties à system-wide guarantees Graph store Transaction manager Transitive closure Deadlock detector Confluent ConfluentConfluent
    28. 28. Flow-level consistency Order-insensitivity (confluence) output  set  =  f(input  set)       {                }   {                }   =  
    29. 29. Flow-level consistency Confluence is compositional output  set  =  f  Ÿ  g(input  set)      
    30. 30. Flow-level consistency Confluence is compositional output  set  =  f  Ÿ  g(input  set)      
    31. 31. Graph store Memory allocator Transitive closure Garbage collector Confluent Not Confluent Confluent Graph store Transaction manager Transitive closure Deadlock detector Confluent ConfluentConfluent Graph queries as dataflow
    32. 32. Graph store Memory allocator Transitive closure Garbage collector Confluent Not Confluent Confluent Graph store Transaction manager Transitive closure Deadlock detector Confluent ConfluentConfluent Graph queries as dataflow Confluent Coordinate  here  
    33. 33. Language-level consistency   DSLs for distributed programming? •  Capture consistency concerns in the type system     Storage Object Flow Language Application
    34. 34. Language-level consistency   CALM Theorem: Monotonic à confluent Conservative, syntactic test for confluence  
    35. 35. Language-level consistency Deadlock detector Garbage collector
    36. 36. Language-level consistency Deadlock detector Garbage collector nonmonotonic  
    37. 37. Storage Object Flow Language Application correct reusable efficient intuitive Where we’ve been; where we’re headed
    38. 38. Storage Object Flow Language Application correct reusable efficient intuitive Where we’ve been; where we’re headed
    39. 39. Storage Object Flow Language Application correct reusable efficient intuitive Where we’ve been; where we’re headed
    40. 40. Remember •  Consistency is an application-level property •  Correctness and performance are compatible •  Meet programmers on their home turf •  Build bridges!
    41. 41. Queries?

    ×