The Potential Dangers of Causal Consistency and an Explicit Solution - SOCC 2012

the potentialdangersof causalconsistencyand an explicitsolutionPeter Bailis, Alan Fekete, Ali Ghodsi,Joseph M. Hellerstein, Ion StoicaUC Berkeley SOCC 2012

A Story

A Story “Sally’s in a coma!”

A Story “Sally’s in “Sally’s a coma!” okay!”

A Story “Sally’s in “Sally’s a coma!” okay!” “Great news!”

The Wrong Story “Sally’s in “Sally’s a coma!” okay!” “Great news!”

The Wrong Story “Sally’s in a coma!” “Great news!”

The Wrong Story “Sally’s in ??? a coma!” “Great news!”

The Right Order“Sally’s in “Sally’s “Great a coma!” okay!” news!”

CAUSALITY: TL;DRMETADATA AND MODERN SERVICESTHE SITE SCALABILITY WALLSCALING WITH EXPLICIT CAUSALITY

1978 “Time, Clocks, and the Ordering of Events in a Distributed System” by Leslie Lamport

1978 “Time, Clocks, and the Ordering of Events in a Distributed System” by Leslie Lamport1994 “Causal Memory” by Ahamad et al.

1978 “Time, Clocks, and the Ordering of Events in a Distributed System” by Leslie Lamport1994 “Causal Memory” by Ahamad et al.2011-12

2011-12

2011-12There aremany hardtrade-offs, IEEE CAP

2011-12 butThere are causalmany hard consistencytrade-offs, can work well IEEE CAP SOSP: COPS

2011-12 butThere are and it’s causalmany hard the best consistencytrade-offs, you can can work do*! well IEEE CAP SOSP: COPS Texas: CAC

“Sally’s okay!”

“Sally’s “Great okay!” news!”

Deliver writesin the “right”order

“Great Deliver writes news!” in the “right”“Sally’s okay!” order “Sally’sin a coma!”

Causal Consistency reads obey partial order called “happens-before”

Write value

Write valueHappens-beforemetadata

Happens-beforemetadata

Happens-before metadata

Happens-before metadata per-DC vector clocks O(datacenters)

Happens-before metadataLESS PARALLELISM per-DC vector clocks O(datacenters)

Happens-before metadataLESS PARALLELISM per-DC vector clocks O(datacenters) per-key vector clock (dependency set) O(keys)

Happens-before metadataLESS PARALLELISM per-DC vector clocks O(datacenters) per-key vector clock (dependency set) O(keys)MORE PARALLELISM

Happens-before metadataLESS PARALLELISM per-DC vector clocks O(datacenters) per-server vector clocks O(servers) per-key vector clock (dependency set) O(keys)MORE PARALLELISM

Have we seen alldependencies?

YHave we seen alldependencies?

Have we seen alldependencies?

YHave we seen alldependencies?

Have we seen alldependencies? Yes: apply write

Have we seen alldependencies? Yes: apply write No: wait

Sounds great, right?

Causal Consistency reads obey partial order called “happens-before”

Causal Consistency reads obey partial order called “happens-before” traditional happens-before: potential causality

If you wish to make anapple pie from scratch,you must first inventthe universe. -Carl Sagan

“Great news!”

“Great “Sally’s news!” okay!”

“Great “Sally’s news!” okay!” “Rad “Great “Can’t wait “Want to go “Lol!” for SOCC!” skiing?”party!” picture!”

“Great “Sally’s news!” okay!” “Rad “Great “Can’t wait “Want to go “Lol!” for SOCC!” skiing?”party!” picture!” “You “I hope my “I love “Coming tonight? who?” paper gets in.” Tahoe!”

“Great “Sally’s news!” okay!” “Rad “Great “Can’t wait “Want to go “Lol!” for SOCC!” skiing?”party!” picture!” “You “I hope my “I love “Coming tonight? who?” paper gets in.” Tahoe!” “You.” “Hello, “I hear the “Snow PC is great!” rocks!”

“Great “Sally’s news!” okay!” “Rad “Great “Can’t wait “Want to go “Lol!” for SOCC!” skiing?” party!” picture!” “You “I hope my “I love “Coming tonight? who?” paper gets in.” Tahoe!” “You.” “Hello, “I hear the “Snow PC is great!” rocks!” “Who’s there?” “Have you “Are you “Greatmet Larry?” submitting?” food here”

“Great “Sally’s news!” okay!” “Rad “Great “Can’t wait “Want to go “Lol!” for SOCC!” skiing?” party!” picture!” “You “I hope my “I love “Coming tonight? who?” paper gets in.” Tahoe!” “You.” “Hello, “I hear the “Snow PC is great!” rocks!” “Who’s there?” “Have you “Are you “Greatmet Larry?” submitting?” food here” “Knock, knock.”

Twitter.com20 Tweetsat login20 moreauto-scroll600+ per min

Twitter.comquicklyapproachesupper bound

r e c a p

p a cerpotential dangerPotential causality graphs are huge,limiting local apply rates and throughput

DANGER!DANGER!

DANGER!DANGER!Unstable queue!

DANGER!DANGER! Unstable queue!Sustained throughputlimited to slowest DC

DANGER!Sustained throughputlimited to slowest DC

DANGER! Sustained throughput limited to slowest DCAdding DCs doesn’t helpslowest site

DANGER! Sustained throughput limited to slowest DCDuring partitions/failures,sustainable throughput iszero writes/s

DANGER!During partitions/failures,sustainable throughput iszero writes/s

DANGER!During partitions/failures,sustainable throughput iszero writes/sMetadata garbage collectionstallsstalls

r e c a p

p a cerpotential dangerWrite throughput limited to slowest DCViolation arbitrarily high visibility latencyAdding DCs does not increase throughput

bad newsPotential dangers spell trouble for causality

bad newsPotential dangers spell trouble for causalityto Are it? live destined without we

bad newsPotential dangers spell trouble for causalityAre we destined to live without it?

ObservationWhy track dependencies that don’t matter?

ObservationWhy track dependencies that don’t matter?Need to know application semantics

Explicit Causality app-defined “happens-before” transitivity enforced subset of potential causality

Explicit Causality app-defined “happens-before” transitivity enforced subset of potential causality not a new idea (e.g., Cheriton and Skeen SOSP 1993, Ladin et al. PODC 1990) but...

Explicit Matters

Explicit Matters Twitter 28% of Tweets in conversations 69% of convos are depth two average depth is 10.7 [Ye and Wu SocInfo 2010, Ritter et al. HLT 2010]

Explicit Matters Twitter 28% of Tweets in conversations 69% of convos are depth two average depth is 10.7 [Ye and Wu SocInfo 2010, Ritter et al. HLT 2010] reply-to degree and depth are limited

Explicit Matters Twitter 28% of Tweets in conversations 69% of convos are depth two average depth is 10.7 [Ye and Wu SocInfo 2010, Ritter et al. HLT 2010] reply-to degree and depth are limited 109 smaller graph for a year of Tweets

Explicit API put(key, value)

Explicit API put_after(key, value, deps)

Explicit API put_after(key, value, deps) (possibly empty) set of references to other writes

Explicit API put_after(key, value, deps) (possibly empty) set of references to other writes track what matters

Explicit API put_after(key, value, deps) (possibly empty) set of references to other writes track what matters frequently in data model already

Explicit API put_after(key, value, deps) (possibly empty) set of references to other writes track what matters frequently in data model already can simulate fencing

Explicit API put_after(key, value, deps) (possibly empty) set of references to other writes track what matters frequently in data model already can simulate fencing won’t track non-explicit references

Extra Note: paper for more details

Extra Note: can treat foreign key references in relational data model as explicit dependencies automatically! can use schema to describe explicit deps... paper for more details

NATURAL EXTENSIONNOT ROCKET SCIENCE

NATURAL EXTENSIONNOT ROCKET SCIENCE http://twitter.com/rbranson/status/256795094387142657

potentialcausality

A WORLD OF PHYSICALOPERATIONS MESSAGES REGISTERS READS WRITES MODEST SCALE

A WORLD OF HUGE SCALE MASSIVEINTERACTIVITY PLANET-WIDE NETWORKS REAL-WORLD INTERFACES UBICOMP BIG DATA

ia l nt s e r t e o gp nda

ia l t e nt s e r o gp n phs a ausality grad ec h ug

ia l t e nt s e r o gp n phs mited gra y li a ausality bilitd ec c ala h ug ts oug hpu thr

ia l t e nt s e r o gp n phs mited gra y li a ausality bilitd ec c ala h ug ts oug hpu thr explicit causality semantic context to the rescue consider modern apps helps with #1, indirectly with #2

Peter Bailis @pbailis pbailis@cs.berkeley.edu http://bailis.org/http://bailis.org/papers/explicit-socc2012.pdf

The Potential Dangers of Causal Consistency and an Explicit Solution - SOCC 2012

by Peter Bailis on Oct 23, 2012

Accessibility

Upload Details

Usage Rights

Statistics

The Potential Dangers of Causal Consistency and an Explicit Solution - SOCC 2012 Presentation Transcript