0
Eventual Consistency != Hopeful
Consistency
Embracing Optimistic Design in the
Persistence Layer
Who am I?
Christos Kalantzis
Netflix Inc.
Engineering Manager – Cloud Database Engineering
@chriskalan
ckalantzis@netflix....
C* Replication & Consistency Recap
• C* is eventually Consistent
– That means it WILL get there…eventually
• Eventually is...
• C* has tunable consistency
– All
– Quorum
– One
C* Replication & Consistency Recap
Remember When...?
• In early 2000s, Read was scaled out by adding
replication Slaves to a Master DB.
• Writes went to a si...
Remember When…?
• Slaves could lose transactions
Remember When…?
• No “Repair” function
Remember When…?
• We trusted it
• We still trust it
C* Consistency Concerns
• “I want high consistency in my Reads/Writes just like I
had in my RDBMS setup”
– You never reall...
Netflix Experiment
• Created a multi-datacenter C* 1.1.7 cluster of 48 nodes in
each DC
• Put load on C* Cluster
– 100K to...
Netflix Experiment
Optimistic vs. Pessimistic Design
• Pessimistic Design
– Design with high consistency, you punish your
users 99.9% of the ...
Optimistic vs. Pessimistic Design
• Optimistic Design
– Trust your data store
• Know your business and your application
– ...
Low Consistency Example 1
Retail
Low Consistency Example 2
Banking
Hurdles Faced
• Engineers are stubborn!
– 1+1=2 .. not eventually 2
• Middle management is scared
– It’s a feat you convin...
How to Overcome those Hurdles
• Prove it through a POC
• Show them the benefits of an improved user
experience
– Its all a...
Q&A
Thank you!
Upcoming SlideShare
Loading in...5
×

Webinar: Eventual Consistency != Hopeful Consistency

1,350

Published on

This session will address Cassandra's tunable consistency model and cover how developers and companies should adopt a more Optimistic Software Design model.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,350
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Webinar: Eventual Consistency != Hopeful Consistency"

  1. 1. Eventual Consistency != Hopeful Consistency Embracing Optimistic Design in the Persistence Layer
  2. 2. Who am I? Christos Kalantzis Netflix Inc. Engineering Manager – Cloud Database Engineering @chriskalan ckalantzis@netflix.com www.linkedin.com/in/christoskalantzis
  3. 3. C* Replication & Consistency Recap • C* is eventually Consistent – That means it WILL get there…eventually • Eventually is not: – A day from now – A minute from now – A second from now • In most cases it is milliseconds
  4. 4. • C* has tunable consistency – All – Quorum – One C* Replication & Consistency Recap
  5. 5. Remember When...? • In early 2000s, Read was scaled out by adding replication Slaves to a Master DB. • Writes went to a single Master, Reads went to Slaves
  6. 6. Remember When…? • Slaves could lose transactions
  7. 7. Remember When…? • No “Repair” function
  8. 8. Remember When…? • We trusted it • We still trust it
  9. 9. C* Consistency Concerns • “I want high consistency in my Reads/Writes just like I had in my RDBMS setup” – You never really had it. • “I want my DB to catch integrity issues” – We’ve been turning FK off for years! • See Rails, Grails & other MVC frameworks • Can I trust that C* will replicate my data when writing at CL 1”
  10. 10. Netflix Experiment • Created a multi-datacenter C* 1.1.7 cluster of 48 nodes in each DC • Put load on C* Cluster – 100K total operations per second (50 K in each DC) • Wrote 1,000,000 records at CL1 in one DC • Read same 1,000,000 records at CL1 in other DC • ALL records were read successfully • You can trust it!
  11. 11. Netflix Experiment
  12. 12. Optimistic vs. Pessimistic Design • Pessimistic Design – Design with high consistency, you punish your users 99.9% of the time • Higher consistency = higher latency • Diminished user experience
  13. 13. Optimistic vs. Pessimistic Design • Optimistic Design – Trust your data store • Know your business and your application – Always ask yourself, is it really that important? • Handle edge cases through contingency plans
  14. 14. Low Consistency Example 1 Retail
  15. 15. Low Consistency Example 2 Banking
  16. 16. Hurdles Faced • Engineers are stubborn! – 1+1=2 .. not eventually 2 • Middle management is scared – It’s a feat you convinced them to use C* – Now you got to convince them to accept low consistency? • Engaging Product team to implement some type of contingency plans
  17. 17. How to Overcome those Hurdles • Prove it through a POC • Show them the benefits of an improved user experience – Its all about the user • You may be working for the wrong company – We’re hiring! – jobs.netflix.com
  18. 18. Q&A
  19. 19. Thank you!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×