NoSQL Rollercoaster

2,473 views

Published on

Slides used during my JFall 2010 talk

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,473
On SlideShare
0
From Embeds
0
Number of Embeds
176
Actions
Shares
0
Downloads
35
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

NoSQL Rollercoaster

  1. 1. A NoSQL Rollercoaster Saturday, November 6, 2010
  2. 2. Saturday, November 6, 2010
  3. 3. IMS The Hierarchical Database (1966) Vern Watts Saturday, November 6, 2010
  4. 4. “A Relational Model for Large Shared Databanks” (1970) Ted Codd Saturday, November 6, 2010
  5. 5. “"SEQUEL: A Structured English Query Language" Don Chamberlin & Ray Boyce (1974) Don Chamberlin Saturday, November 6, 2010
  6. 6. ACID (late 1970’s) Jim Gray Saturday, November 6, 2010
  7. 7. “NoSQL” coined (1998) Carlo Strozzi Saturday, November 6, 2010
  8. 8. “NoSQL” Reintroduced (2008) Eric Evans Saturday, November 6, 2010
  9. 9. select fun, profit from real_world where relational=false? Saturday, November 6, 2010
  10. 10. “In striving to make every user happy, a technology an actually leave the majority unhappy.” “Every good idea is generalized to its level of inapplicability.” (Peter Principle) Saturday, November 6, 2010
  11. 11. Saturday, November 6, 2010
  12. 12. Why? Saturday, November 6, 2010
  13. 13. Internet Scale • Massive data collections • Huge number of requests • Coming from geographic areas across the globe • 24/7 Saturday, November 6, 2010
  14. 14. Total Cost of Ownership • The price of a license • The price of support • The price of hardware Saturday, November 6, 2010
  15. 15. Availability Saturday, November 6, 2010
  16. 16. Data Models Saturday, November 6, 2010
  17. 17. Column Oriented … key named column named column named column named column named column Column Family ≈ Table Empty cells are cheap (sparse table) Can grow “indefinitely” Schemaless No secundary indexes Saturday, November 6, 2010
  18. 18. Column Oriented + Super Columns … key named column named column named column named column named column … named column named column named column … Super Columns Saturday, November 6, 2010
  19. 19. Key Value Store •Schemaless •Versioning 1011 0110 Saturday, November 6, 2010
  20. 20. Graph Database SPARQL? Saturday, November 6, 2010
  21. 21. Document Store Improved Indexing <persons> <person> <name>Wilfred</name> <surname>Springer </person> … </persons> [{ "Name" : "Wilfred", "Surname" : "Springer"}, … ] JSON XML Serverside Processing Saturday, November 6, 2010
  22. 22. Challenges Saturday, November 6, 2010
  23. 23. CAP Theorem Eric Brewer Saturday, November 6, 2010
  24. 24. Availability Consistency Partition Tolerance Pick two Saturday, November 6, 2010
  25. 25. Consistency Consistency Strong Consistency Weak Consistency Eventual Consistency Other Causal Consistency Read-your- writes Consistency Session Consistency Monotonic Read Consistency Monotonic Write Consistency Saturday, November 6, 2010
  26. 26. Strong Consistency A B C 1 2 2 2 0 value = "foo" value = "bar" value = "bar" value = "bar" value = "bar" After the update, any subsequent access will return the updated value. Saturday, November 6, 2010
  27. 27. Weak Consistency A B C 1 value = "bar" value = "bar" / "foo" value = "bar" / "foo" value = "bar" / "foo" 0 value = "foo" >1 >1 >1 The system does not guarantee that at any given point in the future subsequent access will return the updated value Saturday, November 6, 2010
  28. 28. Eventual Consistency If no updates are made to the object, eventually all accesses will return the last updated value. A B C 1 value = "bar" value = "bar" value = "bar" value = "bar" 0 value = "foo" t t t t ≥ 1 Saturday, November 6, 2010
  29. 29. Session Consistency Within the “session”, the system guarantees read-your- writes consistency 2 value = "foo" Session 1 Session 2 A B C 1 value = "bar" 0 value = "foo" 2 value = "bar" Saturday, November 6, 2010
  30. 30. Partition Tolerance A writes new value replicates new value reads new value Saturday, November 6, 2010
  31. 31. Partition Tolerance A writes new value replicates new value reads new value ! Saturday, November 6, 2010
  32. 32. Partition Tolerance A writes new value fails to replicate new value reads old value Saturday, November 6, 2010
  33. 33. Partition Tolerance A failing attempt to write a new value fails to replicate new value Saturday, November 6, 2010
  34. 34. Byzantine Failures Saturday, November 6, 2010
  35. 35. Faults that might occur in a distributed system •Lying •Fabrication of messages •Collisions •Selective non-participation Saturday, November 6, 2010
  36. 36. Byzantine Generals Problem Saturday, November 6, 2010
  37. 37. Solution Strategies Saturday, November 6, 2010
  38. 38. Strong Consistency 1 Saturday, November 6, 2010
  39. 39. Παξοί Saturday, November 6, 2010
  40. 40. Leader Proposer Acceptor Learner Four Roles Saturday, November 6, 2010
  41. 41. Paxos in a Nutshell Request Propose Promise Accept Accepted Response Saturday, November 6, 2010
  42. 42. Processors combine various roles Consensus based on majority vote Paxos guarantees progress for 2F + 1 processors in face of failure of F processors Saturday, November 6, 2010
  43. 43. “Either Paxos, Paxos with cruft, or broken” Mike Burrows (Google Chubby) Saturday, November 6, 2010
  44. 44. Partitioning 2 Saturday, November 6, 2010
  45. 45. Consistent Hashing Key K A B C D E F G H Saturday, November 6, 2010
  46. 46. High Availability 3 Saturday, November 6, 2010
  47. 47. Replication A B C D E F G H Saturday, November 6, 2010
  48. 48. W = 3 N = 4 R = 2 A Replication Factor Saturday, November 6, 2010
  49. 49. W = 3 N = 4 R = 2 A Reconciliation during reads Reconciliation Saturday, November 6, 2010
  50. 50. Handling Temporary Failures 4 Saturday, November 6, 2010
  51. 51. Problem Reconcilation ? Alice Ben CathyDave Wednesday? Thursday Tuesday What's it gonna be? Tuesday ThursdayX ? ? ? Saturday, November 6, 2010
  52. 52. Vector Clocks date = Wednesday vclock = Alice:1 date = Tuesday vclock = Alice:1, Ben:1 date = Tuesday vclock = Alice:1, Ben:1, Dave:1 date = Tuesday vclock = Alice:1, Cathy:1 date = Thursday vclock = Alice:1, Ben:1, Cathy:1, Dave:2 Saturday, November 6, 2010
  53. 53. Conflict Resolution date = Tuesday vclock = Alice:1, Ben:1, Dave:1 date = Thursday vclock = Alice:1, Ben:1, Cathy:1, Dave:2 6 Saturday, November 6, 2010
  54. 54. Recovering from permanent failures 5 Saturday, November 6, 2010
  55. 55. What if nodes die permantly? A B C D E F G H Saturday, November 6, 2010
  56. 56. Merkle Trees # # # # # # # # # # # # # # Saturday, November 6, 2010
  57. 57. Enough is enough!!! Saturday, November 6, 2010
  58. 58. Conclusions Saturday, November 6, 2010
  59. 59. Conclusions Scale!!! Saturday, November 6, 2010
  60. 60. Conclusions Scale!!! No free ride Saturday, November 6, 2010
  61. 61. Conclusions Scale!!! No free ride You gain some Saturday, November 6, 2010
  62. 62. Conclusions Scale!!! No free ride You gain some You lose some Saturday, November 6, 2010
  63. 63. Conclusions Scale!!! No free ride You gain some You lose some Choice Saturday, November 6, 2010
  64. 64. Conclusions Scale!!! No free ride You gain some You lose some Choice Control Saturday, November 6, 2010
  65. 65. Conclusions Scale!!! No free ride You gain some You lose some Choice Control Take control Saturday, November 6, 2010
  66. 66. Conclusions Scale!!! No free ride You gain some You lose some Choice Control Take control Understand your options Saturday, November 6, 2010
  67. 67. Conclusions Scale!!! No free ride You gain some You lose some Choice Control Take control Understand your options Fun!!! Saturday, November 6, 2010
  68. 68. Conclusions Scale!!! No free ride You gain some You lose some Choice Control Take control Understand your options Fun!!! Changes your way of thinking Saturday, November 6, 2010
  69. 69. Conclusions Scale!!! No free ride You gain some You lose some Choice Control Take control Understand your options Fun!!! Changes your way of thinking Saturday, November 6, 2010
  70. 70. No SQL!!! wilfredspringer@gmail.com Saturday, November 6, 2010
  71. 71. BASE • Basically Available • Soft State • Eventually Consistent Saturday, November 6, 2010

×