Beyond (No)SQL
       Sarah Mei
      Pivotal Labs

      @sarahmei
 sarah@pivotallabs.com
?
Fsck CS.
      -DHH
Fsck CS.
      -DHH (paraphrased)
Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
Agenda
Agenda

• The data storage landscape
Agenda

• The data storage landscape
• Relational model & SQL
Agenda

• The data storage landscape
• Relational model & SQL
  RELATIONAL ALGEBRA
Agenda

• The data storage landscape
• Relational model & SQL
  RELATIONAL ALGEBRA
• Evaluating data stores
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
“NoSQL”
“NoSQL”
MongoDB
“NoSQL”
MongoDB

Memcache
“NoSQL”
MongoDB

Memcache

Bigtable
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak

Redis
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak

Redis

Tokyo Cabinet
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak

Redis

T...
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB
                         Document
Cassandra

Project...
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB
                         Document
Cassandra

Project...
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB
                         Document
Cassandra         ...
Photo from mga (http://www.flickr.com/photos/mgiraldo/420642350)
NO.
The Relational Model




Diagram by Wikipedia user AutumnSnow (http://en.wikipedia.org/wiki/File:Relational_model_concepts...
Sets
Sets

• No duplicates
Sets

• No duplicates
• Unordered
More sets
More sets

• TABLE: a set of columns and a set of rows
More sets

• TABLE: a set of columns and a set of rows
• COLUMN: a unique name and a type
More sets

• TABLE: a set of columns and a set of rows
• COLUMN: a unique name and a type
• ROW: a set of name-value pairs
Relational Algebra


Operations you can do on tables (“relations”)
Operations
• projection: subset of available columns
• selection: subset of available rows
• cartesian product
• set union...
Operations
• projection: subset of available columns
• selection: subset of available rows
• cartesian product
• set union...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


Then you select the rows that satisfy the join condition:



 ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
Your choices
Your choices


• Scale the database
Your choices


• Scale the database
• Try a different approach
What SQL Gets You
What SQL Gets You

• Speed
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
What SQL Gets You

• Speed
    (when data is highly structured and small enough)
• Aggregation
• Relational searching
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
• Relational searching
• ACI...
Image by captcreate (http://www.flickr.com/photos/27845211@N02/2662264721)
What SQL Gets You
What SQL Gets You

• Speed
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
What SQL Gets You

• Speed
    (when data is highly structured and small enough)
• Aggregation
• Relational searching
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
• Relational searching
• ACI...
What if you gave up
 data aggregation?
What if you gave up
 data aggregation?

 Document databases:
     MongoDB
     CouchDB
     Riak
What if you gave up
 where clauses?
What if you gave up
 where clauses?
Key-value stores:
     Memcache
     Project Voldemort
     Redis
     Tokyo Cabinet
What if you gave up
  consistency?
What if you gave up
  consistency?


Then things get interesting.
ACID & BASE
ACID & BASE

• Atomicity
ACID & BASE

• Atomicity
• Consistency
ACID & BASE

• Atomicity
• Consistency
• Isolation
ACID & BASE

• Atomicity
• Consistency
• Isolation
• Durability
ACID & BASE

• Atomicity
• Consistency
• Isolation
• Durability
ACID & BASE

• Atomicity     • Basically Available
• Consistency
• Isolation
• Durability
ACID & BASE

• Atomicity     • Basically Available
• Consistency   • Soft state
• Isolation
• Durability
ACID & BASE

• Atomicity     • Basically Available
• Consistency   • Soft state
• Isolation     • Eventually consistent
• ...
Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
Fully ACID                                                        Fully BASE




        Photo by Marcus Vegas (http://www...
Fully ACID                                                        Fully BASE




                                Redis



...
Fully ACID                                                        Fully BASE




  CouchDB                       Redis



...
Fully ACID                                                        Fully BASE




  CouchDB                       Redis    ...
Questions to ask about data

  • Where can I compromise aggregation?
  • Where can I compromise where clauses?
  • Where c...
CAP Theorem
Pick any two:
CAP Theorem
Pick any two:
   • Consistency
CAP Theorem
Pick any two:
   • Consistency
   • Availability
CAP Theorem
Pick any two:
   • Consistency
   • Availability
   • Partition tolerance
Summary
Summary

Every system at scale will have to
compromise consistency at some level.
Summary

Every system at scale will have to
compromise consistency at some level.
Summary

Every system at scale will have to
compromise consistency at some level.


Do it mindfully.
Questions?
Twitter: @sarahmei
Email: sarah@pivotallabs.com
Slides: http://bit.ly/9xS2PK


Please rate this talk! http://bi...
Upcoming SlideShare
Loading in...5
×

Beyond (No)SQL

3,669

Published on

Slides from my RailsConf 2010 talk, Beyond No(SQL). Conceptual tools for evaluating both SQL and NoSQL databases.

Published in: Education, Technology, Design
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,669
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
173
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide






























  • A table is called a “relation” in this model, and it’s defined by a list of column definitions. Each column definition specifies a unique column name and the type of the values that are permitted for that column. The body of the table is a set of rows, which are called tuples in the model. Each row is a set of key-value pairs, or tuple, consisting of the name of the column and the value of the data. Each tuple has to be unique in the table.





  • They transform tables into other tables. Another word for this is “query.”











































  • Beyond (No)SQL

    1. 1. Beyond (No)SQL Sarah Mei Pivotal Labs @sarahmei sarah@pivotallabs.com
    2. 2. ?
    3. 3. Fsck CS. -DHH
    4. 4. Fsck CS. -DHH (paraphrased)
    5. 5. Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
    6. 6. Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
    7. 7. Agenda
    8. 8. Agenda • The data storage landscape
    9. 9. Agenda • The data storage landscape • Relational model & SQL
    10. 10. Agenda • The data storage landscape • Relational model & SQL RELATIONAL ALGEBRA
    11. 11. Agenda • The data storage landscape • Relational model & SQL RELATIONAL ALGEBRA • Evaluating data stores
    12. 12. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
    13. 13. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
    14. 14. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
    15. 15. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
    16. 16. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
    17. 17. “NoSQL”
    18. 18. “NoSQL” MongoDB
    19. 19. “NoSQL” MongoDB Memcache
    20. 20. “NoSQL” MongoDB Memcache Bigtable
    21. 21. “NoSQL” MongoDB Memcache Bigtable CouchDB
    22. 22. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra
    23. 23. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort
    24. 24. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase
    25. 25. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase Riak
    26. 26. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase Riak Redis
    27. 27. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
    28. 28. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
    29. 29. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Document Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
    30. 30. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Document Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
    31. 31. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Document Cassandra Other Project Voldemort Hbase Riak Redis Tokyo Cabinet
    32. 32. Photo from mga (http://www.flickr.com/photos/mgiraldo/420642350)
    33. 33. NO.
    34. 34. The Relational Model Diagram by Wikipedia user AutumnSnow (http://en.wikipedia.org/wiki/File:Relational_model_concepts.png)
    35. 35. Sets
    36. 36. Sets • No duplicates
    37. 37. Sets • No duplicates • Unordered
    38. 38. More sets
    39. 39. More sets • TABLE: a set of columns and a set of rows
    40. 40. More sets • TABLE: a set of columns and a set of rows • COLUMN: a unique name and a type
    41. 41. More sets • TABLE: a set of columns and a set of rows • COLUMN: a unique name and a type • ROW: a set of name-value pairs
    42. 42. Relational Algebra Operations you can do on tables (“relations”)
    43. 43. Operations • projection: subset of available columns • selection: subset of available rows • cartesian product • set union • set intersection • rename
    44. 44. Operations • projection: subset of available columns • selection: subset of available rows • cartesian product • set union • set intersection • rename
    45. 45. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
    46. 46. A Join select * from foo, bar where foo.ipsum = bar.lorem Then you select the rows that satisfy the join condition: bat ipsum cat lorem bat ipsum cat lorem me 5 us 5 me 5 us 5 me 5 them 4 => you 4 them 4 you 4 us 5 you 4 them 4
    47. 47. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
    48. 48. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 8,000,000 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
    49. 49. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 8,000,000 100,000 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
    50. 50. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 8,000,000 100,000 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 800,000,000,000 you 4 us 5 you 4 them 4
    51. 51. Your choices
    52. 52. Your choices • Scale the database
    53. 53. Your choices • Scale the database • Try a different approach
    54. 54. What SQL Gets You
    55. 55. What SQL Gets You • Speed
    56. 56. What SQL Gets You • Speed (when data is highly structured and small enough)
    57. 57. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation
    58. 58. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching
    59. 59. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching • ACID - guaranteed full consistency
    60. 60. Image by captcreate (http://www.flickr.com/photos/27845211@N02/2662264721)
    61. 61. What SQL Gets You
    62. 62. What SQL Gets You • Speed
    63. 63. What SQL Gets You • Speed (when data is highly structured and small enough)
    64. 64. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation
    65. 65. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching
    66. 66. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching • ACID - guaranteed full consistency
    67. 67. What if you gave up data aggregation?
    68. 68. What if you gave up data aggregation? Document databases: MongoDB CouchDB Riak
    69. 69. What if you gave up where clauses?
    70. 70. What if you gave up where clauses? Key-value stores: Memcache Project Voldemort Redis Tokyo Cabinet
    71. 71. What if you gave up consistency?
    72. 72. What if you gave up consistency? Then things get interesting.
    73. 73. ACID & BASE
    74. 74. ACID & BASE • Atomicity
    75. 75. ACID & BASE • Atomicity • Consistency
    76. 76. ACID & BASE • Atomicity • Consistency • Isolation
    77. 77. ACID & BASE • Atomicity • Consistency • Isolation • Durability
    78. 78. ACID & BASE • Atomicity • Consistency • Isolation • Durability
    79. 79. ACID & BASE • Atomicity • Basically Available • Consistency • Isolation • Durability
    80. 80. ACID & BASE • Atomicity • Basically Available • Consistency • Soft state • Isolation • Durability
    81. 81. ACID & BASE • Atomicity • Basically Available • Consistency • Soft state • Isolation • Eventually consistent • Durability
    82. 82. Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
    83. 83. Fully ACID Fully BASE Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
    84. 84. Fully ACID Fully BASE Redis Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
    85. 85. Fully ACID Fully BASE CouchDB Redis Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
    86. 86. Fully ACID Fully BASE CouchDB Redis Bigtable Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
    87. 87. Questions to ask about data • Where can I compromise aggregation? • Where can I compromise where clauses? • Where can I compromise consistency? Where can I localize consistency?
    88. 88. CAP Theorem Pick any two:
    89. 89. CAP Theorem Pick any two: • Consistency
    90. 90. CAP Theorem Pick any two: • Consistency • Availability
    91. 91. CAP Theorem Pick any two: • Consistency • Availability • Partition tolerance
    92. 92. Summary
    93. 93. Summary Every system at scale will have to compromise consistency at some level.
    94. 94. Summary Every system at scale will have to compromise consistency at some level.
    95. 95. Summary Every system at scale will have to compromise consistency at some level. Do it mindfully.
    96. 96. Questions? Twitter: @sarahmei Email: sarah@pivotallabs.com Slides: http://bit.ly/9xS2PK Please rate this talk! http://bit.ly/9MCtX9
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×