Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Beyond (No)SQL
       Sarah Mei
      Pivotal Labs

      @sarahmei
 sarah@pivotallabs.com
?
Fsck CS.
      -DHH
Fsck CS.
      -DHH (paraphrased)
Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
Agenda
Agenda

• The data storage landscape
Agenda

• The data storage landscape
• Relational model & SQL
Agenda

• The data storage landscape
• Relational model & SQL
  RELATIONAL ALGEBRA
Agenda

• The data storage landscape
• Relational model & SQL
  RELATIONAL ALGEBRA
• Evaluating data stores
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
“NoSQL”
“NoSQL”
MongoDB
“NoSQL”
MongoDB

Memcache
“NoSQL”
MongoDB

Memcache

Bigtable
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak

Redis
“NoSQL”
MongoDB

Memcache

Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak

Redis

Tokyo Cabinet
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB

Cassandra

Project Voldemort

Hbase

Riak

Redis

T...
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB
                         Document
Cassandra

Project...
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB
                         Document
Cassandra

Project...
“NoSQL”
MongoDB

Memcache                 Key-value
Bigtable

CouchDB
                         Document
Cassandra         ...
Photo from mga (http://www.flickr.com/photos/mgiraldo/420642350)
NO.
The Relational Model




Diagram by Wikipedia user AutumnSnow (http://en.wikipedia.org/wiki/File:Relational_model_concepts...
Sets
Sets

• No duplicates
Sets

• No duplicates
• Unordered
More sets
More sets

• TABLE: a set of columns and a set of rows
More sets

• TABLE: a set of columns and a set of rows
• COLUMN: a unique name and a type
More sets

• TABLE: a set of columns and a set of rows
• COLUMN: a unique name and a type
• ROW: a set of name-value pairs
Relational Algebra


Operations you can do on tables (“relations”)
Operations
• projection: subset of available columns
• selection: subset of available rows
• cartesian product
• set union...
Operations
• projection: subset of available columns
• selection: subset of available rows
• cartesian product
• set union...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


Then you select the rows that satisfy the join condition:



 ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
A Join
select * from foo, bar where foo.ipsum = bar.lorem


      First you take the cartesian product....

  foo:   bat  ...
Your choices
Your choices


• Scale the database
Your choices


• Scale the database
• Try a different approach
What SQL Gets You
What SQL Gets You

• Speed
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
What SQL Gets You

• Speed
    (when data is highly structured and small enough)
• Aggregation
• Relational searching
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
• Relational searching
• ACI...
Image by captcreate (http://www.flickr.com/photos/27845211@N02/2662264721)
What SQL Gets You
What SQL Gets You

• Speed
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
What SQL Gets You

• Speed
    (when data is highly structured and small enough)
• Aggregation
• Relational searching
What SQL Gets You

• Speed
   (when data is highly structured and small enough)
• Aggregation
• Relational searching
• ACI...
What if you gave up
 data aggregation?
What if you gave up
 data aggregation?

 Document databases:
     MongoDB
     CouchDB
     Riak
What if you gave up
 where clauses?
What if you gave up
 where clauses?
Key-value stores:
     Memcache
     Project Voldemort
     Redis
     Tokyo Cabinet
What if you gave up
  consistency?
What if you gave up
  consistency?


Then things get interesting.
ACID & BASE
ACID & BASE

• Atomicity
ACID & BASE

• Atomicity
• Consistency
ACID & BASE

• Atomicity
• Consistency
• Isolation
ACID & BASE

• Atomicity
• Consistency
• Isolation
• Durability
ACID & BASE

• Atomicity
• Consistency
• Isolation
• Durability
ACID & BASE

• Atomicity     • Basically Available
• Consistency
• Isolation
• Durability
ACID & BASE

• Atomicity     • Basically Available
• Consistency   • Soft state
• Isolation
• Durability
ACID & BASE

• Atomicity     • Basically Available
• Consistency   • Soft state
• Isolation     • Eventually consistent
• ...
Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
Fully ACID                                                        Fully BASE




        Photo by Marcus Vegas (http://www...
Fully ACID                                                        Fully BASE




                                Redis



...
Fully ACID                                                        Fully BASE




  CouchDB                       Redis



...
Fully ACID                                                        Fully BASE




  CouchDB                       Redis    ...
Questions to ask about data

  • Where can I compromise aggregation?
  • Where can I compromise where clauses?
  • Where c...
CAP Theorem
Pick any two:
CAP Theorem
Pick any two:
   • Consistency
CAP Theorem
Pick any two:
   • Consistency
   • Availability
CAP Theorem
Pick any two:
   • Consistency
   • Availability
   • Partition tolerance
Summary
Summary

Every system at scale will have to
compromise consistency at some level.
Summary

Every system at scale will have to
compromise consistency at some level.
Summary

Every system at scale will have to
compromise consistency at some level.


Do it mindfully.
Questions?
Twitter: @sarahmei
Email: sarah@pivotallabs.com
Slides: http://bit.ly/9xS2PK


Please rate this talk! http://bi...
Upcoming SlideShare
Loading in …5
×

Beyond (No)SQL

7,268 views

Published on

Slides from my RailsConf 2010 talk, Beyond No(SQL). Conceptual tools for evaluating both SQL and NoSQL databases.

Published in: Education, Technology, Design

Beyond (No)SQL

  1. Beyond (No)SQL Sarah Mei Pivotal Labs @sarahmei sarah@pivotallabs.com
  2. ?
  3. Fsck CS. -DHH
  4. Fsck CS. -DHH (paraphrased)
  5. Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
  6. Photo by jasonwg (http://www.flickr.com/photos/jasonwg/1382036808)
  7. Agenda
  8. Agenda • The data storage landscape
  9. Agenda • The data storage landscape • Relational model & SQL
  10. Agenda • The data storage landscape • Relational model & SQL RELATIONAL ALGEBRA
  11. Agenda • The data storage landscape • Relational model & SQL RELATIONAL ALGEBRA • Evaluating data stores
  12. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
  13. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
  14. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
  15. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
  16. Photo by TANAKA Juuyoh http://www.flickr.com/photos/tanaka_juuyoh/3121538767/
  17. “NoSQL”
  18. “NoSQL” MongoDB
  19. “NoSQL” MongoDB Memcache
  20. “NoSQL” MongoDB Memcache Bigtable
  21. “NoSQL” MongoDB Memcache Bigtable CouchDB
  22. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra
  23. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort
  24. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase
  25. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase Riak
  26. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase Riak Redis
  27. “NoSQL” MongoDB Memcache Bigtable CouchDB Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
  28. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
  29. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Document Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
  30. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Document Cassandra Project Voldemort Hbase Riak Redis Tokyo Cabinet
  31. “NoSQL” MongoDB Memcache Key-value Bigtable CouchDB Document Cassandra Other Project Voldemort Hbase Riak Redis Tokyo Cabinet
  32. Photo from mga (http://www.flickr.com/photos/mgiraldo/420642350)
  33. NO.
  34. The Relational Model Diagram by Wikipedia user AutumnSnow (http://en.wikipedia.org/wiki/File:Relational_model_concepts.png)
  35. Sets
  36. Sets • No duplicates
  37. Sets • No duplicates • Unordered
  38. More sets
  39. More sets • TABLE: a set of columns and a set of rows
  40. More sets • TABLE: a set of columns and a set of rows • COLUMN: a unique name and a type
  41. More sets • TABLE: a set of columns and a set of rows • COLUMN: a unique name and a type • ROW: a set of name-value pairs
  42. Relational Algebra Operations you can do on tables (“relations”)
  43. Operations • projection: subset of available columns • selection: subset of available rows • cartesian product • set union • set intersection • rename
  44. Operations • projection: subset of available columns • selection: subset of available rows • cartesian product • set union • set intersection • rename
  45. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
  46. A Join select * from foo, bar where foo.ipsum = bar.lorem Then you select the rows that satisfy the join condition: bat ipsum cat lorem bat ipsum cat lorem me 5 us 5 me 5 us 5 me 5 them 4 => you 4 them 4 you 4 us 5 you 4 them 4
  47. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
  48. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 8,000,000 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
  49. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 8,000,000 100,000 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 you 4 us 5 you 4 them 4
  50. A Join select * from foo, bar where foo.ipsum = bar.lorem First you take the cartesian product.... foo: bat ipsum bar: cat lorem me 5 X us 5 8,000,000 100,000 you 4 them 4 bat ipsum cat lorem me 5 us 5 = me 5 them 4 800,000,000,000 you 4 us 5 you 4 them 4
  51. Your choices
  52. Your choices • Scale the database
  53. Your choices • Scale the database • Try a different approach
  54. What SQL Gets You
  55. What SQL Gets You • Speed
  56. What SQL Gets You • Speed (when data is highly structured and small enough)
  57. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation
  58. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching
  59. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching • ACID - guaranteed full consistency
  60. Image by captcreate (http://www.flickr.com/photos/27845211@N02/2662264721)
  61. What SQL Gets You
  62. What SQL Gets You • Speed
  63. What SQL Gets You • Speed (when data is highly structured and small enough)
  64. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation
  65. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching
  66. What SQL Gets You • Speed (when data is highly structured and small enough) • Aggregation • Relational searching • ACID - guaranteed full consistency
  67. What if you gave up data aggregation?
  68. What if you gave up data aggregation? Document databases: MongoDB CouchDB Riak
  69. What if you gave up where clauses?
  70. What if you gave up where clauses? Key-value stores: Memcache Project Voldemort Redis Tokyo Cabinet
  71. What if you gave up consistency?
  72. What if you gave up consistency? Then things get interesting.
  73. ACID & BASE
  74. ACID & BASE • Atomicity
  75. ACID & BASE • Atomicity • Consistency
  76. ACID & BASE • Atomicity • Consistency • Isolation
  77. ACID & BASE • Atomicity • Consistency • Isolation • Durability
  78. ACID & BASE • Atomicity • Consistency • Isolation • Durability
  79. ACID & BASE • Atomicity • Basically Available • Consistency • Isolation • Durability
  80. ACID & BASE • Atomicity • Basically Available • Consistency • Soft state • Isolation • Durability
  81. ACID & BASE • Atomicity • Basically Available • Consistency • Soft state • Isolation • Eventually consistent • Durability
  82. Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
  83. Fully ACID Fully BASE Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
  84. Fully ACID Fully BASE Redis Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
  85. Fully ACID Fully BASE CouchDB Redis Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
  86. Fully ACID Fully BASE CouchDB Redis Bigtable Photo by Marcus Vegas (http://www.flickr.com/photos/vegas/413159909)
  87. Questions to ask about data • Where can I compromise aggregation? • Where can I compromise where clauses? • Where can I compromise consistency? Where can I localize consistency?
  88. CAP Theorem Pick any two:
  89. CAP Theorem Pick any two: • Consistency
  90. CAP Theorem Pick any two: • Consistency • Availability
  91. CAP Theorem Pick any two: • Consistency • Availability • Partition tolerance
  92. Summary
  93. Summary Every system at scale will have to compromise consistency at some level.
  94. Summary Every system at scale will have to compromise consistency at some level.
  95. Summary Every system at scale will have to compromise consistency at some level. Do it mindfully.
  96. Questions? Twitter: @sarahmei Email: sarah@pivotallabs.com Slides: http://bit.ly/9xS2PK Please rate this talk! http://bit.ly/9MCtX9

×