0
NoSQL
Death	
  to	
  Relational	
  Databases(?)
bensco'ield	
  –	
  viget	
  labs
codemash
14	
  january	
  2010




     ...
Motivations



              2
Performance



              3
Scalability



              4
Meh



      5
Flexibility



              6
Complexity



             7
“Comics”	
  Is	
  Hard



                         8
9
Functionality



                10
Charlie	
  Chaplin




                                               Jet	
  Li



            Hank	
  Mann

             ...
Taxonomy



           12
Key-­‐Value	
  Stores



                        13
distributed	
  hash	
  tables



                                14
Performance     high
Scalability     high
Flexibility     high
Complexity      none
Functionality   variable	
  (none)


 ...
Dynamo
GT.M
PStore
Redis



         16
Column-­‐Oriented	
  Stores



                              17
semi-­‐structured



                    18
Performance     high
Scalability     high
Flexibility     moderate
Complexity      low
Functionality   minimal


         ...
BigTable
Cassandra
HBase



            20
Document-­‐Oriented	
  Stores



                                21
also	
  semi-­‐structured



                            22
Performance     high
Scalability     variable	
  (high)
Flexibility     high
Complexity      low
Functionality   variable	...
CouchDB
MongoDB
RDDB
Riak



          24
Graph	
  Databases



                     25
graph	
  theory



                  26
Performance     variable
Scalability     variable
Flexibility     high
Complexity      high
Functionality   graph	
  theor...
ActiveRDF	
  
AllegroGraph
Neo4J



                28
Relational	
  Databases



                          29
Performance     variable
Scalability     variable
Flexibility     low
Complexity      moderate
Functionality   relational	...
Examples



           31
Redis



        32
Data	
  Types
strings
lists
sets
sorted	
  sets




                 33
In-­‐Memory
periodic	
  snapshots	
  /	
  append	
  only	
  'ile
master-­‐slave	
  replication
memory-­‐bound




        ...
require 'redis'

gl = Redis.new

# A string
gl['name'] = 'Kyle Rayner'
gl['name']
gl.delete('name')

# A list
gl.push_tail...
Tokyo	
  Cabinet



                   36
Data	
  Types
binary	
  data
strings




                 37
Tables!?




           38
Related	
  Projects
tyrant
dystopia
promenade




                      39
require 'rufus/tokyo'

# Key-value
jli = Rufus::Tokyo::Cabinet.new('jl.tch')
jli['members'] = [
  'Batman',
  'Black Canar...
require 'rufus/tokyo'

# Table
big7 = Rufus::Tokyo::Table.new('big7.tct')

big7['s']    =   {'name'   =>   'Superman', 'ro...
Cassandra



            42
Genealogy
Dynamo
BigTable




            43
Column-­‐Oriented
columns
supercolumns
column	
  families




                     44
Distributed
automatic	
  replication
eventual	
  consistency
easy	
  expansion




                           45
Availability
weak	
  reads
quorum	
  reads




                  46
require 'cassandra'

op = Cassandra.new('OnePiece')

op.insert(:People, '1', {'name' => 'Luffy'})
op.insert(:People, '2', ...
CouchDB



          48
Web-­‐Inspired
JSON	
  storage
HTTP	
  /	
  RESTful	
  interface




                                    49
Views
prede'ined,	
  updated	
  incrementally
javascript	
  for	
  map/reduce




                                        ...
Updates
full,	
  including	
  embedded	
  documents




                                              51
require 'couchrest'

konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')
naruto = konoha.save_doc {
  'name' => '...
MongoDB



          53
Storage
binary	
  JSON	
  documents




                              54
Access
native	
  clients




                    55
Queries
dynamic
index-­‐based




                56
Updates
allows	
  partial	
  updates




                               57
require 'mongo'

avengers = Mongo::Connection.new.db('avengers')
members = avengers.collection('members')

members.insert ...
Riak



       59
also	
  Web-­‐Inspired
JSON	
  storage
HTTP	
  /	
  RESTful	
  interface
links	
  for	
  relationships




               ...
Decentralized
no	
  privileged	
  nodes




                            61
Con'igurable
store	
  /	
  read	
  /	
  write




                                   62
require 'jiak'

jc = JiakClient.new('127.0.0.1', 8098)
jc.set_bucket_schema('supervillains', {
   'allowed_fields' => ['na...
Neo4J



        64
Structure
nodes	
  and	
  edges
key-­‐value	
  pairs




                        65
Queries
lucene
gremlin




          66
require 'neo4j'

class Person
  include Neo4j::NodeMixin

  property :name, :mutant
  index :name, :mutant

  has_n :crush...
Neo4j::Transaction.run do
  magneto = Person.new('Magneto')
  esme    = Person.new('Esme')
  rogue   = Person.new('Rogue')...
magneto = Person.find(:name => 'Magneto')

# Who likes Magneto?
magneto.relationships.incoming(:crushes).nodes

# Which no...
Simulations



              70
Structure



            71
people
{
  ‘name’:‘Jimmy Olsen’
  ‘title’:‘Superman’s Pal’
  ‘company_id’:12441
}

companies
{
  _id:12441
  ‘name’:‘Daily...
Lack	
  of	
  Structure



                          73
mysql> SELECT * FROM people LIMIT 1 G
*************************** 1. row ***************************
     id: 1
content: -...
But	
  wait!
friendfeed
friendly




               75
mysql> desc people;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | ...
Not	
  Only	
  SQL



                     77
Polyglot	
  Persistence



                          78
Caching



          79
Already	
  in	
  Use
memcached




                       80
Queues



         81
Long-­‐running	
  processes
resque




                              82
Logging



          83
Rails	
  Log	
  Replacement
http://github.com/peburrows/mongo_db_logger




                                              ...
Hybrid	
  Domains



                    85
different	
  domains



                       86
Publishing
e-­‐commerce
documents




               87
Dating
e-­‐commerce
social	
  graph




                  88
different	
  scales



                      89
Photo	
  Sharing
user	
  accounts
uploaded	
  photos




                     90
Next	
  Steps



                91
Explore



          92
Database	
  List
http://internetmindmap.com/database_software


NoSQL	
  Google	
  Group
http://groups.google.com/group/no...
Ignore	
  the	
  Database



                            94
Logical	
  Modeling	
  First
be	
  mindful




                               95
Change	
  the	
  Default



                           96
Application	
  Templates
start	
  with	
  something	
  new




                                    97
bensco'ield
@bsco'ield
ben.sco'ield@viget.com
http:/    /spkr8.com/bsco'ield
http:/    /viget.com/extend
http:	
   /bensco...
Upcoming SlideShare
Loading in...5
×

NoSQL @ CodeMash 2010

6,414

Published on

Overview of the NoSQL movement, given at CodeMash 2010

Published in: Technology
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,414
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
183
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide

Transcript of "NoSQL @ CodeMash 2010"

  1. 1. NoSQL Death  to  Relational  Databases(?) bensco'ield  –  viget  labs codemash 14  january  2010 1
  2. 2. Motivations 2
  3. 3. Performance 3
  4. 4. Scalability 4
  5. 5. Meh 5
  6. 6. Flexibility 6
  7. 7. Complexity 7
  8. 8. “Comics”  Is  Hard 8
  9. 9. 9
  10. 10. Functionality 10
  11. 11. Charlie  Chaplin Jet  Li Hank  Mann Marian  Collier 11
  12. 12. Taxonomy 12
  13. 13. Key-­‐Value  Stores 13
  14. 14. distributed  hash  tables 14
  15. 15. Performance high Scalability high Flexibility high Complexity none Functionality variable  (none) 15
  16. 16. Dynamo GT.M PStore Redis 16
  17. 17. Column-­‐Oriented  Stores 17
  18. 18. semi-­‐structured 18
  19. 19. Performance high Scalability high Flexibility moderate Complexity low Functionality minimal 19
  20. 20. BigTable Cassandra HBase 20
  21. 21. Document-­‐Oriented  Stores 21
  22. 22. also  semi-­‐structured 22
  23. 23. Performance high Scalability variable  (high) Flexibility high Complexity low Functionality variable  (low) 23
  24. 24. CouchDB MongoDB RDDB Riak 24
  25. 25. Graph  Databases 25
  26. 26. graph  theory 26
  27. 27. Performance variable Scalability variable Flexibility high Complexity high Functionality graph  theory 27
  28. 28. ActiveRDF   AllegroGraph Neo4J 28
  29. 29. Relational  Databases 29
  30. 30. Performance variable Scalability variable Flexibility low Complexity moderate Functionality relational  algebra 30
  31. 31. Examples 31
  32. 32. Redis 32
  33. 33. Data  Types strings lists sets sorted  sets 33
  34. 34. In-­‐Memory periodic  snapshots  /  append  only  'ile master-­‐slave  replication memory-­‐bound 34
  35. 35. require 'redis' gl = Redis.new # A string gl['name'] = 'Kyle Rayner' gl['name'] gl.delete('name') # A list gl.push_tail 'to-dos', 'Lose Ion power' gl.push_tail 'to-dos', 'Mourn dead loved ones' gl.push_tail 'to-dos', 'Blow up zombie lanterns' gl.list_range('to-dos', 0, -1) 35
  36. 36. Tokyo  Cabinet 36
  37. 37. Data  Types binary  data strings 37
  38. 38. Tables!? 38
  39. 39. Related  Projects tyrant dystopia promenade 39
  40. 40. require 'rufus/tokyo' # Key-value jli = Rufus::Tokyo::Cabinet.new('jl.tch') jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle' ].to_yaml YAML.load(jli['members']) 40
  41. 41. require 'rufus/tokyo' # Table big7 = Rufus::Tokyo::Table.new('big7.tct') big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'} big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'} big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'} big7['f'] = {'name' => 'Flash', 'role' => 'speedster'} big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'} big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'} big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'} big7.query {|q| q.add_condition 'role', :streq, 'fish-talking' } 41
  42. 42. Cassandra 42
  43. 43. Genealogy Dynamo BigTable 43
  44. 44. Column-­‐Oriented columns supercolumns column  families 44
  45. 45. Distributed automatic  replication eventual  consistency easy  expansion 45
  46. 46. Availability weak  reads quorum  reads 46
  47. 47. require 'cassandra' op = Cassandra.new('OnePiece') op.insert(:People, '1', {'name' => 'Luffy'}) op.insert(:People, '2', {'name' => 'Crocodile'}) op.insert(:People, '3', {'name' => 'Mr. 3'}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}}) luffy_fights = op.get(:Fights, '1', 'opponents') luffy_fights.map {|t, opp| op.get(:People, opp, 'name')} 47
  48. 48. CouchDB 48
  49. 49. Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface 49
  50. 50. Views prede'ined,  updated  incrementally javascript  for  map/reduce 50
  51. 51. Updates full,  including  embedded  documents 51
  52. 52. require 'couchrest' konoha = CouchRest.database!('http://127.0.0.1:5984/konoha') naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind' } shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true } konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } } } puts konoha.views('first/chunin')['rows'].inspect 52
  53. 53. MongoDB 53
  54. 54. Storage binary  JSON  documents 54
  55. 55. Access native  clients 55
  56. 56. Queries dynamic index-­‐based 56
  57. 57. Updates allows  partial  updates 57
  58. 58. require 'mongo' avengers = Mongo::Connection.new.db('avengers') members = avengers.collection('members') members.insert {'name' => 'Ant-Man'} members.insert {'name' => 'Hulk'} members.insert {'name' => 'Iron Man'} members.insert {'name' => 'Thor'} members.insert {'name' => 'Wasp'} members.create_index('name') pym = members.find {'name' => 'Ant-Man'} pym['name'] = 'Giant-Man' pym.save members.remove {'name' => 'Hulk'} members.insert {'name' => 'Captain America'} 58
  59. 59. Riak 59
  60. 60. also  Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface links  for  relationships 60
  61. 61. Decentralized no  privileged  nodes 61
  62. 62. Con'igurable store  /  read  /  write 62
  63. 63. require 'jiak' jc = JiakClient.new('127.0.0.1', 8098) jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power'] }) jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => [] }) kth = jc.fetch('supervillains', 'Normie') 63
  64. 64. Neo4J 64
  65. 65. Structure nodes  and  edges key-­‐value  pairs 65
  66. 66. Queries lucene gremlin 66
  67. 67. require 'neo4j' class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant end end 67
  68. 68. Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magneto end 68
  69. 69. magneto = Person.find(:name => 'Magneto') # Who likes Magneto? magneto.relationships.incoming(:crushes).nodes # Which non-mutants has Magneto dated? magneto.hookups{ !mutant? }.to_a 69
  70. 70. Simulations 70
  71. 71. Structure 71
  72. 72. people { ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441 } companies { _id:12441 ‘name’:‘Daily Planet’ } 72
  73. 73. Lack  of  Structure 73
  74. 74. mysql> SELECT * FROM people LIMIT 1 G *************************** 1. row *************************** id: 1 content: --- company: Daily Planet name: Jimmy Olsen title: Superman’s Pal 74
  75. 75. But  wait! friendfeed friendly 75
  76. 76. mysql> desc people; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | name | varchar(50) | YES | | NULL | | +-------+-------------+------+-----+---------+-------+ mysql> desc attributes; +-----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | person_id | int(11) | YES | | NULL | | | attribute | varchar(50) | YES | | NULL | | | value | varchar(100) | YES | | NULL | | +-----------+--------------+------+-----+---------+-------+ 76
  77. 77. Not  Only  SQL 77
  78. 78. Polyglot  Persistence 78
  79. 79. Caching 79
  80. 80. Already  in  Use memcached 80
  81. 81. Queues 81
  82. 82. Long-­‐running  processes resque 82
  83. 83. Logging 83
  84. 84. Rails  Log  Replacement http://github.com/peburrows/mongo_db_logger 84
  85. 85. Hybrid  Domains 85
  86. 86. different  domains 86
  87. 87. Publishing e-­‐commerce documents 87
  88. 88. Dating e-­‐commerce social  graph 88
  89. 89. different  scales 89
  90. 90. Photo  Sharing user  accounts uploaded  photos 90
  91. 91. Next  Steps 91
  92. 92. Explore 92
  93. 93. Database  List http://internetmindmap.com/database_software NoSQL  Google  Group http://groups.google.com/group/nosql-­‐discussion 93
  94. 94. Ignore  the  Database 94
  95. 95. Logical  Modeling  First be  mindful 95
  96. 96. Change  the  Default 96
  97. 97. Application  Templates start  with  something  new 97
  98. 98. bensco'ield @bsco'ield ben.sco'ield@viget.com http:/ /spkr8.com/bsco'ield http:/ /viget.com/extend http:   /bensco'ield.com / 98
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×