The	
  State	
  of	
  NoSQL
bensco'ield	
  –	
  viget	
  labs
developer	
  day	
  durham
27	
  february	
  2010
Motivations
Performance
Scalability
Meh
Flexibility
Complexity
Functionality
“Comics”	
  Is	
  Hard
Charlie	
  Chaplin




                                               Jet	
  Li



            Hank	
  Mann

             ...
Taxonomy
Key-­‐Value	
  Stores
distributed	
  hash	
  tables
Performance     high
Scalability     high
Flexibility     high
Complexity      none
Functionality   variable	
  (none)
Dynamo
GT.M
PStore
Redis
Column-­‐Oriented	
  Stores
semi-­‐structured
Performance     high
Scalability     high
Flexibility     moderate
Complexity      low
Functionality   minimal
BigTable
Cassandra
HBase
Document-­‐Oriented	
  Stores
also	
  semi-­‐structured
Performance     high
Scalability     variable	
  (high)
Flexibility     high
Complexity      low
Functionality   variable	...
CouchDB
MongoDB
RDDB
Riak
Graph	
  Databases
graph	
  theory
Performance     variable
Scalability     variable
Flexibility     high
Complexity      high
Functionality   graph	
  theory
ActiveRDF	
  
AllegroGraph
Neo4J
Relational	
  Databases
Performance     variable
Scalability     variable
Flexibility     low
Complexity      moderate
Functionality   relational	...
Examples
Redis
Data	
  Types
strings
lists
sets
sorted	
  sets
In-­‐Memory
semi-­‐persistent	
  /	
  fully	
  persistent
master-­‐slave	
  replication
memory-­‐bound
require 'redis'

gl = Redis.new

# A string
gl['name'] = 'Kyle Rayner'
gl['name']
gl.delete('name')

# A list
gl.push_tail...
Tokyo	
  Cabinet
Data	
  Types
binary	
  data
strings
Tables!?
Related	
  Projects
tyrant
dystopia
promenade
require 'rufus/tokyo'

# Key-value
jli = Rufus::Tokyo::Cabinet.new('jl.tch')
jli['members'] = [
  'Batman',
  'Black Canar...
require 'rufus/tokyo'

# Table
big7 = Rufus::Tokyo::Table.new('big7.tct')

big7['s']    =   {'name'   =>   'Superman', 'ro...
Cassandra
Genealogy
Dynamo
BigTable
Column-­‐Oriented
columns
supercolumns
column	
  families
Distributed
automatic	
  replication
eventual	
  consistency
easy	
  expansion
Availability
weak	
  reads/writes
quorum	
  reads/writes
require 'cassandra'

op = Cassandra.new('OnePiece')

op.insert(:People, '1', {'name' => 'Luffy'})
op.insert(:People, '2', ...
CouchDB
Web-­‐Inspired
JSON	
  storage
HTTP	
  /	
  RESTful	
  interface
Views
predeWined,	
  updated	
  incrementally
javascript	
  for	
  map/reduce
Updates
full,	
  including	
  embedded	
  documents
require 'couchrest'

konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')
naruto = konoha.save_doc {
  'name' => '...
MongoDB
Storage
binary	
  JSON	
  documents
http://bsonspec.org
Access
native	
  clients
Queries
dynamic
index-­‐based
Updates
allows	
  partial	
  updates
require 'mongo'

avengers = Mongo::Connection.new.db('avengers')
members = avengers.collection('members')

members.insert ...
Riak
Ask	
  Sean
also	
  Web-­‐Inspired
JSON	
  storage
HTTP	
  /	
  RESTful	
  interface
links	
  for	
  relationships
Decentralized
no	
  privileged	
  nodes
ConWigurable
store	
  /	
  read	
  /	
  write
require 'jiak'

jc = JiakClient.new('127.0.0.1', 8098)
jc.set_bucket_schema('supervillains', {
   'allowed_fields' => ['na...
Neo4J
Structure
nodes	
  and	
  edges
key-­‐value	
  pairs
Queries
lucene
gremlin
require 'neo4j'

class Person
  include Neo4j::NodeMixin

  property :name, :mutant
  index :name, :mutant

  has_n :crush...
Neo4j::Transaction.run do
  magneto = Person.new('Magneto')
  esme    = Person.new('Esme')
  rogue   = Person.new('Rogue')...
magneto = Person.find(:name => 'Magneto')

# Who likes Magneto?
magneto.relationships.incoming(:crushes).nodes

# Which no...
Simulations
Structure
people
{
  ‘name’:‘Jimmy Olsen’
  ‘title’:‘Superman’s Pal’
  ‘company_id’:12441
}

companies
{
  _id:12441
  ‘name’:‘Daily...
Lack	
  of	
  Structure
mysql> SELECT * FROM people LIMIT 1 G
*************************** 1. row ***************************
     id: 1
content: -...
But	
  wait!
friendfeed
friendly
mysql> desc people;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | ...
Not	
  Only	
  SQL
Polyglot	
  Persistence
Caching
Already	
  in	
  Use
memcached
Queues
Long-­‐running	
  processes
resque
Logging
Rails	
  Log	
  Replacement
http://github.com/peburrows/mongo_db_logger
Hybrid	
  Domains
different	
  domains
Publishing
e-­‐commerce
documents
Dating
e-­‐commerce
social	
  graph
different	
  scales
Photo	
  Sharing
user	
  accounts
uploaded	
  photos
Next	
  Steps
Explore
Database	
  List
http://internetmindmap.com/database_software


NoSQL	
  Google	
  Group
http://groups.google.com/group/no...
Ignore	
  the	
  Database
Logical	
  Modeling	
  First
be	
  mindful
Change	
  the	
  Default
Application	
  Templates
start	
  with	
  something	
  new
benscoWield
@bscoWield
ben.scoWield@viget.com
http:/    /spkr8.com/bscoWield
http:/    /viget.com/extend
http:	
   /bensco...
The State of NoSQL
The State of NoSQL
The State of NoSQL
The State of NoSQL
Upcoming SlideShare
Loading in...5
×

The State of NoSQL

4,283

Published on

Given at Developer Day Durham 2010 by Ben Scofiel

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,283
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
127
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

The State of NoSQL

  1. 1. The  State  of  NoSQL bensco'ield  –  viget  labs developer  day  durham 27  february  2010
  2. 2. Motivations
  3. 3. Performance Scalability
  4. 4. Meh
  5. 5. Flexibility Complexity Functionality
  6. 6. “Comics”  Is  Hard
  7. 7. Charlie  Chaplin Jet  Li Hank  Mann Marian  Collier
  8. 8. Taxonomy
  9. 9. Key-­‐Value  Stores
  10. 10. distributed  hash  tables
  11. 11. Performance high Scalability high Flexibility high Complexity none Functionality variable  (none)
  12. 12. Dynamo GT.M PStore Redis
  13. 13. Column-­‐Oriented  Stores
  14. 14. semi-­‐structured
  15. 15. Performance high Scalability high Flexibility moderate Complexity low Functionality minimal
  16. 16. BigTable Cassandra HBase
  17. 17. Document-­‐Oriented  Stores
  18. 18. also  semi-­‐structured
  19. 19. Performance high Scalability variable  (high) Flexibility high Complexity low Functionality variable  (low)
  20. 20. CouchDB MongoDB RDDB Riak
  21. 21. Graph  Databases
  22. 22. graph  theory
  23. 23. Performance variable Scalability variable Flexibility high Complexity high Functionality graph  theory
  24. 24. ActiveRDF   AllegroGraph Neo4J
  25. 25. Relational  Databases
  26. 26. Performance variable Scalability variable Flexibility low Complexity moderate Functionality relational  algebra
  27. 27. Examples
  28. 28. Redis
  29. 29. Data  Types strings lists sets sorted  sets
  30. 30. In-­‐Memory semi-­‐persistent  /  fully  persistent master-­‐slave  replication memory-­‐bound
  31. 31. require 'redis' gl = Redis.new # A string gl['name'] = 'Kyle Rayner' gl['name'] gl.delete('name') # A list gl.push_tail 'to-dos', 'Lose Ion power' gl.push_tail 'to-dos', 'Mourn dead loved ones' gl.push_tail 'to-dos', 'Blow up zombie lanterns' gl.list_range('to-dos', 0, -1)
  32. 32. Tokyo  Cabinet
  33. 33. Data  Types binary  data strings
  34. 34. Tables!?
  35. 35. Related  Projects tyrant dystopia promenade
  36. 36. require 'rufus/tokyo' # Key-value jli = Rufus::Tokyo::Cabinet.new('jl.tch') jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle' ].to_yaml YAML.load(jli['members'])
  37. 37. require 'rufus/tokyo' # Table big7 = Rufus::Tokyo::Table.new('big7.tct') big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'} big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'} big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'} big7['f'] = {'name' => 'Flash', 'role' => 'speedster'} big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'} big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'} big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'} big7.query {|q| q.add_condition 'role', :streq, 'fish-talking' }
  38. 38. Cassandra
  39. 39. Genealogy Dynamo BigTable
  40. 40. Column-­‐Oriented columns supercolumns column  families
  41. 41. Distributed automatic  replication eventual  consistency easy  expansion
  42. 42. Availability weak  reads/writes quorum  reads/writes
  43. 43. require 'cassandra' op = Cassandra.new('OnePiece') op.insert(:People, '1', {'name' => 'Luffy'}) op.insert(:People, '2', {'name' => 'Crocodile'}) op.insert(:People, '3', {'name' => 'Mr. 3'}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}}) luffy_fights = op.get(:Fights, '1', 'opponents') luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}
  44. 44. CouchDB
  45. 45. Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface
  46. 46. Views predeWined,  updated  incrementally javascript  for  map/reduce
  47. 47. Updates full,  including  embedded  documents
  48. 48. require 'couchrest' konoha = CouchRest.database!('http://127.0.0.1:5984/konoha') naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind' } shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true } konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } } } puts konoha.views('first/chunin')['rows'].inspect
  49. 49. MongoDB
  50. 50. Storage binary  JSON  documents http://bsonspec.org
  51. 51. Access native  clients
  52. 52. Queries dynamic index-­‐based
  53. 53. Updates allows  partial  updates
  54. 54. require 'mongo' avengers = Mongo::Connection.new.db('avengers') members = avengers.collection('members') members.insert {'name' => 'Ant-Man'} members.insert {'name' => 'Hulk'} members.insert {'name' => 'Iron Man'} members.insert {'name' => 'Thor'} members.insert {'name' => 'Wasp'} members.create_index('name') pym = members.find {'name' => 'Ant-Man'} pym['name'] = 'Giant-Man' pym.save members.remove {'name' => 'Hulk'} members.insert {'name' => 'Captain America'}
  55. 55. Riak
  56. 56. Ask  Sean
  57. 57. also  Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface links  for  relationships
  58. 58. Decentralized no  privileged  nodes
  59. 59. ConWigurable store  /  read  /  write
  60. 60. require 'jiak' jc = JiakClient.new('127.0.0.1', 8098) jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power'] }) jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => [] }) kth = jc.fetch('supervillains', 'Normie')
  61. 61. Neo4J
  62. 62. Structure nodes  and  edges key-­‐value  pairs
  63. 63. Queries lucene gremlin
  64. 64. require 'neo4j' class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant end end
  65. 65. Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magneto end
  66. 66. magneto = Person.find(:name => 'Magneto') # Who likes Magneto? magneto.relationships.incoming(:crushes).nodes # Which non-mutants has Magneto dated? magneto.hookups{ !mutant? }.to_a
  67. 67. Simulations
  68. 68. Structure
  69. 69. people { ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441 } companies { _id:12441 ‘name’:‘Daily Planet’ }
  70. 70. Lack  of  Structure
  71. 71. mysql> SELECT * FROM people LIMIT 1 G *************************** 1. row *************************** id: 1 content: --- company: Daily Planet name: Jimmy Olsen title: Superman’s Pal
  72. 72. But  wait! friendfeed friendly
  73. 73. mysql> desc people; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | name | varchar(50) | YES | | NULL | | +-------+-------------+------+-----+---------+-------+ mysql> desc attributes; +-----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | person_id | int(11) | YES | | NULL | | | attribute | varchar(50) | YES | | NULL | | | value | varchar(100) | YES | | NULL | | +-----------+--------------+------+-----+---------+-------+
  74. 74. Not  Only  SQL
  75. 75. Polyglot  Persistence
  76. 76. Caching
  77. 77. Already  in  Use memcached
  78. 78. Queues
  79. 79. Long-­‐running  processes resque
  80. 80. Logging
  81. 81. Rails  Log  Replacement http://github.com/peburrows/mongo_db_logger
  82. 82. Hybrid  Domains
  83. 83. different  domains
  84. 84. Publishing e-­‐commerce documents
  85. 85. Dating e-­‐commerce social  graph
  86. 86. different  scales
  87. 87. Photo  Sharing user  accounts uploaded  photos
  88. 88. Next  Steps
  89. 89. Explore
  90. 90. Database  List http://internetmindmap.com/database_software NoSQL  Google  Group http://groups.google.com/group/nosql-­‐discussion
  91. 91. Ignore  the  Database
  92. 92. Logical  Modeling  First be  mindful
  93. 93. Change  the  Default
  94. 94. Application  Templates start  with  something  new
  95. 95. benscoWield @bscoWield ben.scoWield@viget.com http:/ /spkr8.com/bscoWield http:/ /viget.com/extend http:   /benscoWield.com /
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×