The State of NoSQL

6,372 views
5,663 views

Published on

Given at Developer Day Durham 2010 by Ben Scofiel

Published in: Technology

The State of NoSQL

  1. The  State  of  NoSQL bensco'ield  –  viget  labs developer  day  durham 27  february  2010
  2. Motivations
  3. Performance Scalability
  4. Meh
  5. Flexibility Complexity Functionality
  6. “Comics”  Is  Hard
  7. Charlie  Chaplin Jet  Li Hank  Mann Marian  Collier
  8. Taxonomy
  9. Key-­‐Value  Stores
  10. distributed  hash  tables
  11. Performance high Scalability high Flexibility high Complexity none Functionality variable  (none)
  12. Dynamo GT.M PStore Redis
  13. Column-­‐Oriented  Stores
  14. semi-­‐structured
  15. Performance high Scalability high Flexibility moderate Complexity low Functionality minimal
  16. BigTable Cassandra HBase
  17. Document-­‐Oriented  Stores
  18. also  semi-­‐structured
  19. Performance high Scalability variable  (high) Flexibility high Complexity low Functionality variable  (low)
  20. CouchDB MongoDB RDDB Riak
  21. Graph  Databases
  22. graph  theory
  23. Performance variable Scalability variable Flexibility high Complexity high Functionality graph  theory
  24. ActiveRDF   AllegroGraph Neo4J
  25. Relational  Databases
  26. Performance variable Scalability variable Flexibility low Complexity moderate Functionality relational  algebra
  27. Examples
  28. Redis
  29. Data  Types strings lists sets sorted  sets
  30. In-­‐Memory semi-­‐persistent  /  fully  persistent master-­‐slave  replication memory-­‐bound
  31. require 'redis' gl = Redis.new # A string gl['name'] = 'Kyle Rayner' gl['name'] gl.delete('name') # A list gl.push_tail 'to-dos', 'Lose Ion power' gl.push_tail 'to-dos', 'Mourn dead loved ones' gl.push_tail 'to-dos', 'Blow up zombie lanterns' gl.list_range('to-dos', 0, -1)
  32. Tokyo  Cabinet
  33. Data  Types binary  data strings
  34. Tables!?
  35. Related  Projects tyrant dystopia promenade
  36. require 'rufus/tokyo' # Key-value jli = Rufus::Tokyo::Cabinet.new('jl.tch') jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle' ].to_yaml YAML.load(jli['members'])
  37. require 'rufus/tokyo' # Table big7 = Rufus::Tokyo::Table.new('big7.tct') big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'} big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'} big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'} big7['f'] = {'name' => 'Flash', 'role' => 'speedster'} big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'} big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'} big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'} big7.query {|q| q.add_condition 'role', :streq, 'fish-talking' }
  38. Cassandra
  39. Genealogy Dynamo BigTable
  40. Column-­‐Oriented columns supercolumns column  families
  41. Distributed automatic  replication eventual  consistency easy  expansion
  42. Availability weak  reads/writes quorum  reads/writes
  43. require 'cassandra' op = Cassandra.new('OnePiece') op.insert(:People, '1', {'name' => 'Luffy'}) op.insert(:People, '2', {'name' => 'Crocodile'}) op.insert(:People, '3', {'name' => 'Mr. 3'}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}}) luffy_fights = op.get(:Fights, '1', 'opponents') luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}
  44. CouchDB
  45. Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface
  46. Views predeWined,  updated  incrementally javascript  for  map/reduce
  47. Updates full,  including  embedded  documents
  48. require 'couchrest' konoha = CouchRest.database!('http://127.0.0.1:5984/konoha') naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind' } shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true } konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } } } puts konoha.views('first/chunin')['rows'].inspect
  49. MongoDB
  50. Storage binary  JSON  documents http://bsonspec.org
  51. Access native  clients
  52. Queries dynamic index-­‐based
  53. Updates allows  partial  updates
  54. require 'mongo' avengers = Mongo::Connection.new.db('avengers') members = avengers.collection('members') members.insert {'name' => 'Ant-Man'} members.insert {'name' => 'Hulk'} members.insert {'name' => 'Iron Man'} members.insert {'name' => 'Thor'} members.insert {'name' => 'Wasp'} members.create_index('name') pym = members.find {'name' => 'Ant-Man'} pym['name'] = 'Giant-Man' pym.save members.remove {'name' => 'Hulk'} members.insert {'name' => 'Captain America'}
  55. Riak
  56. Ask  Sean
  57. also  Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface links  for  relationships
  58. Decentralized no  privileged  nodes
  59. ConWigurable store  /  read  /  write
  60. require 'jiak' jc = JiakClient.new('127.0.0.1', 8098) jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power'] }) jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => [] }) kth = jc.fetch('supervillains', 'Normie')
  61. Neo4J
  62. Structure nodes  and  edges key-­‐value  pairs
  63. Queries lucene gremlin
  64. require 'neo4j' class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant end end
  65. Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magneto end
  66. magneto = Person.find(:name => 'Magneto') # Who likes Magneto? magneto.relationships.incoming(:crushes).nodes # Which non-mutants has Magneto dated? magneto.hookups{ !mutant? }.to_a
  67. Simulations
  68. Structure
  69. people { ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441 } companies { _id:12441 ‘name’:‘Daily Planet’ }
  70. Lack  of  Structure
  71. mysql> SELECT * FROM people LIMIT 1 G *************************** 1. row *************************** id: 1 content: --- company: Daily Planet name: Jimmy Olsen title: Superman’s Pal
  72. But  wait! friendfeed friendly
  73. mysql> desc people; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | name | varchar(50) | YES | | NULL | | +-------+-------------+------+-----+---------+-------+ mysql> desc attributes; +-----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | person_id | int(11) | YES | | NULL | | | attribute | varchar(50) | YES | | NULL | | | value | varchar(100) | YES | | NULL | | +-----------+--------------+------+-----+---------+-------+
  74. Not  Only  SQL
  75. Polyglot  Persistence
  76. Caching
  77. Already  in  Use memcached
  78. Queues
  79. Long-­‐running  processes resque
  80. Logging
  81. Rails  Log  Replacement http://github.com/peburrows/mongo_db_logger
  82. Hybrid  Domains
  83. different  domains
  84. Publishing e-­‐commerce documents
  85. Dating e-­‐commerce social  graph
  86. different  scales
  87. Photo  Sharing user  accounts uploaded  photos
  88. Next  Steps
  89. Explore
  90. Database  List http://internetmindmap.com/database_software NoSQL  Google  Group http://groups.google.com/group/nosql-­‐discussion
  91. Ignore  the  Database
  92. Logical  Modeling  First be  mindful
  93. Change  the  Default
  94. Application  Templates start  with  something  new
  95. benscoWield @bscoWield ben.scoWield@viget.com http:/ /spkr8.com/bscoWield http:/ /viget.com/extend http:   /benscoWield.com /

×