The State of NoSQL

  • 4,146 views
Uploaded on

Given at Developer Day Durham 2010 by Ben Scofiel

Given at Developer Day Durham 2010 by Ben Scofiel

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,146
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
126
Comments
0
Likes
10

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The  State  of  NoSQL bensco'ield  –  viget  labs developer  day  durham 27  february  2010
  • 2. Motivations
  • 3. Performance Scalability
  • 4. Meh
  • 5. Flexibility Complexity Functionality
  • 6. “Comics”  Is  Hard
  • 7. Charlie  Chaplin Jet  Li Hank  Mann Marian  Collier
  • 8. Taxonomy
  • 9. Key-­‐Value  Stores
  • 10. distributed  hash  tables
  • 11. Performance high Scalability high Flexibility high Complexity none Functionality variable  (none)
  • 12. Dynamo GT.M PStore Redis
  • 13. Column-­‐Oriented  Stores
  • 14. semi-­‐structured
  • 15. Performance high Scalability high Flexibility moderate Complexity low Functionality minimal
  • 16. BigTable Cassandra HBase
  • 17. Document-­‐Oriented  Stores
  • 18. also  semi-­‐structured
  • 19. Performance high Scalability variable  (high) Flexibility high Complexity low Functionality variable  (low)
  • 20. CouchDB MongoDB RDDB Riak
  • 21. Graph  Databases
  • 22. graph  theory
  • 23. Performance variable Scalability variable Flexibility high Complexity high Functionality graph  theory
  • 24. ActiveRDF   AllegroGraph Neo4J
  • 25. Relational  Databases
  • 26. Performance variable Scalability variable Flexibility low Complexity moderate Functionality relational  algebra
  • 27. Examples
  • 28. Redis
  • 29. Data  Types strings lists sets sorted  sets
  • 30. In-­‐Memory semi-­‐persistent  /  fully  persistent master-­‐slave  replication memory-­‐bound
  • 31. require 'redis' gl = Redis.new # A string gl['name'] = 'Kyle Rayner' gl['name'] gl.delete('name') # A list gl.push_tail 'to-dos', 'Lose Ion power' gl.push_tail 'to-dos', 'Mourn dead loved ones' gl.push_tail 'to-dos', 'Blow up zombie lanterns' gl.list_range('to-dos', 0, -1)
  • 32. Tokyo  Cabinet
  • 33. Data  Types binary  data strings
  • 34. Tables!?
  • 35. Related  Projects tyrant dystopia promenade
  • 36. require 'rufus/tokyo' # Key-value jli = Rufus::Tokyo::Cabinet.new('jl.tch') jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle' ].to_yaml YAML.load(jli['members'])
  • 37. require 'rufus/tokyo' # Table big7 = Rufus::Tokyo::Table.new('big7.tct') big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'} big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'} big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'} big7['f'] = {'name' => 'Flash', 'role' => 'speedster'} big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'} big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'} big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'} big7.query {|q| q.add_condition 'role', :streq, 'fish-talking' }
  • 38. Cassandra
  • 39. Genealogy Dynamo BigTable
  • 40. Column-­‐Oriented columns supercolumns column  families
  • 41. Distributed automatic  replication eventual  consistency easy  expansion
  • 42. Availability weak  reads/writes quorum  reads/writes
  • 43. require 'cassandra' op = Cassandra.new('OnePiece') op.insert(:People, '1', {'name' => 'Luffy'}) op.insert(:People, '2', {'name' => 'Crocodile'}) op.insert(:People, '3', {'name' => 'Mr. 3'}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}}) op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}}) luffy_fights = op.get(:Fights, '1', 'opponents') luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}
  • 44. CouchDB
  • 45. Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface
  • 46. Views predeWined,  updated  incrementally javascript  for  map/reduce
  • 47. Updates full,  including  embedded  documents
  • 48. require 'couchrest' konoha = CouchRest.database!('http://127.0.0.1:5984/konoha') naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind' } shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true } konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } } } puts konoha.views('first/chunin')['rows'].inspect
  • 49. MongoDB
  • 50. Storage binary  JSON  documents http://bsonspec.org
  • 51. Access native  clients
  • 52. Queries dynamic index-­‐based
  • 53. Updates allows  partial  updates
  • 54. require 'mongo' avengers = Mongo::Connection.new.db('avengers') members = avengers.collection('members') members.insert {'name' => 'Ant-Man'} members.insert {'name' => 'Hulk'} members.insert {'name' => 'Iron Man'} members.insert {'name' => 'Thor'} members.insert {'name' => 'Wasp'} members.create_index('name') pym = members.find {'name' => 'Ant-Man'} pym['name'] = 'Giant-Man' pym.save members.remove {'name' => 'Hulk'} members.insert {'name' => 'Captain America'}
  • 55. Riak
  • 56. Ask  Sean
  • 57. also  Web-­‐Inspired JSON  storage HTTP  /  RESTful  interface links  for  relationships
  • 58. Decentralized no  privileged  nodes
  • 59. ConWigurable store  /  read  /  write
  • 60. require 'jiak' jc = JiakClient.new('127.0.0.1', 8098) jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power'] }) jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => [] }) kth = jc.fetch('supervillains', 'Normie')
  • 61. Neo4J
  • 62. Structure nodes  and  edges key-­‐value  pairs
  • 63. Queries lucene gremlin
  • 64. require 'neo4j' class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant end end
  • 65. Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magneto end
  • 66. magneto = Person.find(:name => 'Magneto') # Who likes Magneto? magneto.relationships.incoming(:crushes).nodes # Which non-mutants has Magneto dated? magneto.hookups{ !mutant? }.to_a
  • 67. Simulations
  • 68. Structure
  • 69. people { ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441 } companies { _id:12441 ‘name’:‘Daily Planet’ }
  • 70. Lack  of  Structure
  • 71. mysql> SELECT * FROM people LIMIT 1 G *************************** 1. row *************************** id: 1 content: --- company: Daily Planet name: Jimmy Olsen title: Superman’s Pal
  • 72. But  wait! friendfeed friendly
  • 73. mysql> desc people; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | name | varchar(50) | YES | | NULL | | +-------+-------------+------+-----+---------+-------+ mysql> desc attributes; +-----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------+--------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | person_id | int(11) | YES | | NULL | | | attribute | varchar(50) | YES | | NULL | | | value | varchar(100) | YES | | NULL | | +-----------+--------------+------+-----+---------+-------+
  • 74. Not  Only  SQL
  • 75. Polyglot  Persistence
  • 76. Caching
  • 77. Already  in  Use memcached
  • 78. Queues
  • 79. Long-­‐running  processes resque
  • 80. Logging
  • 81. Rails  Log  Replacement http://github.com/peburrows/mongo_db_logger
  • 82. Hybrid  Domains
  • 83. different  domains
  • 84. Publishing e-­‐commerce documents
  • 85. Dating e-­‐commerce social  graph
  • 86. different  scales
  • 87. Photo  Sharing user  accounts uploaded  photos
  • 88. Next  Steps
  • 89. Explore
  • 90. Database  List http://internetmindmap.com/database_software NoSQL  Google  Group http://groups.google.com/group/nosql-­‐discussion
  • 91. Ignore  the  Database
  • 92. Logical  Modeling  First be  mindful
  • 93. Change  the  Default
  • 94. Application  Templates start  with  something  new
  • 95. benscoWield @bscoWield ben.scoWield@viget.com http:/ /spkr8.com/bscoWield http:/ /viget.com/extend http:   /benscoWield.com /