NoSQL
Death	
  to	
  Relational	
  Databases(?)
bensco'ield	
  –	
  viget	
  labs
codemash
14	
  january	
  2010




                                            1
Motivations



              2
Performance



              3
Scalability



              4
Meh



      5
Flexibility



              6
Complexity



             7
“Comics”	
  Is	
  Hard



                         8
9
Functionality



                10
Charlie	
  Chaplin




                                               Jet	
  Li



            Hank	
  Mann

                           Marian	
  Collier



                                                           11
Taxonomy



           12
Key-­‐Value	
  Stores



                        13
distributed	
  hash	
  tables



                                14
Performance     high
Scalability     high
Flexibility     high
Complexity      none
Functionality   variable	
  (none)


                                     15
Dynamo
GT.M
PStore
Redis



         16
Column-­‐Oriented	
  Stores



                              17
semi-­‐structured



                    18
Performance     high
Scalability     high
Flexibility     moderate
Complexity      low
Functionality   minimal


                           19
BigTable
Cassandra
HBase



            20
Document-­‐Oriented	
  Stores



                                21
also	
  semi-­‐structured



                            22
Performance     high
Scalability     variable	
  (high)
Flexibility     high
Complexity      low
Functionality   variable	
  (low)


                                     23
CouchDB
MongoDB
RDDB
Riak



          24
Graph	
  Databases



                     25
graph	
  theory



                  26
Performance     variable
Scalability     variable
Flexibility     high
Complexity      high
Functionality   graph	
  theory


                                  27
ActiveRDF	
  
AllegroGraph
Neo4J



                28
Relational	
  Databases



                          29
Performance     variable
Scalability     variable
Flexibility     low
Complexity      moderate
Functionality   relational	
  algebra


                                        30
Examples



           31
Redis



        32
Data	
  Types
strings
lists
sets
sorted	
  sets




                 33
In-­‐Memory
periodic	
  snapshots	
  /	
  append	
  only	
  'ile
master-­‐slave	
  replication
memory-­‐bound




                                                       34
require 'redis'

gl = Redis.new

# A string
gl['name'] = 'Kyle Rayner'
gl['name']
gl.delete('name')

# A list
gl.push_tail 'to-dos', 'Lose Ion power'
gl.push_tail 'to-dos', 'Mourn dead loved ones'
gl.push_tail 'to-dos', 'Blow up zombie lanterns'

gl.list_range('to-dos', 0, -1)




                                                   35
Tokyo	
  Cabinet



                   36
Data	
  Types
binary	
  data
strings




                 37
Tables!?




           38
Related	
  Projects
tyrant
dystopia
promenade




                      39
require 'rufus/tokyo'

# Key-value
jli = Rufus::Tokyo::Cabinet.new('jl.tch')
jli['members'] = [
  'Batman',
  'Black Canary',
  'Blue Beetle',
  'Captain Marvel',
  'Doctor Light',
  'Doctor Fate',
  'Guy Gardner',
  'Martian Manhunter',
  'Mister Miracle'
].to_yaml

YAML.load(jli['members'])




                                            40
require 'rufus/tokyo'

# Table
big7 = Rufus::Tokyo::Table.new('big7.tct')

big7['s']    =   {'name'   =>   'Superman', 'role' => 'deus ex machina'}
big7['b']    =   {'name'   =>   'Batman', 'role' => 'mastermind'}
big7['gl']   =   {'name'   =>   'Green Lantern', 'role' => 'space cop'}
big7['f']    =   {'name'   =>   'Flash', 'role' => 'speedster'}
big7['mm']   =   {'name'   =>   'Martian Manhunter', 'role' => '?'}
big7['ww']   =   {'name'   =>   'Wonder Woman', 'role' => 'hitter'}
big7['a']    =   {'name'   =>   'Aquaman', 'role' => 'fish-talking'}

big7.query {|q|
  q.add_condition 'role', :streq, 'fish-talking'
}




                                                                           41
Cassandra



            42
Genealogy
Dynamo
BigTable




            43
Column-­‐Oriented
columns
supercolumns
column	
  families




                     44
Distributed
automatic	
  replication
eventual	
  consistency
easy	
  expansion




                           45
Availability
weak	
  reads
quorum	
  reads




                  46
require 'cassandra'

op = Cassandra.new('OnePiece')

op.insert(:People, '1', {'name' => 'Luffy'})
op.insert(:People, '2', {'name' => 'Crocodile'})
op.insert(:People, '3', {'name' => 'Mr. 3'})

op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}})
op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}})

luffy_fights = op.get(:Fights, '1', 'opponents')
luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}




                                                              47
CouchDB



          48
Web-­‐Inspired
JSON	
  storage
HTTP	
  /	
  RESTful	
  interface




                                    49
Views
prede'ined,	
  updated	
  incrementally
javascript	
  for	
  map/reduce




                                          50
Updates
full,	
  including	
  embedded	
  documents




                                              51
require 'couchrest'

konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')
naruto = konoha.save_doc {
  'name' => 'Naruto Uzumaki',
  'chakra' => 'wind'
}
shikamaru = konoha.save_doc {
  'name' => 'Shikamaru Nara',
  'chunin' => true
}

konoha.save_doc {
  '_id' => '_design/first',
  :views => {
    :chunin => {
      :map => 'function(doc){if(doc.chunin){emit(null, doc);}}'
    }
  }
}

puts konoha.views('first/chunin')['rows'].inspect



                                                                  52
MongoDB



          53
Storage
binary	
  JSON	
  documents




                              54
Access
native	
  clients




                    55
Queries
dynamic
index-­‐based




                56
Updates
allows	
  partial	
  updates




                               57
require 'mongo'

avengers = Mongo::Connection.new.db('avengers')
members = avengers.collection('members')

members.insert    {'name'   =>   'Ant-Man'}
members.insert    {'name'   =>   'Hulk'}
members.insert    {'name'   =>   'Iron Man'}
members.insert    {'name'   =>   'Thor'}
members.insert    {'name'   =>   'Wasp'}

members.create_index('name')

pym = members.find {'name' => 'Ant-Man'}
pym['name'] = 'Giant-Man'
pym.save

members.remove {'name' => 'Hulk'}

members.insert {'name' => 'Captain America'}




                                                  58
Riak



       59
also	
  Web-­‐Inspired
JSON	
  storage
HTTP	
  /	
  RESTful	
  interface
links	
  for	
  relationships




                                    60
Decentralized
no	
  privileged	
  nodes




                            61
Con'igurable
store	
  /	
  read	
  /	
  write




                                   62
require 'jiak'

jc = JiakClient.new('127.0.0.1', 8098)
jc.set_bucket_schema('supervillains', {
   'allowed_fields' => ['name', 'alias', 'power']
})

jc.store({
   'bucket' => 'supervillains',
   'key' => 'Normie',
   'object' => {
     'name' => 'Norman Osborn',
     'alias' => 'Green Goblin',
     'power' => 'Supreme jerkfacedness'
   },
   'links' => []
})

kth = jc.fetch('supervillains', 'Normie')




                                                    63
Neo4J



        64
Structure
nodes	
  and	
  edges
key-­‐value	
  pairs




                        65
Queries
lucene
gremlin




          66
require 'neo4j'

class Person
  include Neo4j::NodeMixin

  property :name, :mutant
  index :name, :mutant

  has_n :crushes
  has_n :hookups
  has_n :marriages

  def initialize(name, mutant = true)
    name = name
    mutant = mutant
  end
end




                                        67
Neo4j::Transaction.run do
  magneto = Person.new('Magneto')
  esme    = Person.new('Esme')
  rogue   = Person.new('Rogue')
  magda   = Person.new('Magda', false)
  wasp    = Person.new('Wasp', false)

  magneto.crushes   << wasp
  magneto.hookups   << rogue
  magneto.marriages << magda

  esme.crushes      << magneto
  rogue.hookups     << magneto
  magda.marriages   << magneto
end




                                         68
magneto = Person.find(:name => 'Magneto')

# Who likes Magneto?
magneto.relationships.incoming(:crushes).nodes

# Which non-mutants has Magneto dated?
magneto.hookups{ !mutant? }.to_a




                                                 69
Simulations



              70
Structure



            71
people
{
  ‘name’:‘Jimmy Olsen’
  ‘title’:‘Superman’s Pal’
  ‘company_id’:12441
}

companies
{
  _id:12441
  ‘name’:‘Daily Planet’
}




                             72
Lack	
  of	
  Structure



                          73
mysql> SELECT * FROM people LIMIT 1 G
*************************** 1. row ***************************
     id: 1
content: ---
company: Daily Planet
name: Jimmy Olsen
title: Superman’s Pal




                                                                 74
But	
  wait!
friendfeed
friendly




               75
mysql> desc people;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id    | int(11)     | YES |      | NULL    |       |
| name | varchar(50) | YES |       | NULL    |       |
+-------+-------------+------+-----+---------+-------+


mysql> desc attributes;
+-----------+--------------+------+-----+---------+-------+
| Field     | Type         | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| id        | int(11)      | YES |      | NULL    |       |
| person_id | int(11)      | YES |      | NULL    |       |
| attribute | varchar(50) | YES |       | NULL    |       |
| value     | varchar(100) | YES |      | NULL    |       |
+-----------+--------------+------+-----+---------+-------+




                                                              76
Not	
  Only	
  SQL



                     77
Polyglot	
  Persistence



                          78
Caching



          79
Already	
  in	
  Use
memcached




                       80
Queues



         81
Long-­‐running	
  processes
resque




                              82
Logging



          83
Rails	
  Log	
  Replacement
http://github.com/peburrows/mongo_db_logger




                                              84
Hybrid	
  Domains



                    85
different	
  domains



                       86
Publishing
e-­‐commerce
documents




               87
Dating
e-­‐commerce
social	
  graph




                  88
different	
  scales



                      89
Photo	
  Sharing
user	
  accounts
uploaded	
  photos




                     90
Next	
  Steps



                91
Explore



          92
Database	
  List
http://internetmindmap.com/database_software


NoSQL	
  Google	
  Group
http://groups.google.com/group/nosql-­‐discussion




                                                    93
Ignore	
  the	
  Database



                            94
Logical	
  Modeling	
  First
be	
  mindful




                               95
Change	
  the	
  Default



                           96
Application	
  Templates
start	
  with	
  something	
  new




                                    97
bensco'ield
@bsco'ield
ben.sco'ield@viget.com
http:/    /spkr8.com/bsco'ield
http:/    /viget.com/extend
http:	
   /bensco'ield.com
     /




                                 98

NoSQL @ CodeMash 2010