Given at Developer Day Durham 2010 by Ben Scofiel

1. The
State
of
NoSQL
bensco'ield
–
viget
labs
developer
day
durham
27
february
2010

2. Motivations

3. Performance
Scalability

4. Meh

5. Flexibility
Complexity
Functionality

6. “Comics”
Is
Hard

8. Charlie
Chaplin
Jet
Li
Hank
Mann
Marian
Collier

9. Taxonomy

10. Key-­‐Value
Stores

11. distributed
hash
tables

12. Performance high
Scalability high
Flexibility high
Complexity none
Functionality variable
(none)

13. Dynamo
GT.M
PStore
Redis

14. Column-­‐Oriented
Stores

15. semi-­‐structured

16. Performance high
Scalability high
Flexibility moderate
Complexity low
Functionality minimal

17. BigTable
Cassandra
HBase

18. Document-­‐Oriented
Stores

19. also
semi-­‐structured

20. Performance high
Scalability variable
(high)
Flexibility high
Complexity low
Functionality variable
(low)

21. CouchDB
MongoDB
RDDB
Riak

22. Graph
Databases

23. graph
theory

24. Performance variable
Scalability variable
Flexibility high
Complexity high
Functionality graph
theory

25. ActiveRDF
AllegroGraph
Neo4J

26. Relational
Databases

27. Performance variable
Scalability variable
Flexibility low
Complexity moderate
Functionality relational
algebra

28. Examples

29. Redis

30. Data
Types
strings
lists
sets
sorted
sets

31. In-­‐Memory
semi-­‐persistent
/
fully
persistent
master-­‐slave
replication
memory-­‐bound

32. require 'redis'
gl = Redis.new
# A string
gl['name'] = 'Kyle Rayner'
gl['name']
gl.delete('name')
# A list
gl.push_tail 'to-dos', 'Lose Ion power'
gl.push_tail 'to-dos', 'Mourn dead loved ones'
gl.push_tail 'to-dos', 'Blow up zombie lanterns'
gl.list_range('to-dos', 0, -1)

33. Tokyo
Cabinet

34. Data
Types
binary
data
strings

35. Tables!?

36. Related
Projects
tyrant
dystopia
promenade

37. require 'rufus/tokyo'
# Key-value
jli = Rufus::Tokyo::Cabinet.new('jl.tch')
jli['members'] = [
'Batman',
'Black Canary',
'Blue Beetle',
'Captain Marvel',
'Doctor Light',
'Doctor Fate',
'Guy Gardner',
'Martian Manhunter',
'Mister Miracle'
].to_yaml
YAML.load(jli['members'])

40. require 'rufus/tokyo'
# Table
big7 = Rufus::Tokyo::Table.new('big7.tct')
big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'}
big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'}
big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'}
big7['f'] = {'name' => 'Flash', 'role' => 'speedster'}
big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'}
big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'}
big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'}
big7.query {|q|
q.add_condition 'role', :streq, 'fish-talking'
}

41. Cassandra

42. Genealogy
Dynamo
BigTable

43. Column-­‐Oriented
columns
supercolumns
column
families

44. Distributed
automatic
replication
eventual
consistency
easy
expansion

45. Availability
weak
reads/writes
quorum
reads/writes

46. require 'cassandra'
op = Cassandra.new('OnePiece')
op.insert(:People, '1', {'name' => 'Luffy'})
op.insert(:People, '2', {'name' => 'Crocodile'})
op.insert(:People, '3', {'name' => 'Mr. 3'})
op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}})
op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}})
luffy_fights = op.get(:Fights, '1', 'opponents')
luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}

47. CouchDB

48. Web-­‐Inspired
JSON
storage
HTTP
/
RESTful
interface

49. Views
predeWined,
updated
incrementally
javascript
for
map/reduce

50. Updates
full,
including
embedded
documents

51. require 'couchrest'
konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')
naruto = konoha.save_doc {
'name' => 'Naruto Uzumaki',
'chakra' => 'wind'
}
shikamaru = konoha.save_doc {
'name' => 'Shikamaru Nara',
'chunin' => true
}
konoha.save_doc {
'_id' => '_design/first',
:views => {
:chunin => {
:map => 'function(doc){if(doc.chunin){emit(null, doc);}}'
}
}
}
puts konoha.views('first/chunin')['rows'].inspect

52. MongoDB

53. Storage
binary
JSON
documents
http://bsonspec.org

54. Access
native
clients

55. Queries
dynamic
index-­‐based

56. Updates
allows
partial
updates

57. require 'mongo'
avengers = Mongo::Connection.new.db('avengers')
members = avengers.collection('members')
members.insert {'name' => 'Ant-Man'}
members.insert {'name' => 'Hulk'}
members.insert {'name' => 'Iron Man'}
members.insert {'name' => 'Thor'}
members.insert {'name' => 'Wasp'}
members.create_index('name')
pym = members.find {'name' => 'Ant-Man'}
pym['name'] = 'Giant-Man'
pym.save
members.remove {'name' => 'Hulk'}
members.insert {'name' => 'Captain America'}

58. Riak

59. Ask
Sean

60. also
Web-­‐Inspired
JSON
storage
HTTP
/
RESTful
interface
links
for
relationships

61. Decentralized
no
privileged
nodes

62. ConWigurable
store
/
read
/
write

63. require 'jiak'
jc = JiakClient.new('127.0.0.1', 8098)
jc.set_bucket_schema('supervillains', {
'allowed_fields' => ['name', 'alias', 'power']
})
jc.store({
'bucket' => 'supervillains',
'key' => 'Normie',
'object' => {
'name' => 'Norman Osborn',
'alias' => 'Green Goblin',
'power' => 'Supreme jerkfacedness'
},
'links' => []
})
kth = jc.fetch('supervillains', 'Normie')

64. Neo4J

65. Structure
nodes
and
edges
key-­‐value
pairs

66. Queries
lucene
gremlin

68. require 'neo4j'
class Person
include Neo4j::NodeMixin
property :name, :mutant
index :name, :mutant
has_n :crushes
has_n :hookups
has_n :marriages
def initialize(name, mutant = true)
name = name
mutant = mutant
end
end

69. Neo4j::Transaction.run do
magneto = Person.new('Magneto')
esme = Person.new('Esme')
rogue = Person.new('Rogue')
magda = Person.new('Magda', false)
wasp = Person.new('Wasp', false)
magneto.crushes << wasp
magneto.hookups << rogue
magneto.marriages << magda
esme.crushes << magneto
rogue.hookups << magneto
magda.marriages << magneto
end

70. magneto = Person.find(:name => 'Magneto')
# Who likes Magneto?
magneto.relationships.incoming(:crushes).nodes
# Which non-mutants has Magneto dated?
magneto.hookups{ !mutant? }.to_a

71. Simulations

72. Structure

73. people
{
‘name’:‘Jimmy Olsen’
‘title’:‘Superman’s Pal’
‘company_id’:12441
}
companies
{
_id:12441
‘name’:‘Daily Planet’
}

74. Lack
of
Structure

75. mysql> SELECT * FROM people LIMIT 1 G
*************************** 1. row ***************************
id: 1
content: ---
company: Daily Planet
name: Jimmy Olsen
title: Superman’s Pal

76. But
wait!
friendfeed
friendly

77. mysql> desc people;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id | int(11) | YES | | NULL | |
| name | varchar(50) | YES | | NULL | |
+-------+-------------+------+-----+---------+-------+
mysql> desc attributes;
+-----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| id | int(11) | YES | | NULL | |
| person_id | int(11) | YES | | NULL | |
| attribute | varchar(50) | YES | | NULL | |
| value | varchar(100) | YES | | NULL | |
+-----------+--------------+------+-----+---------+-------+

78. Not
Only
SQL

79. Polyglot
Persistence

80. Caching

81. Already
in
Use
memcached

82. Queues

83. Long-­‐running
processes
resque

84. Logging

85. Rails
Log
Replacement
http://github.com/peburrows/mongo_db_logger

86. Hybrid
Domains

87. different
domains

88. Publishing
e-­‐commerce
documents

89. Dating
e-­‐commerce
social
graph

90. different
scales

91. Photo
Sharing
user
accounts
uploaded
photos

92. Next
Steps

93. Explore

94. Database
List
http://internetmindmap.com/database_software
NoSQL
Google
Group
http://groups.google.com/group/nosql-­‐discussion

95. Ignore
the
Database

96. Logical
Modeling
First
be
mindful

97. Change
the
Default

98. Application
Templates
start
with
something
new

99. benscoWield
@bscoWield
ben.scoWield@viget.com
http:/ /spkr8.com/bscoWield
http:/ /viget.com/extend
http:
/benscoWield.com
/