4 real
railswaycon 2010, berlin.
jan krutisch <jan.krutisch@mindmatters.de>
http://jan.krutisch.de/
Montag, 31. Mai 2010

http://www.cashbits.de/
Montag, 31. Mai 2010

mongodb you say?
Montag, 31. Mai 2010

document database
Montag, 31. Mai 2010

NoS
QL
inclu
ded!
document database
Montag, 31. Mai 2010

no fixed schema
no migrations
Montag, 31. Mai 2010

rich data structure
Montag, 31. Mai 2010

id title descr pos_lat pos_lng
Montag, 31. Mai 2010

{
"_id" : ObjectId("4c00245062610475a005afcd"),
"address" : "Bernstorffstr. 174n22767 HamburgnDE",
"description" : null,
"position" : {
"lat" : 53.5600912,
"lng" : 9.9596977
},
"tags" : [
"hausarzt",
"naturheilverfahren",
"akupunktur",
"allgemeinmedizin"
],
"title" : "Dr. med. Lilo Eisenbarth",
"loxicon_id" : 808261
}
Montag, 31. Mai 2010

Montag, 31. Mai 2010

Montag, 31. Mai 2010
✗

Montag, 31. Mai 2010

BSON
‣ Binary serialized JSON
‣ http://bsonspec.org/
‣ Goals: Lightweight, Traversable, Efficient
‣ Format for Datastorage and Wire
Montag, 31. Mai 2010

rich queries
Montag, 31. Mai 2010

Queries
‣ expressed BSON query documents
‣ very flexible
‣ relatively simple query expressions
‣ pretty close to SQL conceptually
‣ examples will follow
‣ on top: map/reduce for aggregation
Montag, 31. Mai 2010

Scaling
‣ Master > Slave replication
‣ Replica Pairs (with Arbiter)
‣ Replica Sets (Target: 1.6)
‣ Autosharding (Target: 1.6)
Montag, 31. Mai 2010

A few words on durability
‣ MongoDB only fsyncs every <n> seconds
‣ There‘s a desaster waiting to happen!
‣ When in production, replicate!
‣ This is not as bad as it sounds.
Montag, 31. Mai 2010

Installation/Hosting
Montag, 31. Mai 2010

OS X:
$ brew install mongodb
Montag, 31. Mai 2010

Ubuntu/Debian:
theres an apt for that
Montag, 31. Mai 2010

excu
se
the p
Ubuntu/Debian: un
theres an apt for that
Montag, 31. Mai 2010

http://www.mongodb.org/display/DOCS/Downloads
Montag, 31. Mai 2010

http://www.mongodb.org/display/DOCS/Downloads
Montag, 31. Mai 2010

also
Montag, 31. Mai 2010

http://mongohq.com/
Montag, 31. Mai 2010

http://mongomachine.com/
Montag, 31. Mai 2010

basic usage
Montag, 31. Mai 2010

$ mongo
Montag, 31. Mai 2010

> use test
switched to db test
db.quotes.save({
text: "You can observe a lot just by watching.",
from: "Yogi Berra", created_at: new Date()
});
db.quotes.save({
text: "Silence is one of the hardest arguments to refute.",
from: "Josh Billings", created_at: new Date()
});
Montag, 31. Mai 2010

let‘s query
Montag, 31. Mai 2010

db.quotes.find();
// returns all records in collection.
db.quotes.find({from: "Yogi Berra"});
{
"_id" : ObjectId("4c0022551496fc2051e93695"),
"text" : "You can observe a lot just by watching.",
"from" : "Yogi Berra",
"created_at" : "Fri May 28 2010 22:06:45 GMT+0200 (CEST)"
}
Montag, 31. Mai 2010

$lt <
$gt >
$lte <=
$gte >=
$ne !=
Montag, 31. Mai 2010

db.quotes.find({from: {"$ne": "Yogi Berra"}});
{
"_id" : ObjectId("4c0022551496fc2051e93696"),
"text" : "Silence is one of the hardest arguments to refute.",
"from" : "Josh Billings",
"created_at" : "Fri May 28 2010 22:06:45 GMT+0200 (CEST)"
}
Montag, 31. Mai 2010

$in IN (2,3,4)
$nin NOT IN
$all [2,3] ~ [1,2,3]
Montag, 31. Mai 2010

db.quotes.find({from:{ "$in":["Yogi Berra","Josh Billings"]}});
{ "_id" : ObjectId("4c0022551496fc2051e93695"), "text" : "You can..."...}
{ "_id" : ObjectId("4c0022551496fc2051e93696"), "text" : "Silence..."...}
db.arrays.save({list: [1,2,3]});
db.arrays.save({list: [4,5,6]});
db.arrays.save({list: [3,4,5]});
db.arrays.find({list:{ "$in":[3,4]}});
{ "_id" : ObjectId("4c0025461496fc2051e93697"), "list" : [ 1, 2, 3 ] }
{ "_id" : ObjectId("4c0025501496fc2051e93698"), "list" : [ 4, 5, 6 ] }
{ "_id" : ObjectId("4c0025591496fc2051e93699"), "list" : [ 3, 4, 5 ] }
db.arrays.find({list:{ "$all":[3,5]}});
{ "_id" : ObjectId("4c0025591496fc2051e93699"), "list" : [ 3, 4, 5 ] }
Montag, 31. Mai 2010

$mod yah, RLY
$size okay
$exists NOT NULL
$type huh?
Montag, 31. Mai 2010

db.arrays.find({list:{ "$mod":[4,0]}});
{ "_id" : ObjectId("4c0025501496fc2051e93698"), "list" : [ 4, 5, 6 ] }
{ "_id" : ObjectId("4c0025591496fc2051e93699"), "list" : [ 3, 4, 5 ] }
db.arrays.find({list:{ "$size":3}});
{ "_id" : ObjectId("4c0025461496fc2051e93697"), "list" : [ 1, 2, 3 ] }
{ "_id" : ObjectId("4c0025501496fc2051e93698"), "list" : [ 4, 5, 6 ] }
{ "_id" : ObjectId("4c0025591496fc2051e93699"), "list" : [ 3, 4, 5 ] }
db.arrays.find({list: {"$exists": true}});
[...]
db.arrays.find({list: {"$type": 1}});
[...]
Montag, 31. Mai 2010

...and...
Montag, 31. Mai 2010

db.quotes.find({from: /^Yog/});
[...]
db.quotes.find({from: /^Yog/});
[...]
db.quotes.find("this.from == 'Yogi Berra'");
[...]
db.quotes.find({"$where": "this.from == 'Yogi Berra'"});
[...]
Montag, 31. Mai 2010

sort()
Montag, 31. Mai 2010

db.quotes.find().sort({from:1})
{"from" : "Josh Billings" ... }
{"from" : "Yogi Berra" ...}
db.quotes.find().sort({from:-1})
{"from" : "Yogi Berra" ...}
{"from" : "Josh Billings" ... }
Montag, 31. Mai 2010

limit()
Montag, 31. Mai 2010

skip() // == OFFSET
Montag, 31. Mai 2010

count()
Montag, 31. Mai 2010

db.quotes.find().count();
2
Montag, 31. Mai 2010

Indices
Montag, 31. Mai 2010

Indexing
‣ Same concept as SQL-Indices
‣ You want them. (Same concept as with...)
‣ Sort order, unique, compound, geospatial
Montag, 31. Mai 2010

db.quotes.ensureIndex({from: 1});
db.quotes.ensureIndex({from: -1});
db.quotes.ensureIndex({text: 1}, {unique: true});
db.quotes.ensureIndex({from: 1, text: 1});
db.quotes.dropIndexes();
db.quotes.dropIndex({from: 1, text: 1});
db.quotes.reIndex();
Montag, 31. Mai 2010

map/reduce,
we can haz it, too
Montag, 31. Mai 2010

function() {
this.tags.forEach(function(z) {
emit(z, {count: 1});
});
}
function(key, values) {
var total = 0;
values.forEach(function(v) { total += v });
return {count: total}
}
Montag, 31. Mai 2010

(it‘s not fast...)
Montag, 31. Mai 2010

one more thing
Montag, 31. Mai 2010

GridFS
Montag, 31. Mai 2010

GridFS file storage
‣ Binary fields in BSON limited to 4MB
‣ GridFS API fixes that, files stored as chunks
‣ Use the language drivers
Montag, 31. Mai 2010

I‘m in u‘r rubies,
querying teh MongoDB!
Montag, 31. Mai 2010

ruby integration
‣ mongo gem
‣ bson/ bson_ext gem
‣ mongo_mapper
‣ mongoid
Montag, 31. Mai 2010

Basic driver usage
Montag, 31. Mai 2010

require 'rubygems'
require 'mongo'
db = Mongo::Connection.new.db("test")
doc = {
:text => "You can observe a lot just by watching.",
:from => "Yogi Berra",
:created_at => Time.now
}
db['quotes'].insert(doc)
db['quotes'].find.each do |row|
puts row.inspect
end
{
"_id"=>$oid4bffe2896261046e79000001,
"from"=>"Yogi Berra",
"created_at"=>Fri May 28 15:34:33 UTC 2010,
"text"=>"You can observe a lot just by watching."
}
Montag, 31. Mai 2010

require 'rubygems'
require 'mongo'
db = Mongo::Connection.new.db("test")
100.times do |i|
db['numbers'].insert({"i" => i})
end
db['numbers'].find("i" => {"$lt" => 2}).each do |row|
puts row.inspect
end
# {"_id"=>$oid4bffe4396261046f25000001, "i"=>0}
# {"_id"=>$oid4bffe4396261046f25000002, "i"=>1}
Montag, 31. Mai 2010

db['text_entries'].drop_index("tags_1")
db['text_entries'].create_index("tags")
db['text_entries'].index_information
Montag, 31. Mai 2010

GridFS usage
Montag, 31. Mai 2010

db = Mongo::Connection.new.db("test")
grid = Mongo::Grid.new(db)
id = grid.put("You can put Strings in here",
:filename => 'test.txt')
file = grid.get(id)
puts file.filename
puts file.read
grid.delete(id)
grid.put(
File.open("/Users/jankrutisch/Dropbox/Photos/IMGP8989.jpg")
)
Montag, 31. Mai 2010

fs = Mongo::GridFileSystem.new(db)
fs.open("test.txt", "w") do |f|
f.write "You can put stuff in here"
end
fs.open("test.txt", "r") do |f|
puts f.read
end
fs.delete("test.txt")
Montag, 31. Mai 2010

ODMs
Montag, 31. Mai 2010

mongo_mapper
‣ By John Nunemaker (@jnunemaker)
‣ works
‣ a few quirks
‣ almost completely undocumented
‣ Some stuff is still missing
Montag, 31. Mai 2010

Montag, 31. Mai 2010

# config/initializers/mongo_mapper.rb
File.open(File.join(Rails.root, 'config/mongodb.yml'), 'r') do |f|
@settings = YAML.load(f)[Rails.env]
end
MongoMapper.connection = Mongo::Connection.from_uri(@settings["connection"]) if @settings["connection"]
MongoMapper.database = @settings["database"]
Montag, 31. Mai 2010

class Loop
include MongoMapper::Document
key :name
key :public, Boolean
key :message_id
key :plays_and_downloads, Integer
belongs_to :user
timestamps!
end
Montag, 31. Mai 2010

@loops = Loop.all(
! :user_id => {"$exists" => true},
! :order => 'created_at DESC',
! :limit => 10
)
Montag, 31. Mai 2010

„created_at DESC“ ?!?
Montag, 31. Mai 2010

mongoid
Montag, 31. Mai 2010

mongoid
‣ By Durran Jordan (Hashrocket)
‣ Two major versions:
‣ 1.x (currently 1.9) for Rails 2.3 compatibility
‣ 2.x (currently 2.x beta) for Rails 3 compatibility
‣ Good documentation
‣ API is better (?)
Montag, 31. Mai 2010

File.open(File.join(RAILS_ROOT, 'config/mongodb.yml'), 'r') do |f|
@settings = YAML.load(f)[RAILS_ENV]
end
Mongoid::Config.instance.from_hash(@settings)
Montag, 31. Mai 2010

class Loop
include Mongoid::Document
include Mongoid::Timestamps
field :name
field :public, :type => Boolean
field :message_id
field :plays_and_downloads, :type => Integer
belongs_to_related :user
end
Montag, 31. Mai 2010

Criteria API
‣ A bit like Arel
‣ chainable method calls
‣ Named scopes
Montag, 31. Mai 2010

Embedded Documents
Montag, 31. Mai 2010

class Person
include Mongoid::Document
field :first_name
field :last_name
embeds_one :address
embeds_many :phones
end
class Address
include Mongoid::Document
field :street
field :city
field :state
field :post_code
embedded_in :person, :inverse_of => :address
end
Montag, 31. Mai 2010

GridFS
Montag, 31. Mai 2010

acts_as_attachment
Montag, 31. Mai 2010

class Loop
include Mongoid::Document
include Mongoid::Timestamps
include Mongoid::Grid
field :name
field :public, :type => Boolean
field :message_id
field :plays_and_downloads, :type => Integer
attachment :nan
end
Montag, 31. Mai 2010

using MongoDB 4 real
Montag, 31. Mai 2010

Installation is easy
Montag, 31. Mai 2010

(if you‘re using Ubuntu)
Montag, 31. Mai 2010

setting up replication
Montag, 31. Mai 2010

$ mongod --master
or
master = true # mongodb.conf
$ mongod --slave --source
slaveserver.example.com
slave = true
source = slaveserver.example.com
Montag, 31. Mai 2010

OpLog size!
Montag, 31. Mai 2010

„security“
Montag, 31. Mai 2010

memory usage?
Montag, 31. Mai 2010

limits?
Montag, 31. Mai 2010

stability?
Montag, 31. Mai 2010

bonus level
Montag, 31. Mai 2010

you‘ve made it this far!
Montag, 31. Mai 2010

MongoDB explain()ed
Montag, 31. Mai 2010

db.text_entries.find({tags: "restaurant"}).limit(10).explain();
{
! "cursor" : "BtreeCursor tags_1",
! "indexBounds" : [
! ! [
! ! ! {
! ! ! ! "tags" : "restaurant"
! ! ! },
! ! ! {
! ! ! ! "tags" : "restaurant"
! ! ! }
! ! ]
! ],
! "nscanned" : 210,
! "nscannedObjects" : 210,
! "n" : 10,
! "millis" : 0,
! [...]
}
Montag, 31. Mai 2010

My personal impression
Montag, 31. Mai 2010

Extremely easy to grasp
Montag, 31. Mai 2010

Normalisation suXXorZ
Montag, 31. Mai 2010

Migrations hurt
Montag, 31. Mai 2010

Seems to be very fast
Montag, 31. Mai 2010

Some issues for small
projects
Montag, 31. Mai 2010

I
Montag, 31. Mai 2010

thanks for listening.
Montag, 31. Mai 2010

Moi
‣ jan.krutisch@mindmatters.de
‣ http://jan.krutisch.de/
‣ http://github.com/halfbyte/
‣ http://twitter.com/halfbyte
‣ http://www.mindmatters.de/
Montag, 31. Mai 2010

Pointers
‣ http://www.mongodb.org/
‣ http://www.mongoid.org/
‣ http://github.com/jnunemaker/mongo_mapper
Montag, 31. Mai 2010