MongoDB is the MashupDB

Stash the Hash!MongoDB is the MashupDB Mongo Austin February 15, 2011 WYNNNETHERLAND

whoami

Two years ago...

x No SQL? x

MongoDB x

Lets just skip to here already!x

One year ago

CTO of AmazonNoSQL Smackdown!

is it web scale?

How do we get to here? xx

Enlightenment

Almost ^ Two years with

Our journey

TweetCongress.org

“We the Tweeple of the UnitedStates, in order to form a moreperfect government, establishcommunication, and promotetransparency do hereby tweet theCongress of the United States ofAmerica.”

and aggregator ^The idea: A Twitter directory for the US Congress

When does NOSQL make sense?★ Your data is stored and retrieved mainly by primary key, without complex joins.★ You have a non-trivial amount of data, and the thought of managing lots of RDBMS shards and replication failure scenarios gives you the fear.http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/

Key value stores Project Voldemort me ver y cool Ringo Scalaris So space Kai ojects in this Dynomite MemcacheDB pr ThruDB CouchDB Cassandra HBase Hypertable Redis Tokyo Cabinet/Tyrant Riak Neo4J

Tokyo Cabinet

CouchDBApache CouchDB is a distributed, fault-tolerant andschema-free document-oriented database accessible viaa RESTful HTTP/JSON API.http://couchdb.apache.org/

SQL CouchDB Prede ned, explicit schema Dynamic, implicit schema Collection of named documents with varying Uniform tables of data structure Normalized. Objects spread across tables. Denormalized. Docs usually self contained. Data Duplication reduced. often duplicated. Must know schema to read/write a complete Must know only document name object Dynamic queries of static schemas Static queries of dynamic schemas http://damienkatz.net/ﬁles/What is CouchDB.pdfhttp://damienkatz.net/ﬁles/What is CouchDB.pdf

SQL CouchDB Prede ned, explicit schema Dynamic, implicit schema Collection of named documents with varying Uniform tables of data structure Normalized. Objects spread across tables. Duplication reduced. TheDenormalized. Docs usually selfdetails devilsoften duplicated.contained. Data in the Must know schema to read/write a complete Must know only document name object Dynamic queries of static schemas Static queries of dynamic schemas http://damienkatz.net/ﬁles/What is CouchDB.pdfhttp://damienkatz.net/ﬁles/What is CouchDB.pdf

Mongo speaks JSON

Document Storage (BSON) B is for Binary { author: joe, created: Date(03-28-2009), title: Yet another blog post, text: Here is the text..., tags: [ example, joe ], comments: [ { author: jim, comment: I disagree }, { author: nancy, comment: Good post } ] } Sure wish JSON did this...http://www.mongodb.org/display/DOCS/BSON

That looks like JSON

versatility

Runs like Hayes.Hits like Mays.

Introducing★ Built For Speedcan be very fast★ Dynamic Queries and Indexes★ Replication and Failover★ Sharding★ Map / Reduce★ Geospatial stuﬀs

MongoDB is great for ★ Websites Your own queryable API mirror ★ Caching ★ High volume, low value ★ High scalability stash the hash ★ Storage of program objects and JSON

Not as great for ★ Highly transactional ★ Financial stuﬀs ★ Problems requiring SQL

Installation ★ mkdir -p /data/db ★ download pre-built for OSX and unzip to /usr/local/ ★ cp -R /usr/local/pathtomongo/bin /usr/local/bin Ruby driver for MongoDB ★ gem install mongo Native C ★ gem install mongo_ext extensions ( go turbo! ) ★ gem install mongo_mapper

brew install mongodb

Contents of mongo/bin whats in the box? ★ mongod - The MongoDB server ★ mongo - the JavaScript interactive shell ★ mongoexport - export data as JSON or csv ★ mongoimport - As advertised was? ★ mongodump - Like mysqldump why? What did you think it ★ mongorestore - Restore from mongodump les ★ mongos - Auto-sharding module (getting better with every build)

Some new terms

When I say think database database Well that one isnt new...

Databases in MongoDB ★ Made up of multiple collections ★ Are created on-the- y when rst referenced

When I say think collection table

Collections in MongoDB ★ Schema-less but typed! ★ For grouping documents into smaller query sets (speed) ★ Indexable by one or more key ★ Are created on-the- y when rst referenced ★ Capped collections: Fixed size, older records dropped after limit reached

When I say think document record or row

The Ruby ecosystem

The Ruby driver

Queryingdb.collection.find({first_name: John}) # finds all Johnsdb.collection.find({first_name: /^wynn/i}) # regexdb.collection.find_first({_id:1}) # finds first with _id of 1db.collection.find({age: {$gte: 21}}) # finds possible drinkersdb.collection.find({author.first_name:John}) # subdocumentdb.collection.find({$where:this.age >= 6 && this.age <= 18})

More Querying $in, $nin, $all, $ne, $gt, $gte, $lt, $lte, $size, $where :fields (like :select in active record) :limit, :offset for pagination :sort ascending or descending [[foo, 1], [bar, -1]] count and group (uses map/reduce) db.collection.mapReduce(mapfunction,reducefunction[,options]);

Updating$inc, $set, $unset, $push, $pushAll, $addToSet,$pop, $pull, $pullAll, $rename, $bit

akfast w it hB re Nune makerJohn

MongoMapper from @jnunemaker ★ Mongo is not MySQL ★ DSL for modeling domain should also teach you Mongo ★ Now in version 0.8.6 I voted for "Nunemapper"

Features★ Typecasting★ Callbacks (ActiveSupport Callbacks)★ Validations★ Connection and database can diﬀer per document★ Create and Update with single or multiple★ Delete and Destroy and _all counterparts Be careful. Ordering can be tricky.★ Find: id, ids, :all, : rst, :last★ Associations

Exampleclass User include MongoMapper::Document key :name, String, :required => true, :length => 5..100 key :email, String, :required => true, :index => true key :age, Integer, :numeric => true key :active, Boolean, :default => true one :address many :articlesend Included as module, not subclassedclass Address include MongoMapper::Document key :street, String key :city, String key :state, String, :length => 2 key :zip, Integer, :numeric => true, :length => 5end

QueriesUser.where(:age.gt => 27).sort(:age).allUser.where(:age.gt => 27).sort(:age.desc).allUser.where(:age.gt => 27).sort(:age).limit(1).allUser.where(:age.gt => 27).sort(:age).skip(1).limit(1).all

Scopesclass User include MongoMapper::Document # plain old vanilla scopes with fancy queries scope :johns, where(:name => John) # plain old vanilla scopes with hashes scope :bills, :name => Bill # dynamic scopes with parameters scope :by_name, lambda { |name| where(:name => name) } scope :by_ages, lambda { |low, high| where(:age.gte => low, :age.lte => high) } # Yep, even plain old methods work as long as they return a query def self.by_tag(tag) where(:tags => tag) end # You can even make a method that returns a scope def self.twenties; by_ages(20, 29) end key :name, String key :tags, Arrayend

Scopes# simple scopespp User.johns.firstpp User.bills.first# scope with argpp User.by_name(Frank).first# scope with two argspp User.by_ages(20, 29).all# chaining class methods on scopespp User.by_ages(20, 40).by_tag(ruby).all# scope made using method that returns scopepp User.twenties.all

Mongoid

Mongomatic

Mongomaticrequire mongomatic class User < Mongomatic::Base def validate self.errors.add "name", "cant be empty" if self["name"].blank? self.errors.add "email", "cant be empty" if self["email"].blank? endend # set the db for all models:Mongomatic.db = Mongo::Connection.new.db("mongomatic_test")# or you can set it for a specific model:User.db = Mongo::Connection.new.db("mongomatic_test_user")

MongomaticUser.empty?=> trueu = User.new(:name => "Ben")=> #<user:0x00000100d0cbf8 @doc="{"name"=">"Ben"}, @removed=false>u.valid?=> falseu["email"] = "me@somewhere.com"u.valid?=> trueu.insert=> BSON::ObjectId(4c32834f0218236321000001) User.empty?=> false

MongoDB is open source

Mongo runs in the cloud

#mongodb

How can you help? ★ We need an awesome admin GUI ★ Port some plugins (might get easier with ActiveModel support coming soon) ★ Build something cool

Lessons learned in production the fine print ★ The laws of computing are still in eﬀect Upserts FTW! ★ Indexes are important no matter what the salesman told ya about performance ★ Data modeling. Deep or Wide? The answer is yes! ★ MongoDB and MongoMapper are in active development Very responsive yet very volatile changes!

Get the backstory on

Questions? wynn@hp.com @pengwynn

http://www.mongodb.org	3
http://paper.li	2
http://www.slideshare.net	1
http://www.2g-cenergy.com	1
http://janeknight.typepad.com	1
http://users.sch.gr	1
http://www.istikbal.gr	1

MongoDB is the MashupDB

by Wynn Netherland on Feb 16, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

7 Embeds 10

Statistics

MongoDB is the MashupDB — Presentation Transcript