Benefits of using MongoDB: Reduce Complexity & Adapt to Changes

RedDotRubyConf 2011Beneﬁts of MongoDB: Reduce Complexity & Adapt to Changes Vinova Pte Ltd

About me• Alex Nguyen• Co-founder at Vinova• http://vinova.sg/• https://github.com/vinova/

Agenda• What’s MongoDB?• Why MongoDB reduce complexity?• Why MongoDB adapt to changes better?• Case studies

I don’t hate SQLJust found a better tool for most of my use cases

What’s MongoDB? “MongoDB (from "humongous") is a scalable, high- performance, open source, document-oriented database”mongodb.org

What’s MongoDB?http://www.slideshare.net/kbanker/mongodb-schema-design-mongo-chicago

What’s MongoDB?• Collections ~ Tables• Documents ~ Rows

MongoDB Philosophy • Reduce transactional semantics for performance • No-relational is the best way to scale horizontallymongodb.org

MongoDB Features• JSON style documents • Map / Reduce• Index on any attribute • GridFS to store ﬁles• Rich queries • Server-side JavaScript• In-place update • Capped collections• Auto-sharding • Full-text-search (coming soon)

MongoDBs ﬂexibility data structure, ability to index &query data, and auto-sharding make it a strong tool thatadapt to changes well. It also help to reduce complexity comparing to tradition RDBMS.

Why MongoDB reduce complexity?• Get rid of migrations• Get rid of relationships (most of)• Reduce number of database requests• JSON (client, server, and database)

Get rid of migrations• No create table• No alter column• No add column• No change column

Get rid of relationships• Many one-to-one and one-to-many relationships is not necessary • User :has_one :setting • User :has_many :addresses • User :has_many :roles • Post :has_many :tags

Reduce number of database requests• Pre-joined• Rich queries• Atomic, in-place updates

JSON• MongoDB knows JSON• Don’t have to convert data from / to JSON

Adapt to changes• Changes in schema• Changes in data & algorithms• Changes for performance & scaling

Changes in schema• In modern apps, schema changes quite often (weekly, monthly ...)• Alter tables are expensive in RDBMS• Dynamic schema document makes those changes seamlessly

Changes in data & algorithms• Atomic, in-place updates are very powerful to modify data $inc, $set, $unset, $push, $pop, $rename, $bit• Rich$all, $exists,and aggregators $in, queries $size, $type, regexp count(), size(), distinct(), min(), max()• Map/Reduce

Changes forperformance & scaling• Very fast & ready to scale => • Don’t have to use additional tools (memcached ...) • Don’t have to change platforms

Case Studies• Store crawled info as embedded documents• Product listing• Find unique slug• Voting

Store crawled info asembedded documents• Data from 3rd party sources• Sources and data formats can be changed in the future

Store crawled info asembedded documents product = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "amazon" : { "asin" : ..., "price" : ..., .... } };

Store crawled info asembedded documents product = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "amazon" : { "asin" : ..., "price" : ..., "shipping_cost" : ..., ... } };

Store crawled info asembedded documents product = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "amazon" : { "asin" : ..., "price" : ..., "shipping_cost" : ..., .... }, "walmart" : { "price" : ..., ... } };

Store crawled info asembedded documents def Product.find_by_asin(asin) Product.where(amazon.asin => asin).first end

Product listing• A product can be listed on multiple categories on certain months

Product listing• Need an extra table to express which product is listed in which category and on which month product_id category_id month 1 2 2011-03 1 2 2011-04 SQL

Product listing • To query products listed in category 2 and month ‘2011-04’Product.join(:listings).where(category_id = ? AND month = ?, 2,‘2011-04’) SQL

Product listing • Store listings in product itselfproduct = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "listings" : [ [1, "2011-01"], [1, "2011-04"], [3,"2011-01"] ]}; Mongo

Product listing • Store listings in product itselfproduct = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "listings" : [ [1, "2011-01"], [1, "2011-04"], [3,"2011-01"] ]}; • Query is simplerProduct.where("listings" => [1, 2011-04]) Mongo

Product listing • Store listings in product itselfproduct = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "listings" : [ [1, "2011-01"], [1, "2011-04"], [3,"2011-01"] ]}; • Query is simplerProduct.where("listings" => [1, 2011-04]) • Can index listings arraydb.products.ensureIndex({"listings" : 1 }); Mongo

Simplify product listing • Clearer but more data storageproduct = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "name" : "Product ABC", "listings" : [ {"category_id" : 1, "month" : "2011-01" }, {"category_id" : 1,"month" : "2011-04" }, {"category_id" : 3, "month" : "2011-01" }]};db.products.find("listings" : {"category_id" : 1, "month" :"2011-04" }) Mongo

Find unique slug • book1 = #<Book id: .., title => “Ruby”, ... > • book2 = #<Book id: .., title => “Ruby”, ... > • book2.uniq_slug => /books/ruby-1 • Need n queries to ﬁnd an unique slugdef uniq_slug slug = original_slug = title.to_slug counter = 0 while (where(:slug => slug).count > 0) counter += 1 slug = "#{original_slug}-#{counter}" end slugend SQL

Find unique slug • Need one query using regexp matchingdef find_uniq_slug original_slug = title.to_slug slug_pattern = /^#{original_slug}(-d+)?$/ book = where(:slug => slug_pattern). order(:slug.desc).limit(1) if book max_counter = book.slug.match(/-(d+)$/)[1].to_i "#{original_slug}-#{max_counter + 1}" else original_slug endenddb.books.ensureIndex({"slug" : -1 }) Mongo

Voting• A user can only vote each post once• up / down votes has different points• Cached votes_count and votes_point in post for sorting and querying • Post.max(:votes_point) • Post.order_by(:votes_count.desc)

Voting• Use extra votes table to store vote data SQL

Votingdef vote(user_id, post_id, value) # Validate not_voted = Vote.where(:user_id => user_id, :post_id => post_id).count == 0 if not_voted # Create a new vote Vote.create( :user_id => user_id, :post_id => post_id, :value => value ) # Get post post = Post.find(post_id) # Update votes_point & votes_count post.votes_point += POINT[value] post.votes_count += 1 post.save endend SQL

Votingdef vote(user_id, post_id, value) # Validate not_voted = Vote.where(:user_id => user_id, :post_id => post_id).count == 0 if not_voted # Create a new vote Vote.create( :user_id => user_id, 4 requests :post_id => post_id, :value => value ) # Get post post = Post.find(post_id) # Update votes_point & votes_count post.votes_point += POINT[value] post.votes_count += 1 post.save endend SQL

Votingdef unvote(user_id, post_id) # Get current vote vote = Vote.where(:user_id => user_id, :post_id => post_id).first # Check if voted if vote # Destroy vote vote.destroy # Get post post = Post.find(post_id) # Update votes_point & votes_count post.votes_point -= POINT[vote.value] post.votes_count -= 1 post.save endend SQL

Votingdef unvote(user_id, post_id) # Get current vote vote = Vote.where(:user_id => user_id, :post_id => post_id).first # Check if voted if vote # Destroy vote 4 requests vote.destroy # Get post post = Post.find(post_id) # Update votes_point & votes_count post.votes_point -= POINT[vote.value] post.votes_count -= 1 post.save endend SQL

Voting • Embed votes data to post • use arrays to store who vote up and who vote downpost = { "_id" : ObjectId("4d8ace4b0dc3e43231bb930d"), "title" : "Post ABC", .... "votes" : { "up" : [ user_id_1 ], "down" : [ user_id_2 ], "count" => 2, "point" => -1 Mongo }};

def vote(user_id, post_id, value) # Find post with post_id that was not up voted or down voted by user_id query = { post_id => post_id, votes.up => { $ne => user_id }, votes.down => { $ne => user_id } } # Push user_id to votes.up_ids if vote up or votes.down_ids if vote_down # and update votes.point and votes.count update = { $push => { (value == :up ? votes.up : votes.down) => user_id }, $inc => { votes.point => POINT[value], votes.count => +1 } } # Validate, update and get result post = Post.collection.find_and_modify( :query => query, :update => update, :new => true # return post after update votes data )end Mongo

def vote(user_id, post_id, value) # Find post with post_id that was not up voted or down voted by user_id query = { post_id => post_id, votes.up => { $ne => user_id }, votes.down => { $ne => user_id } } # Push user_id to votes.up_ids if vote up or votes.down_ids if vote_down # and update votes.point and votes.count update = { $push => { (value == :up ? votes.up : votes.down) => user_id }, $inc => { votes.point => POINT[value], votes.count => +1 } } # Validate, update and get result post = Post.collection.find_and_modify( :query => query, one request :update => update, :new => true # return post after update votes data )end Mongo

def unvote(user_id, post_id) # Find post with post_id that was up voted or down voted by user_id query = { post_id => post_id, $or => { votes.up => user_id, votes.down => user_id } } # Pull user_id from both votes.up_ids and votes.down_ids # and update votes.point and votes.count update = { $pull => { votes.up => user_id, votes.down => user_id }, $inc => { votes.point => -POINT[value], votes.count => -1 } } # Validate, update and get result post = Post.collection.find_and_modify( :query => query, :update => update, :new => true # return post after update votes data )end Mongo

def unvote(user_id, post_id) # Find post with post_id that was up voted or down voted by user_id query = { post_id => post_id, $or => { votes.up => user_id, votes.down => user_id } } # Pull user_id from both votes.up_ids and votes.down_ids # and update votes.point and votes.count update = { $pull => { votes.up => user_id, votes.down => user_id }, $inc => { votes.point => -POINT[value], votes.count => -1 } } # Validate, update and get result post = Post.collection.find_and_modify( :query => query, one request :update => update, :new => true # return post after update votes data )end Mongo

Voting• For a complete solution:• gem install voteable_mongoid• visit https://github.com/vinova/voteable_mongoid

Summary• MongoDB is • Flexible • Powerful • Fun

Thank you Alex Nguyen @tiendungalex@vinova.sg

ReferencesIntroduction to MongoDB • http://scribd.com/doc/26506063/Introduction-To-MongoDB • http://slideshare.net/jnunemaker/why-mongodb-is-awesomeSchema Design • http://slideshare.net/kbanker/mongodb-schema-design-mongo-chicagoIndexing & Query Optimization • http://slideshare.net/mongodb/indexing-with-mongodb • http://slideshare.net/mongodb/mongodb-indexing-the-details

http://www.qlazzy.com	66
http://stnguyen.com	32
http://sontung23.wordpress.com	27
http://localhost:4000	21
http://imperavi.com	4
http://www.raptor-editor.com	1
http://nicedit.com	1

Benefits of using MongoDB: Reduce Complexity & Adapt to Changes

by Alex Nguyen on Apr 23, 2011

Accessibility

Categories

Upload Details

Usage Rights

7 Embeds 152

Statistics

Benefits of using MongoDB: Reduce Complexity & Adapt to Changes Presentation Transcript