MongoDB Best Practices for Developers

© All rights reserved: Moshe Kaplan Big Data – Leading Platforms© All rights reserved: Moshe Kaplan MongoDB for Developers
For Developers

MongoDB
For Developers
Moshe Kaplan
Scale Hacker
http://top-performance.blogspot.com
http://blogs.microsoft.co.il/vprnd

© All rights reserved: Moshe Kaplan Big Data – Leading Platforms
It’s all About
Scale

HELLO. MY NAME IS MONGODB
Introduction

Who is Using mongoDB?
5

Who is Behind mongoDB

Key Value Store (with benefits)
• insert
• get
• multiget
• remove
• truncate
7
<Key, Value>
://wiki.apache.org/cassandra/API

When Should I Choose NoSQL?
• Eventually Consistent
• Document Store
• Key Value
8
http://guyharrison.squarespace.com/blog/tag/nosq

What mongoDB is Made of?
9
http://www.10gen.com/products/mongodb

Why MongoDB?
What? Why?
JSON End to End
No Schema “No DBA”, Just Serialize
Write 10K Inserts/sec on virtual machine
Read Similar to MySQL
HA 10 min to setup a cluster
Sharding Out of the Box
GeoData Great for that
No Schema None: no downtime to create new columns
Buzz Trend is with NoSQL
10

NoSQL and Data Modeling
What is the Difference

Database for Software Engineers
Class
Subclass
Document
Subdocument

Same Terminology
• Database  Database
• Table  Collection
• Row  Document

A Blog Case Study in MySQL
http://www.slideshare.net/nateabele/building-apps-with-mongodb

as a SW Engineer would like it to be…

Migration from RDBMS to NoSQL
How to do that?

Data Migration
• Map the table structure
• Export the data and Import It
• Add Indexes
http://igcse-geography-lancaster.wikispaces.com/1.2+MIGRATION

Selected Migration Tool

Usage Details> Install ruby
> gem install mongify
… Modify the code to your needs
… Create configuration files
> mongify translation db.config >
translation.rb
> mongify process db.config translation.rb

Date Functions
• Year(), Month()… function included
• … buy only in the JavaScript engine
• Solution: New fields!
• [original field]
• [original field]_[year part]
• [original field]_[month part]
• [original field]_[day part]
• [original field]_[hour part]

NO SCHEMA IS A GOOD THING BUT…
Schemaless

Default Values
• No Schema
• No Default Values
• App Challenge
• Timestamps…
No single source of truth

Casting and Type Safety
• No Schema
• No …
• App Challenge

Auto Numbers
• Start using _id
{
"_id" : 0,
"health" : 1,
"stateStr" : "PRIMARY",
"uptime" : 59917
}
• Counter tables
• Dedicated database
• 1:1 Mapping
• Counter++ using findAndModify

The ORM Solution

Data Analysts
http://www.designersplayground.com/pr/internet-meme-list/data-analyst-2/

Data Analysts
• This is not SQL
• There are no joins
• No perfect tools
Pentaho
RockMongoMongoVUE RoboMongo

No Joins
• Do in the application
• Leverage the power of NoSQL

Limited Resultset
• 16MB document size
• Limit and Skip
• Adjusted WHERE
• GridFS

Bottom Line
• Powerful tool
• Embrace the Challenge
• Schema-less limitations: counters, data types
• Tools for Data Scientists
• Data design

Mastering a New Query
Language

Connect to the Database
• Connect:
• > mongo
• Show current database:
• >> db
• Show Databases
• >> show databases;
• Show Collections
• >> show collections; or show tables;

Databases Manipulation: Create & Drop
• Change Database:
• >> use <database>
• Create Database
• Just switch and create an object…
• Delete Database
• > use mydb;
• > db.dropDatabase();

Collections Manipulation
• Create Collcation
>db.createCollection(collectionName)
• Delete Collection
> db.collectionName.drop()
Or just insert to it

SELECT: No SQL, just ORM…
• Select All
• db.things.find()
• WHERE
• db.posts.find({“comments.email” : ”b@c.com”})
• Pattern Matching
• db.posts.find( {“title” : /mongo/i} )
• Sort
• db.posts.find().sort({email : 1, date : -1});
• Limit
• db.posts.find().limit(3)

Specific fields
Select All
db.users.find(
{ },
{ user_id: 1, status: 1, _id: 0 }
)
1: Show; 0: don’t show

WHERE
• != “A” { $ne: "A" }
• > 25 { $gt: 25 }
• > 25 AND <= 50 { $gt: 25, $lte: 50 }
• Like ‘bc%’ /^bc/
• < 25 OR >= 50 { $or : [ { $lt: 25 }, { $gte : 50 } ] }

Join
• Wrong Place…
• Or Map Reduce

39
 db.article.aggregate(
 { $group : {
 _id : { author : "$author“, name : “$name” },
 docsPerAuthor : { $sum : 1 },
 viewsPerAuthor : { $sum : "$pageViews" }
 }}
 );
GROUP BY
< GROUP BY author, name
< SUM(pageViews)
< SUM(1) = N

40
db.posts.update(
{“comments.email”: ”b@c.com”},
{$set : {“comments.email”: ”d@c.com”}}
}
SET age = age + 3
• db.users.update(
• { status: "A" } ,
• { $inc: { age: 3 } },
• { multi: true }
• )
UPDATE

41
j = { name : "mongo" }
k = { x : 3 }
db.things.insert( j )
db.things.insert( k )
INSERT

42
db.users.remove(
{ status: "D" }
)
DELETE

Performance Tuning
Make a Change

MONGODB TUNING

journalCommitInterval = 300:
Write to disk: 2ms <= t <= 300ms
Default 100ms, increase to 300ms to save resources
Disk
The Journal
Memory
Journal Data
1 2

RAM Optimization:
dataSize + indexSize < RAM
OS
Data Index
Journal

PROFILING AND SLOW LOG

Profiling Configuration
• Enable:
• mongod --profile=1 --slowms=15
• db.setProfilingLevel([level] , [time])
• How much:
• 0 (none)  1 (slow queries only)  2 (all)
• 100ms: default
• Where:
• system.profile collection @ local db

Profiling Results Analysis
• Last 5 >1ms: show profile
• w/o commands:
db.system.profile.find( { op: { $ne : 'command' } } ).pretty()
• Specific database:
db.system.profile.find( { ns : 'mydb.test' } ).pretty()
• Slower than:
db.system.profile.find( { millis : { $gt : 5 } } ).pretty()
• Between dates:
db.system.profile.find({ts : {
$gt : new ISODate("2012-12-09T03:00:00Z") ,
$lt : new ISODate("2012-12-09T03:40:00Z")
}}).pretty()

Explain
> db.courses.find().explain();
{ "cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 11, “nscannedObjects" : 11, "nscanned" : 11,
"nscannedObjectsAllPlans" : 11, "nscannedAllPlans" : 11,
"scanAndOrder" : false, "indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {},
"server" : "primary.domain.com:27017"
}

INDEXES

Index Management
• Regular Index
• db.users.ensureIndex( { user_id: 1 } )
• Multiple + DESC Index
• db.users.ensureIndex( { user_id: 1, age: -1 } )
• Sub Document Index
• db.users.ensureIndex( { address.zipcode: 1 } )
• List Indexes
• db.users.getIndexes()
• Drop Indexes
• db.users.dropIndex(“indexName”)

Known Index Issues
• Bound filter should be the last (in the index as well).
• BitMap Indexes not really working
• You should design your indexes carefully

STATS &
SCHEMA DESIGN

Sparse Matrix? I don’t Think so
• mongostat
• > db.stats();
• > db.collectionname.stats();
• Fragmentation if storageSize/size > 2
• db.collectionanme.runCommand(“compact”)
• Padding (wrong design) if paddingFactor > 2

High Availability
Going Real Time

(Do Not) Master/Slave

• In mongo.conf
• # Replication Options
• replSet=myReplSet
• > rs.initiate()
• > rs.conf()
• > rs.add(“host:port")
• rs.reconfig()
Replication Set

• rs.addArb(“host:port")
• Also:
• Low Priority
• Hidden
• (Weighted) Voting
Arbiter

Show Status: rs.status();
• {"set" : “myReplSet", "date" : ISODate("2013-02-05T10:23:28Z"),
• "myState" : 1,
• "members" : [
• {
• "_id" : 0, "name"
: "primary.example.com:27017",
• "health" : 1, "state" :
1,
• "stateStr" : "PRIMARY",
"uptime" : 164545,
• "optime" : Timestamp(1359901753000, 1),
• "optimeDate" : ISODate("2013-02-
03T14:29:13Z"), "self" : true
• },
• {
• "_id" : 1, "name"

Replica Set Recovery
• Create a new mongod
• Either install a plain vanilla
• Or duplicate existing mongod (better)
• Connect to the system
• Use the previous machine IP
• Or change configuration to remove old and add new

Sharding and Scale out:
Make a big Change
Map Reduce and Aggregation

Secondary Read Enabling

The Strategy : Sharding

MongoDB Implementation

Summary
• NoSQL
• Schemaless
• HA
• Sharding

Thank You !
Moshe Kaplan
moshe.kaplan@brightaqua.com
054-2291978

MongoDB Best Practices for Developers

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to MongoDB Best Practices for Developers

Similar to MongoDB Best Practices for Developers (20)

More from Moshe Kaplan

More from Moshe Kaplan (20)

Recently uploaded

Recently uploaded (20)

MongoDB Best Practices for Developers