2011 mongo sf-schemadesign

Schema Design at Scale Eliot Horowitz @eliothorowitz MongoSF May 24, 2011

Schema• Single biggest performance factor• More choices than in an RDBMS• Embedding, index design, shard keys

Embedding• Great for read performance• One seek to load entire object• One roundtrip to database• Writes can be slow if adding to objects all the time• Should you embed comments?

Blog Post - Embedded { _id : “/post/eliot/2011-05-24/1”), author : "eliot", text : "About MongoDB...", tags : [ "tech", "databases" ], comments : [ { author : "Fred", date : "Sat Apr 25 2010 20:51:03GMT-0700", text : "Best Post Ever!" } ]}

Blog Post - Not Embedded blog.posts { _id : “/post/eliot/2011-05-24/1”), author : "eliot", text : "About MongoDB...", tags : [ "tech", "databases" ] } blog.comments { post : “/post/eliot/2011-05-24/1” author : "Fred", date : "May 24 2011", text : "Best Post Ever!" }

Blog Post - Hybridblog.comments { _id : “/post/eliot/2011-05-24/1---1” comments : [ { author : "Fred", date : "May 24 2011", text : "Best Post Ever!" } , { author : "Bob", date : "May 24 2011", text : "Awesome" } , ] }

Indexes• Index common queries• Make sure there aren’t duplicates: (A) and (A,B) aren’t needed• Right-balanced indexes keep working set small

Random Index Access Have to keepentire index in ram •email address •hash

Right-Balanced Index AccessOnly have to keep small portion in •Time Based ram •ObjectId •Auto Increment

Covered Indexes• Keep data sequential in index• find( { email : “eliot@10gen.com” } , { first : 1 , last : 1 , state : 1 } )• index: { email : 1 , first : 1 , last : 1 , state : 1 }

Choosing a Shard Key• Shard key determines how data is partitioned• Hard to change• Most important performance decision

Range Based• collection is broken into chunks by range• chunks default to 200mb or 100,000 objects

Use Case: User Proﬁles { email : “eliot@10gen.com” , addresses : [ { state : “NY” } ] }• Shard by email• Lookup by email hits 1 node• Index on { “addresses.state” : 1 }

Use Case: Activity Stream { user_id : XXX, event_id : YYY , data : ZZZ }• Shard by user_id• Looking up an activity stream hits 1 node• Writing even is distributed• Index on { “event_id” : 1 } for deletes

Use Case: Photos { photo_id : ???? , data : <binary> } What’s the right key?• auto increment• MD5( data )• now() + MD5(data)• month() + MD5(data)

Use Case: Logging { machine : “app.foo.com” , app : “apache” , when : “2010-12-02:11:33:14” , data : XXX } Possible Shard keys• { machine : 1 }• { when : 1 }• { machine : 1 , app : 1 }• { app : 1 }

Download MongoDB http://www.mongodb.org and let us know what you think @eliothorowitz    @mongodb 10gen is hiring!http://www.10gen.com/jobs

http://www.10gen.com	4859
http://vermelho.jugem.jp	51
url_unknown	17
http://www.slideshare.net	9
http://translate.googleusercontent.com	6
http://blog.naver.com	4
http://207.46.192.232	2
http://a0.twimg.com	1
http://paper.li	1
http://localhost:8080	1

2011 mongo sf-schemadesign

by Dwight Merriman on Jun 01, 2011

Accessibility

Tags

Upload Details

Usage Rights

10 Embeds 4,951

Statistics

2011 mongo sf-schemadesign — Presentation Transcript