Sharding with MongoDB (Eliot Horowitz)

open-source, high-performance, document-oriented database

Why Sharding • Vertical scaling is limited by physics and cost • Hard to scale vertically in the cloud • Can scale wider than higher

What is Sharding • ad-hoc partitioning • consistent hashing (Dynamo) • range based partitioning (BigTable/PNUTS)

Overview • Automatic partitioning and management • range based • Can convert from single master to sharded system with 0 downtime • Almost no functionality lost over single master • Fully consistent

Range Based • collection is broken into chunks by range

Auto-sharding Shards mongo mongo mongo .. Conﬁg mongo mongo mongo Servers mongod mongod mongod mongos mongos .. client

Conﬁg Servers • 3 of them • changes are made with 2 phase commit • if any are down, meta data goes read only • system is online as long as 1/3 is up

Shards • Can be master, master/slave or replica sets • Replica sets gives sharding + full auto- failover • Regular mongod processes

mongos • Sharding Router • Acts just like a mongod to clients • Can have 1 or as many as you want • Can run on appserver so no extra network trafﬁc

Writes • Inserts : require shard key, routed • Removes: routed and/or scattered • Updates: routed or scattered

Queries • By shard key: routed • sorted by shard key: routed in order • by non shard key: scatter gather • sorted by non shard key: distributed merge sort

Operations • split: breaking a chunk into 2 • migrate: move a chunk from 1 shard to another

Balancing • distributes chunks automatically • can look at: disk ops, cpu, data size

Multi DataCenter • intelligent geo honing • auto failover

Sharding Limitations • Unique indexes not based on shard key • current version limited to ~ 20 petabytes

After Party! Download MongoDB http://www.mongodb.org and let us know what you think @eliothorowitz @mongodb

http://doryokujin.hatenablog.jp	2422
http://d.hatena.ne.jp	2384
http://www.slideshare.net	572
http://rafinguer.blogspot.com	327
http://www.10gen.com	86
http://rafinguer.blogspot.com.es	57
http://www.nosqldatabases.com	56
http://webcache.googleusercontent.com	38
http://rafinguer.blogspot.mx	32
http://rafinguer.blogspot.com.ar	24
http://mongospanish.blogspot.com	14
http://lanyrd.com	8
http://www.mongodb.org	6
http://translate.googleusercontent.com	5
http://www.relitec.es	4
http://localhost:8080	1
http://localhost:3000	1
http://www.lmodules.com	1
http://rafinguer.blogspot.nl	1

Sharding with MongoDB (Eliot Horowitz)

by MongoSF on May 01, 2010

Accessibility

Categories

Upload Details

Usage Rights

19 Embeds 6,039

Statistics

Sharding with MongoDB (Eliot Horowitz) Presentation Transcript