The 5 Things
You Need To Know
About Scaling
A. Jesse Jiryu Davis



Senior Python Engineer
1. Indexes
db.people.find({name: 'Jesse'}).explain()!
{!
! "cursor" : "BasicCursor",!
! "n" : 10,!
! "nscanned" : 100000,!
! "millis"...
db.people.find({name: 'Jesse'}).explain()!
{!
! "cursor" : "BasicCursor",!
! "n" : 10,!
! "nscanned" : 100000,!
! "millis"...
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!
{!
! "cursor" : "BtreeCursor name_1",!
! "n" : 1,!
! "nscanned...
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!
{!
! "cursor" : "BtreeCursor name_1",!
! "n" : 1,!
! "nscanned...
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!
{!
! "cursor" : "BtreeCursor name_1",!
! "n" : 1,!
! "nscanned...
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!
{!
! "cursor" : "BtreeCursor name_1",!
! "n" : 1,!
! "nscanned...
db.people.find({name: 'Jesse', hacks: 'Python'}).explain()!
{!
! "cursor" : "BtreeCursor name_1",!
! "n" : 1,!
! "nscanned...
2. File System
• ext3 is bad

• ext4 and xfs are good
• Set noatime

• This goes in /etc/fstab
2. File System
3. Working Set Size
3. Working Set Size
Disk!
RAM!
4. Disks
4. Disks
Option 1: Spinning disk.
100 seeks / sec
4. Disks
Option 2: Spinning disks!
~100 seeks / second!
~100 seeks / second! ~100 seeks / second! ~100 seeks / second!
4. Disks
Option 3: SSD
5. Shard
Primary!
Secondary!
Secondary!
MongoS!
Primary!
Secondary!
Secondary!
Primary!
Secondary!
Secondary!
Primary!
Secondary!
S...
1. Indexes	

2. File System	

3.Working Set	

4. Disks	

5. Shard
1. Indexes	

2. File System	

3.Working Set	

4. Disks	

5. Shard
The 5 Things You Need To
Know About Scaling
A. Jesse Jir...
Upcoming SlideShare
Loading in...5
×

Five Things you Need to Know About Scaling

534

Published on

Published in: Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
534
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
38
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Five Things you Need to Know About Scaling

  1. 1. The 5 Things You Need To Know About Scaling A. Jesse Jiryu Davis
 
 Senior Python Engineer
  2. 2. 1. Indexes
  3. 3. db.people.find({name: 'Jesse'}).explain()! {! ! "cursor" : "BasicCursor",! ! "n" : 10,! ! "nscanned" : 100000,! ! "millis" : 40,! }! ! db.people.createIndex({name: 1});! ! db.people.find({name: 'Jesse'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 10,! ! "nscanned" : 10,! ! "millis" : 0,! }! 1. Indexes
  4. 4. db.people.find({name: 'Jesse'}).explain()! {! ! "cursor" : "BasicCursor",! ! "n" : 10,! ! "nscanned" : 100000,! ! "millis" : 40,! }! ! db.people.createIndex({name: 1});! ! db.people.find({name: 'Jesse'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 10,! ! "nscanned" : 10,! ! "millis" : 0,! }! 1. Indexes
  5. 5. db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 10,! ! "millis" : 0,! }! ! db.people.createIndex({name: 1, hacks: 1});! ! db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 1,! ! "millis" : 0,! }! 1. Indexes
  6. 6. db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 10,! ! "millis" : 0,! }! ! db.people.createIndex({name: 1, hacks: 1});! ! db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 1,! ! "millis" : 0,! }! 1. Indexes
  7. 7. db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 10,! ! "millis" : 0,! }! ! db.people.createIndex({name: 1, hacks: 1});! ! db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 1,! ! "millis" : 0,! }! 1. Indexes
  8. 8. db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 10,! ! "millis" : 0,! }! ! db.people.createIndex({name: 1, hacks: 1});! ! db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1_hacks_1",! ! "n" : 1,! ! "nscanned" : 1,! ! "millis" : 0,! }! ! db.people.dropIndex({name: 1})! ! 1. Indexes
  9. 9. db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1",! ! "n" : 1,! ! "nscanned" : 10,! ! "millis" : 0,! }! ! db.people.createIndex({name: 1, hacks: 1});! ! db.people.find({name: 'Jesse', hacks: 'Python'}).explain()! {! ! "cursor" : "BtreeCursor name_1_hacks_1",! ! "n" : 1,! ! "nscanned" : 1,! ! "millis" : 0,! }! ! db.people.dropIndex({name: 1})! ! 1. Indexes
  10. 10. 2. File System • ext3 is bad
 • ext4 and xfs are good
  11. 11. • Set noatime
 • This goes in /etc/fstab 2. File System
  12. 12. 3. Working Set Size
  13. 13. 3. Working Set Size Disk! RAM!
  14. 14. 4. Disks
  15. 15. 4. Disks Option 1: Spinning disk. 100 seeks / sec
  16. 16. 4. Disks Option 2: Spinning disks!
  17. 17. ~100 seeks / second! ~100 seeks / second! ~100 seeks / second! ~100 seeks / second!
  18. 18. 4. Disks Option 3: SSD
  19. 19. 5. Shard
  20. 20. Primary! Secondary! Secondary! MongoS! Primary! Secondary! Secondary! Primary! Secondary! Secondary! Primary! Secondary! Secondary! 5. Shard
  21. 21. 1. Indexes 2. File System 3.Working Set 4. Disks 5. Shard
  22. 22. 1. Indexes 2. File System 3.Working Set 4. Disks 5. Shard The 5 Things You Need To Know About Scaling A. Jesse Jiryu Davis
 bit.ly/scaling-five
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×