1. High
Performance, Scala
ble MongoDB in a Bare
Metal Cloud
Harold Hannon, Sr. Software Architect

2. 100k servers
24k customers
23
million domains

3. 13 data centers
Reach 16 network POPs
20Gb fiber interconnects

6. The 3 V’s

7. Physical Deployment

8. Public Cloud

9. The Dream

10. The Reality

11. Deployment Complexity

12. Bare Metal Cloud

13. Bare Metal
• Build to your specs
• Robust, quickly scaled environment
• Management of all aspects of environment
• Great for Big Data Solutions like MongoDB

14. Cloud Subscription
• Preconfigured
• Performance Tuned
• Bare Metal Single Tenant
• Complex Environment Configurations

15. Pre-configurations
• Set SSD Read Ahead Defaults to 16 Blocks – SSD drives have
excellent seek times allowing for shrinking the Read Ahead to
16 blocks. Spinning disks might require slight buffering so these
have been set to 32 blocks.
• noatime – Adding the noatime option eliminates the need for
the system to make writes to the file system for files which are
simply being read — or in other words: Faster file access and
less disk wear.

16. • Turn NUMA Off in BIOS – Linux, NUMA and MongoDB
tend not to work well together. If you are running
MongoDB on NUMA hardware, we recommend turning it
off (running with an interleave memory policy). If you
don’t, problems will manifest in strange ways like
massive slow downs for periods of time or high system
CPU time.
• Set ulimit – We have set the ulimit to 64000 for open files
and 32000 for user processes to prevent failures due to a
loss of available file handles or user processes.

17. Use ext4 – We have selected ext4 over ext3. We found ext3
to be very slow in allocating files (or removing them).
Additionally, access within large files is poor with ext3.

18. Deployment Serenity
Shard 1
mongos mongos mongos
mongod mongod mongod
config
Shard 2
mongos mongos mongos
mongod mongod mongod
config
Shard 3
mongos mongos mongos
mongod mongod mongod
config

19. The Proof is in the Pudding

20. Tests Performed
• Small MongoDB Cloud Subscription vs Shared
Virtual Instance
• Medium MongoDB Cloud Subscription vs
Shared Virtual Instance
• SSD and 15K SAS
• Large MongoDB Cloud Subscription vs Shared
Virtual Instance
• SSD and 15K SAS

21. Small Test
Small (SM) Cloud Subscription MongoDB Server
Single 4-core Intel 1270 CPU
64-bit CentOS
8GB RAM
2 x 500GB SATAII – RAID1
1Gb Network
Virtual Provider Instance
4 Virtual Compute Units
64-bit CentOS
7.5GB RAM
2 x 500GB Network Storage – RAID1
1Gb Network

22. Small Test
Tests Performed
Small Data Set (8GB of .5mb documents)
200 iterations of 6:1 query-to-update operations
Concurrent client connections exponentially increased from 1 to 32
Test duration spanned 48 hours

23. Small Test
Average Read Operations per Second by Concurrent Client

24. Small Test
Peak Read Operations per Second by Concurrent Client

25. Small Test
Average Write Operations per Second by Concurrent Client

26. Small Test
Peak Write Operations per Second by Concurrent Client

27. Medium Test
Medium (MD) Cloud Subscription MongoDB Server
Dual 6-core Intel 5670 CPUs
64-bit CentOS
36GB RAM
2 x 64GB SSD – RAID1 (Journal Mount)
4 x 300GB 15K SAS – RAID10 (Data Mount)
1Gb Network – Bonded
Virtual Provider Instance
26 Virtual Compute Units
64-bit CentOS
30GB RAM
2 x 64GB Network Storage – RAID1 (Journal Mount)
4 x 300GB Network Storage – RAID10 (Data Mount)
1Gb Network

28. Medium Test
Tests Performed
Small Data Set (32GB of .5mb documents)
200 iterations of 6:1 query-to-update operations
Concurrent client connections exponentially increased from 1 to 128
Test duration spanned 48 hours

29. Medium Test 15k SAS
Average Read Operations per Second by Concurrent Client

30. Medium Test 15k SAS
Peak Read Operations per Second by Concurrent Client

31. Medium Test 15k SAS
Average Write Operations per Second by Concurrent Client

32. Medium Test 15k SAS
Peak Write Operations per Second by Concurrent Client

33. Medium Test SSD
Average Read Operations per Second by Concurrent Client

34. Medium Test SSD
Peak Read Operations per Second by Concurrent Client

35. Medium Test SSD
Average Write Operations per Second by Concurrent Client

36. Medium Test SSD
Peak Write Operations per Second by Concurrent Client

37. Large Test
Large (LG) Cloud Subscription MongoDB Server
Dual 8-core Intel E5-2620 CPUs
64-bit CentOS
128GB RAM
2 x 64GB SSD – RAID1 (Journal Mount)
6 x 600GB 15K SAS – RAID10 (Data Mount)
1Gb Network – Bonded
Virtual Provider Instance
26 Virtual Compute Units
64-bit CentOS
64GB RAM (Maximum available on this provider)
2 x 64GB Network Storage – RAID1 (Journal Mount)
6 x 600GB Network Storage – RAID10 (Data Mount)
1Gb Network

38. Large Test
Tests Performed
Small Data Set (64GB of .5mb documents)
200 iterations of 6:1 query-to-update operations
Concurrent client connections exponentially increased from 1 to 128
Test duration spanned 48 hours

39. Large Test 15k SAS
Average Read Operations per Second by Concurrent Client

40. Large Test 15k SAS
Peak Read Operations per Second by Concurrent Client

41. Large Test 15k SAS
Average Write Operations per Second by Concurrent Client

42. Large Test 15k SAS
Peak Write Operations per Second by Concurrent Client

43. Large Test SSD
Average Read Operations per Second by Concurrent Client

44. Large Test SSD
Peak Read Operations per Second by Concurrent Client

45. Large Test SSD
Average Write Operations per Second by Concurrent Client

46. Large Test SSD
Peak Write Operations per Second by Concurrent Client

47. RSD
Cloud Provider MongoDB Cloud
Subscription
Small 6-36% 1-9%
Medium 8-43% 1-8%
Large 8-93% 1-9%

49. Avoid Storage I/O
Altogether

50. If you must Virtualize
Go with Local Disks

51. Journal and Data on
Separate Mounts

52. Watch Sync Delay

53. More information:
www.softlayer.com