Virtualizing MongoDB: Cloud,
EC2, GCE or Dedicated?
David Mytton
MongoDB World - July 2014
blog.serverdensity.com
David Mytton
Server monitoring, cloud management,
dashboards and alerting
serverdensity.com
Server Density Architecture
Server Density Architecture
● ~100 servers - Ubuntu 12.04
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
● Nginx, Python...
Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
● Nginx, Python...
Two choices for deployment
Two choices for deployment
● Virtualized
● Bare metal
Advantages of virtualization
● Easy to manage
Advantages of virtualization
● Easy to manage
● Fast boot
Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
● Templating/snapshots
Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
● Templating/snapshots
● Containment
Disadvantages of virtualization
● Another layer
Disadvantages of virtualization
● Another layer
● Hypervisor overhead
Disadvantages of virtualization
● Another layer
● Hypervisor overhead
● Host contention
Disadvantages of virtualization
● Another layer
● Hypervisor overhead
● Host contention
● i/o performance
Advantages of bare metal
● Dedicated resources
Advantages of bare metal
● Dedicated resources
● Direct access to hardware
Advantages of bare metal
● Dedicated resources
● Direct access to hardware
● Customisable specs
Advantages of bare metal
● Dedicated resources
● Direct access to hardware
● Customisable specs
● Performance
Disadvantages of bare metal
● Build/deploy time
Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
● Difficult to migrate/snapshot
Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
● Capex/lifetime
● Difficult to migrate/snapshot
Why cloud?
● Flexible
Why cloud?
● Flexible
● Unlimited resources
Why cloud?
● Flexible
● Unlimited resources
● Cheap to get started
Why cloud?
● Flexible
● Unlimited resources
● Cheap to get started
● Other products
Why colo?
Why colo?
● Vastly cheaper
● Complete control
Performance problems?
Performance problems?
Easy answer: move to bare metal!
Optimising virtualization
● Dedicated CPUs
Optimising virtualization
● Dedicated CPUs
● High speed networking
Optimising virtualization
● Dedicated CPUs
● High speed networking
● Disk i/o (MB/s & IOPS)
Optimising EC2
Optimising EC2
● Newer, 64bit instance types
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
● L...
Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
● L...
Optimising EC2
● PIOPS EBS volumes
Optimising EC2
● PIOPS EBS volumes
● RAID for redundancy & performance
Optimising EC2
● PIOPS EBS volumes
● RAID for redundancy & performance
● blockdev --setra 32
Common EC2 Problems
Common EC2 Problems
● Networking
Common EC2 Problems
● Networking
● Older hardware
Common EC2 Problems
● Networking
● Older hardware
● Complex pricing (PIOPS)
Why EC2?
Why EC2?
● Most popular
Why EC2?
● Most popular
● Range of instance types
Why EC2?
● Most popular
● Range of instance types
● Many regions
Optimising GCE
● High memory instances
Optimising GCE
● High memory instances
● Volume size = IOPS available
Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
● Single volume or wasted space
Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
● Single volume or wasted spac...
Common GCE problems
● Volume too small
Common GCE problems
● Volume too small
● RAIDing unnecessary
Common GCE problems
● Volume too small
● RAIDing unnecessary
● Hitting limits?
Why GCE?
● Best network performance
Why GCE?
● Best network performance
● Easy to size disks / pricing
Why GCE?
● Best network performance
● Easy to size disks / pricing
● Global snapshots
Why GCE?
● Best network performance
● Easy to size disks / pricing
● Global snapshots
● Live migrate
Networking performance
AWS
GCE
bit.ly/googlevsamazon
Tweak: separate disks
Tweak: separate disks
● Avoids i/o contention
Tweak: separate disks
● Optimise IOPS/size/price for
journal, dbpath, logs
● Avoids i/o contention
Tweak: separate disks
● Optimise IOPS/size/price for
journal, dbpath, logs
● --directoryperdb - even
separate out each dat...
--directoryperdb warning
--directoryperdb warning
● Snapshotting more complex
--directoryperdb warning
● Snapshotting more complex
● Requires fsync lock +
snapshot all volumes
Summary
● VMs for flexibility
Summary
● VMs for flexibility
● … but need tweaks
Summary
● VMs for flexibility
● Bare metal/colo for price/performance
● … but need tweaks
Summary
● VMs for flexibility
● Bare metal/colo for price/performance
● … but ops overhead
● … but need tweaks
どもありがとうございます
@davidmytton
david@serverdensity.com
blog.serverdensity.com
Upcoming SlideShare
Loading in …5
×

Virtualizing MongoDB: Cloud, EC2, OpenStack, VMs...or Dedicated?

2,359 views
2,290 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,359
On SlideShare
0
From Embeds
0
Number of Embeds
1,434
Actions
Shares
0
Downloads
49
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Virtualizing MongoDB: Cloud, EC2, OpenStack, VMs...or Dedicated?

  1. 1. Virtualizing MongoDB: Cloud, EC2, GCE or Dedicated? David Mytton MongoDB World - July 2014 blog.serverdensity.com
  2. 2. David Mytton
  3. 3. Server monitoring, cloud management, dashboards and alerting serverdensity.com
  4. 4. Server Density Architecture
  5. 5. Server Density Architecture ● ~100 servers - Ubuntu 12.04
  6. 6. Server Density Architecture ● ~100 servers - Ubuntu 12.04 ● 50:50 virtual/dedicated
  7. 7. Server Density Architecture ● ~100 servers - Ubuntu 12.04 ● 50:50 virtual/dedicated ● 30TB/m incoming data
  8. 8. Server Density Architecture ● ~100 servers - Ubuntu 12.04 ● 50:50 virtual/dedicated ● 30TB/m incoming data ● Nginx, Python, MongoDB
  9. 9. Server Density Architecture ● ~100 servers - Ubuntu 12.04 ● 50:50 virtual/dedicated ● 30TB/m incoming data ● Nginx, Python, MongoDB ● Softlayer > Google Cloud
  10. 10. Two choices for deployment
  11. 11. Two choices for deployment ● Virtualized ● Bare metal
  12. 12. Advantages of virtualization ● Easy to manage
  13. 13. Advantages of virtualization ● Easy to manage ● Fast boot
  14. 14. Advantages of virtualization ● Easy to manage ● Fast boot ● Easier to resize/migrate
  15. 15. Advantages of virtualization ● Easy to manage ● Fast boot ● Easier to resize/migrate ● Templating/snapshots
  16. 16. Advantages of virtualization ● Easy to manage ● Fast boot ● Easier to resize/migrate ● Templating/snapshots ● Containment
  17. 17. Disadvantages of virtualization ● Another layer
  18. 18. Disadvantages of virtualization ● Another layer ● Hypervisor overhead
  19. 19. Disadvantages of virtualization ● Another layer ● Hypervisor overhead ● Host contention
  20. 20. Disadvantages of virtualization ● Another layer ● Hypervisor overhead ● Host contention ● i/o performance
  21. 21. Advantages of bare metal ● Dedicated resources
  22. 22. Advantages of bare metal ● Dedicated resources ● Direct access to hardware
  23. 23. Advantages of bare metal ● Dedicated resources ● Direct access to hardware ● Customisable specs
  24. 24. Advantages of bare metal ● Dedicated resources ● Direct access to hardware ● Customisable specs ● Performance
  25. 25. Disadvantages of bare metal ● Build/deploy time
  26. 26. Disadvantages of bare metal ● Build/deploy time ● More difficult to resize
  27. 27. Disadvantages of bare metal ● Build/deploy time ● More difficult to resize ● Difficult to migrate/snapshot
  28. 28. Disadvantages of bare metal ● Build/deploy time ● More difficult to resize ● Capex/lifetime ● Difficult to migrate/snapshot
  29. 29. Why cloud? ● Flexible
  30. 30. Why cloud? ● Flexible ● Unlimited resources
  31. 31. Why cloud? ● Flexible ● Unlimited resources ● Cheap to get started
  32. 32. Why cloud? ● Flexible ● Unlimited resources ● Cheap to get started ● Other products
  33. 33. Why colo?
  34. 34. Why colo? ● Vastly cheaper ● Complete control
  35. 35. Performance problems?
  36. 36. Performance problems? Easy answer: move to bare metal!
  37. 37. Optimising virtualization ● Dedicated CPUs
  38. 38. Optimising virtualization ● Dedicated CPUs ● High speed networking
  39. 39. Optimising virtualization ● Dedicated CPUs ● High speed networking ● Disk i/o (MB/s & IOPS)
  40. 40. Optimising EC2
  41. 41. Optimising EC2 ● Newer, 64bit instance types
  42. 42. Optimising EC2 ● Newer, 64bit instance types ● Use network / EBS optimised instances
  43. 43. Optimising EC2 ● Newer, 64bit instance types ● Use network / EBS optimised instances ● High memory (CPU not important)
  44. 44. Optimising EC2 ● Newer, 64bit instance types ● Use network / EBS optimised instances ● High memory (CPU not important) ● Local storage ephemeral
  45. 45. Optimising EC2 ● Newer, 64bit instance types ● Use network / EBS optimised instances ● High memory (CPU not important) ● Local storage ephemeral ● Volumes for log, journal and data
  46. 46. Optimising EC2 ● PIOPS EBS volumes
  47. 47. Optimising EC2 ● PIOPS EBS volumes ● RAID for redundancy & performance
  48. 48. Optimising EC2 ● PIOPS EBS volumes ● RAID for redundancy & performance ● blockdev --setra 32
  49. 49. Common EC2 Problems
  50. 50. Common EC2 Problems ● Networking
  51. 51. Common EC2 Problems ● Networking ● Older hardware
  52. 52. Common EC2 Problems ● Networking ● Older hardware ● Complex pricing (PIOPS)
  53. 53. Why EC2?
  54. 54. Why EC2? ● Most popular
  55. 55. Why EC2? ● Most popular ● Range of instance types
  56. 56. Why EC2? ● Most popular ● Range of instance types ● Many regions
  57. 57. Optimising GCE ● High memory instances
  58. 58. Optimising GCE ● High memory instances ● Volume size = IOPS available
  59. 59. Optimising GCE ● High memory instances ● Volume size = IOPS available ● VMs have limits too
  60. 60. Optimising GCE ● High memory instances ● Volume size = IOPS available ● VMs have limits too ● Single volume or wasted space
  61. 61. Optimising GCE ● High memory instances ● Volume size = IOPS available ● VMs have limits too ● Single volume or wasted space ● Network / RAID already handled
  62. 62. Common GCE problems ● Volume too small
  63. 63. Common GCE problems ● Volume too small ● RAIDing unnecessary
  64. 64. Common GCE problems ● Volume too small ● RAIDing unnecessary ● Hitting limits?
  65. 65. Why GCE? ● Best network performance
  66. 66. Why GCE? ● Best network performance ● Easy to size disks / pricing
  67. 67. Why GCE? ● Best network performance ● Easy to size disks / pricing ● Global snapshots
  68. 68. Why GCE? ● Best network performance ● Easy to size disks / pricing ● Global snapshots ● Live migrate
  69. 69. Networking performance AWS GCE bit.ly/googlevsamazon
  70. 70. Tweak: separate disks
  71. 71. Tweak: separate disks ● Avoids i/o contention
  72. 72. Tweak: separate disks ● Optimise IOPS/size/price for journal, dbpath, logs ● Avoids i/o contention
  73. 73. Tweak: separate disks ● Optimise IOPS/size/price for journal, dbpath, logs ● --directoryperdb - even separate out each database ● Avoids i/o contention
  74. 74. --directoryperdb warning
  75. 75. --directoryperdb warning ● Snapshotting more complex
  76. 76. --directoryperdb warning ● Snapshotting more complex ● Requires fsync lock + snapshot all volumes
  77. 77. Summary ● VMs for flexibility
  78. 78. Summary ● VMs for flexibility ● … but need tweaks
  79. 79. Summary ● VMs for flexibility ● Bare metal/colo for price/performance ● … but need tweaks
  80. 80. Summary ● VMs for flexibility ● Bare metal/colo for price/performance ● … but ops overhead ● … but need tweaks
  81. 81. どもありがとうございます @davidmytton david@serverdensity.com blog.serverdensity.com

×