• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
MongoPhilly - Deployment Notes
 

MongoPhilly - Deployment Notes

on

  • 1,437 views

 

Statistics

Views

Total Views
1,437
Views on SlideShare
1,437
Embed Views
0

Actions

Likes
1
Downloads
31
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    MongoPhilly - Deployment Notes MongoPhilly - Deployment Notes Presentation Transcript

    • MongoDB Deployment Notes Kyle Banker kyle@10gen.comWednesday, April 27, 2011
    • In this talk: • Refresher: MongoDB versioning, supported OSs • Sizing: Memory, CPU, I/O • Security • Backups • Durability strategies • EC2 notesWednesday, April 27, 2011
    • MongoDB Version Policy • Production: even minor version numbers. • 1.4.x, 1.6.x, 1.8.x •Development: odd minor version numbers. •1.5.x, 1.7.x • Critical bugs are back-ported to the latest production version. Try to run the latest stable version.Wednesday, April 27, 2011
    • Operating Systems • For production: Use a 64-bit OS • 32-bit OSs have 2G limit on mapped data. • However: clients can be 32 bit. • MongoDB supports: • Linux, OS X • Windows • Solaris • Server must run on little-endian.Wednesday, April 27, 2011
    • Memory • Keep your working set in memory. • Memory-mapped files • Maps Files on Filesystem to Virtual Memory • Not Physical RAM • Page Faults - not in memory - from disk - expensive • Indices • Part of the regular DB files • Large databases may need to warm their caches. • Might be a reason to keep fast disks.Wednesday, April 27, 2011
    • I/O • Disk I/O determines: • Performance of non-working set queries. • Speed of index builds. • Time to warm cache. • Disk configurations: • Consider using fast disks; consumer 5400 RPM drives aren’t always enough. • Raid 0 - Striping: improved write performance • Raid 1 - Mirroring: survive single disk failure • Raid 10 - both • Consider solid state drives: • Expensive, getting cheaper • Significantly reduced seek time, increased IO throughputWednesday, April 27, 2011
    • CPU • MongoDB uses multiple cores. • For working-set queries, CPU usage is minimal. • CPU-intensive processes: • Index construction. • Certain kinds of aggregation. • Heavy update loads. • JavaScript ops (map/reduce, group) are single-threaded. • Look for new aggregation framework in 2.0.Wednesday, April 27, 2011
    • Security • Mongo supports basic security • Authenticate aser on a per Database basis • Start database with --auth • Admin user stored in the admin database use admin db.addUser("administrator", "password") db.auth(“administrator”, “password”) •Regular users stored in other databases use personnel db.addUser("joe", "password") db.addUser(“fred”, “password”, true)Wednesday, April 27, 2011
    • Replica Set Authentication • Each member gets a “key file” containing a password. • Start each member with --keyfile, pointing to the file.Wednesday, April 27, 2011
    • Run in a secure environment. • A properly-configured firewall is essential. • Take all the precautions you’d take running any server.Wednesday, April 27, 2011
    • Backups • Backups are often taken from a secondary node. • Eliminates impact to primary node and client.Wednesday, April 27, 2011
    • Backups • mongodump / mongorestore • Datafile backup or snapshot.Wednesday, April 27, 2011
    • mongodump and mongorestore • Binary, compact object dump • Each consistent object is written • Not necessarily consistent from start to finish • unless you lock database: • db.runCommand({fsync:1,lock:1}) • Use mongorestore to restore database • database does not have to be online to be restoredWednesday, April 27, 2011
    • Filesystem Backup • First make sure that the datafiles are synced and locked: • fsync - Flushes buffers to disk. • lock - Blocks writes. db.runCommand({fsync:1,lock:1}) • Use file-system / LVM / storage snapshot. • To unlock: db.$cmd.sys.unlock.findOne();Wednesday, April 27, 2011
    • Write Concern What failures do you need to recover from? • Loss of a single database node? • Loss of a group of nodes?Wednesday, April 27, 2011
    • Write Concern: Primary only • Write acknowledged when in memory on primary onlyWednesday, April 27, 2011
    • Write Concern: - Primary and Secondaries • W=2 • Write acknowledged when in memory on master + slave • Will survive failure of a single nodeWednesday, April 27, 2011
    • Write Concern: Primary and Secondaries with Fsync • W=n • Pick a “majority” of nodes • Use the “fsync” option only when --journal is enabled. • fsync in batches, not on every write (since it is blocking) • Waits until the 100ms sync window passesWednesday, April 27, 2011
    • Replica delay • Protection against app faults • Protection against administration mistakes • Replica runs n seconds behind the primary • Delayed replicas must be priority 0 (cannot fail over)Wednesday, April 27, 2011
    • Jounaling (1.8+) • Run with --journal • Group commits (every 100ms) • Fast recovery from unclean shutdown • Good option: run 1 slave with --journalWednesday, April 27, 2011
    • EC2 Notes • Default filesystem is ext3 • For best performance, choose ext4 or xfs • Use RAID (using MDADM or LVM) •EBS volumes are limited by network. • EBS can experience spikes in latency • 400ms-600msWednesday, April 27, 2011
    • EC2 Notes • EBS snapshots can be used for backups • EBS can disappear • S3 can be used for longer term backups • Use Amazon availability zones • High Availability • Disaster RecoveryWednesday, April 27, 2011
    • Questions?Wednesday, April 27, 2011