Mongo Admin’ing
             Jesse Wolgamott
twitter: @jwo slides: http://github.com/jwo
Quick Install

• Dev: brew install mongodb
• Server: apt-get install mongodb
  • http://www.mongodb.org/display/
    DOCS/Ubuntu+and+Debian+packages
Don’t Fear the
Command Line
   Mongoid / Mongo Mapper


         Ruby Driver


Command Line Javascript (mongo)
Quick Server Status

db
db.system.namespaces.find()
db.serverStatus()
db.currentOp()
Command Line Basics
use dba
db.collection.find({json params})
db.collection.findOne({json params})
db.collection.mapReduce(map,reduce)
db.collection.save({json})
Why Bother?
• Mongo queries enhance Mongoid queries
• Drop to command line when needed
• Game.where(:outcome=>{‘$in’=>[‘winning’
  ,‘tie’]})
• Game.collection.update({:outcome=>‘win
  ning’, ‘$set’=>{:winning=>true}})
All Admin is
     Command Line

• Setup Replica Sets
• Check on Status
Replica / Slave /
       Journaling

“If you aren’t running a replica set, you don’t
            care about your data”
                            -- Mongo DB dude
Master / Slave
  What You’d Expect


           M


  Slave   Slave   Slave
Master / Slave
  What You’d Expect


           M              Writes Go Here



  Slave   Slave   Slave
Master / Slave
  What You’d Expect


            M              Writes Go Here



  Slave    Slave   Slave


          Reads CAN Go
              Here
Replica Sets
    The new Hotness (and Recommended)


Primary   Secondary   Secondary   Secondary



                                   Arbiter
Replica Sets
       The new Hotness (and Recommended)


   Primary   Secondary   Secondary   Secondary



Writes Go Here                        Arbiter
Replica Sets
       The new Hotness (and Recommended)


   Primary   Secondary   Secondary   Secondary



Writes Go Here      CAN Read          Arbiter
Replica Sets
       The new Hotness (and Recommended)


   Primary   Secondary   Secondary   Secondary



Writes Go Here      CAN Read          Arbiter


                               Nothing Goes
                                  Here
Arbiter?

• Arbiters exist when even number of hosts
• They cast votes on primary elections
• Can exist on your app server
Journaling

• Added in 1.8
• mongod -- journal
• Will write out changes to a journal
  directory
• You can restore from journal
Replica Set Fun
config = {
   _id: 'houstonrb',
   members: [
     {_id: 0, host: 'localhost:20000'},
     {_id: 1, host: 'localhost:20001'},
     {_id: 2, host: 'localhost:20002', arbiterOnly: true}]
}

// Start it up
rs.initiate(config);

// Check it Out
rs.status()

// Who’s your daddy?
rs.isMaster()
Non-Master Reads
error: { "$err" : "not master", "code" : 10107 }

               OK, so we’ll do:

         db.getMongo().setSlaveOk()

                 // Awesome
Backup / Restore
mongodump --db houstonrb

mongorestore --db houstonrb -c
collections --drop --indexesLast
DirectoryName

* Have all your .bson files in a
directory named your database
Repairing

• The Data Gobbler -- mongodb likes hard-
  drive space and won’t give it back

• Use mongod    --repair to reclaim your
  space!
Nightly Plan
service mongodb stop && mongod --repair --dbpath /var/
lib/mongodb && service mongodb start

rm -f /data/mongodump-backup.tar.bz2 && mongodump -
o /data/backups && tar -jcf /data/mongodump-
backup.tar.bz2 /data/backups && rm -rf /data/backups/
Catch Up!
• When a Replica Set gets ‘stale’
• rs.status will remain “5” (Recovering)
• On Master, will say “Error RS102 too stale
  to catch up”
• stop mongo on all servers, copy over all
  data files, including local.*, start all
Table Scans == Devil
• When you get to 500,000 records, your
  DB may slow to crawl.
• /var/log/mongodb/mongodb.log for logged
  queries (table scans)

• db.collection.getIndexKeys()
• Mongoid: rake
  db:mongoid:create_indexes
No, Really

• I’ve been surprised LOTS at how often a
  table scan happens.
• Test it out: mongod --notablescan
• Stuff will break.
Sharding

• Don’t Shard Early
• Helps in map-reduce
• You define a “shard-key”, like lastname
  from A-H on this shard
Scaling (Secret Sauce)

• Many Shard Masters, even more slaves
• Add Shards for write and data size scaling
• Add Slaves for read scaling (map-reduce)
  and redundancy
• Sharding is Hard
Neat Snippets

            Replication Setup
             Sharding Setup

https://github.com/mongodb/mongo-snippets
EC2 Notes

• Reformat drive to EXT4 or XFS
• Don’t use EBS for long term backups
• Can use EBS for setting up replica sets

Mongo admin'ing

  • 1.
    Mongo Admin’ing Jesse Wolgamott twitter: @jwo slides: http://github.com/jwo
  • 2.
    Quick Install • Dev:brew install mongodb • Server: apt-get install mongodb • http://www.mongodb.org/display/ DOCS/Ubuntu+and+Debian+packages
  • 3.
    Don’t Fear the CommandLine Mongoid / Mongo Mapper Ruby Driver Command Line Javascript (mongo)
  • 4.
  • 5.
    Command Line Basics usedba db.collection.find({json params}) db.collection.findOne({json params}) db.collection.mapReduce(map,reduce) db.collection.save({json})
  • 6.
    Why Bother? • Mongoqueries enhance Mongoid queries • Drop to command line when needed • Game.where(:outcome=>{‘$in’=>[‘winning’ ,‘tie’]}) • Game.collection.update({:outcome=>‘win ning’, ‘$set’=>{:winning=>true}})
  • 7.
    All Admin is Command Line • Setup Replica Sets • Check on Status
  • 8.
    Replica / Slave/ Journaling “If you aren’t running a replica set, you don’t care about your data” -- Mongo DB dude
  • 9.
    Master / Slave What You’d Expect M Slave Slave Slave
  • 10.
    Master / Slave What You’d Expect M Writes Go Here Slave Slave Slave
  • 11.
    Master / Slave What You’d Expect M Writes Go Here Slave Slave Slave Reads CAN Go Here
  • 12.
    Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Arbiter
  • 13.
    Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Writes Go Here Arbiter
  • 14.
    Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Writes Go Here CAN Read Arbiter
  • 15.
    Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Writes Go Here CAN Read Arbiter Nothing Goes Here
  • 16.
    Arbiter? • Arbiters existwhen even number of hosts • They cast votes on primary elections • Can exist on your app server
  • 17.
    Journaling • Added in1.8 • mongod -- journal • Will write out changes to a journal directory • You can restore from journal
  • 18.
    Replica Set Fun config= { _id: 'houstonrb', members: [ {_id: 0, host: 'localhost:20000'}, {_id: 1, host: 'localhost:20001'}, {_id: 2, host: 'localhost:20002', arbiterOnly: true}] } // Start it up rs.initiate(config); // Check it Out rs.status() // Who’s your daddy? rs.isMaster()
  • 19.
    Non-Master Reads error: {"$err" : "not master", "code" : 10107 } OK, so we’ll do: db.getMongo().setSlaveOk() // Awesome
  • 20.
    Backup / Restore mongodump--db houstonrb mongorestore --db houstonrb -c collections --drop --indexesLast DirectoryName * Have all your .bson files in a directory named your database
  • 21.
    Repairing • The DataGobbler -- mongodb likes hard- drive space and won’t give it back • Use mongod --repair to reclaim your space!
  • 22.
    Nightly Plan service mongodbstop && mongod --repair --dbpath /var/ lib/mongodb && service mongodb start rm -f /data/mongodump-backup.tar.bz2 && mongodump - o /data/backups && tar -jcf /data/mongodump- backup.tar.bz2 /data/backups && rm -rf /data/backups/
  • 23.
    Catch Up! • Whena Replica Set gets ‘stale’ • rs.status will remain “5” (Recovering) • On Master, will say “Error RS102 too stale to catch up” • stop mongo on all servers, copy over all data files, including local.*, start all
  • 24.
    Table Scans ==Devil • When you get to 500,000 records, your DB may slow to crawl. • /var/log/mongodb/mongodb.log for logged queries (table scans) • db.collection.getIndexKeys() • Mongoid: rake db:mongoid:create_indexes
  • 25.
    No, Really • I’vebeen surprised LOTS at how often a table scan happens. • Test it out: mongod --notablescan • Stuff will break.
  • 26.
    Sharding • Don’t ShardEarly • Helps in map-reduce • You define a “shard-key”, like lastname from A-H on this shard
  • 27.
    Scaling (Secret Sauce) •Many Shard Masters, even more slaves • Add Shards for write and data size scaling • Add Slaves for read scaling (map-reduce) and redundancy • Sharding is Hard
  • 28.
    Neat Snippets Replication Setup Sharding Setup https://github.com/mongodb/mongo-snippets
  • 29.
    EC2 Notes • Reformatdrive to EXT4 or XFS • Don’t use EBS for long term backups • Can use EBS for setting up replica sets