Administration        Michael DelNegro Principal Database Administrator               AOL                1
Presentation Overview• Introduction• My Applications• Tips• Tools• Resources• Upcoming                    2
About Me• DBA at AOL (Dulles) for six years• Background in Sybase• Now MySQL, PostgreSQL and NoSQL• Was: Blogsmith, Uncut ...
About MongoDB• “Scalable, high-performance, open source,  document-oriented database”• Databases (Databases) • Collections...
More About MongoDB• JSON-style Documents• Javascript Shell• Capped Collections• Flexible Schemas• Replication w/ Autofailo...
MongoDB Support• Operating Systems • Linux, Windows, Mac OS X, Solaris • 32bit, 64bit• Drivers • Java, Javascript, Perl, R...
MongoDB Use Cases• Website Data Store• Caching tier• Document and Content Mgmt Systems• Event Logging• Real-time Stats/Ana...
MongoDB Misuse• Complex Transactional Systems• Traditional Business Intelligence• SQL is a Must                      8
MongoDB at AOL• In use since Summer 2010• I currently administer two applications for  MapQuest and Patch• There are other...
MapQuest App• Tracks User Profile Information• V. 1.6.5.latest (just upgraded from 1.6.3)• 26 Total Hosts, CentOS 5, 16GB R...
Patch App• Tracks User Activity• Master, 2 Slaves• V. 1.6.3• About 100GB of data• Throwaway Data (for now)• Ruby on Rails ...
About Patch• “HyperLocal” news sites across the  country• Fills gap in coverage left by local  newspapers• Currently 800 s...
Nearby Patch Sites• Vienna (ex. vienna.patch.com)• Ashburn• Reston• McLean• CollegePark• GreaterAnnapolis• 50+ in DC Area ...
Upcoming Ops Plans• Upgrade to 1.8• Migrate Patch to Replica Sets• Move MapQuest to bigger hardware (16GB  -> 64GB memory)...
Admin Tips• Slaves are a MUST pre1.8• Use 64 bit version • 32 bit version has 2.5 GB storage limit• Use xfs or ext4• Keep ...
More Admin Tips• Increase File Descriptor Limits• Do not use kill -9 (pre-1.8)• Consider having a slave on replication del...
Even More Admin Tips• Omit parenthesis to see command details• 5 Primitives of Mongo • insert, remove, update, find, getMor...
Admin Tools• mongodump-mongorestore • use fsync and lock database to ensure    consistent backup• fsync and lock are a mus...
More Admin Tools• mongostat• db.printReplicationInfo()• db.serverStatus()• db.<collection>.stats()• Database Profiler• Expl...
Admin Resources• mongodb.org • Events • Forums• Wordnik Mongo Admin Tools (Github)• Mongo Snippets (Github)• IRC (freenode...
More Admin Resources• slideshare (Use Time-Based Search)• GUI Admin Tools • MongoVUE • Others• Kristina Chodorows Blog• Bo...
Even More Resources• Follow @MongoQuestion (StackOverflow)• MongoDB on Quora (@q_mongodb)• 10gen Deployment Strategies Slid...
New MongoDB Release• 1.8 (Released March 16) • Single Server Durability (Journaling) • Enhancements to Sharding & Replica ...
Future Releases• 2.0 (May/June?) • Better Map-Reduce and Aggregation • Improved Concurrency • Online Compaction • TTL Time...
Thank You!• www.slideshare.net/radiocats• @radiocats on Twitter• www.linkedin.com/in/mdelnegro                   25
Upcoming SlideShare
Loading in …5
×

Mongo db admin_20110329

1,689 views

Published on

MongoDB Admin Talk Given at AOL on 3/29/2011 - Keynote

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,689
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
71
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Mongo db admin_20110329

    1. 1. Administration Michael DelNegro Principal Database Administrator AOL 1
    2. 2. Presentation Overview• Introduction• My Applications• Tips• Tools• Resources• Upcoming 2
    3. 3. About Me• DBA at AOL (Dulles) for six years• Background in Sybase• Now MySQL, PostgreSQL and NoSQL• Was: Blogsmith, Uncut Video, Travel, Autos, Journals, Real Estate, Ficlets, Shopping• Currently: Patch, MapQuest, HSS, Datalayer, Demand• I Heart Big Data 3
    4. 4. About MongoDB• “Scalable, high-performance, open source, document-oriented database”• Databases (Databases) • Collections (Tables) • Documents (Rows) • Fields (Columns) - K/V Pairs• Indexes• No Joins • Favors Embedding Data instead of FKs 4
    5. 5. More About MongoDB• JSON-style Documents• Javascript Shell• Capped Collections• Flexible Schemas• Replication w/ Autofailover• Sharding• GridFS: File Storage• Map-Reduce 5
    6. 6. MongoDB Support• Operating Systems • Linux, Windows, Mac OS X, Solaris • 32bit, 64bit• Drivers • Java, Javascript, Perl, Ruby, Scala, Erlang, C, C#, C++, Haskell, PHP, Python • R, Smalltalk, node.js, ColdFusion 6
    7. 7. MongoDB Use Cases• Website Data Store• Caching tier• Document and Content Mgmt Systems• Event Logging• Real-time Stats/Analytics• Archiving• High Volume Problems 7
    8. 8. MongoDB Misuse• Complex Transactional Systems• Traditional Business Intelligence• SQL is a Must 8
    9. 9. MongoDB at AOL• In use since Summer 2010• I currently administer two applications for MapQuest and Patch• There are other MongoDB applications in use throughout the company and more on the way 9
    10. 10. MapQuest App• Tracks User Profile Information• V. 1.6.5.latest (just upgraded from 1.6.3)• 26 Total Hosts, CentOS 5, 16GB RAM• 300 million users, 130 million saved maps• Replica Sets• 3 Sharded Collections • lookup, east coast, west coast• Java 10
    11. 11. Patch App• Tracks User Activity• Master, 2 Slaves• V. 1.6.3• About 100GB of data• Throwaway Data (for now)• Ruby on Rails 11
    12. 12. About Patch• “HyperLocal” news sites across the country• Fills gap in coverage left by local newspapers• Currently 800 sites are live• 1000+ by end of 2011 12
    13. 13. Nearby Patch Sites• Vienna (ex. vienna.patch.com)• Ashburn• Reston• McLean• CollegePark• GreaterAnnapolis• 50+ in DC Area 13
    14. 14. Upcoming Ops Plans• Upgrade to 1.8• Migrate Patch to Replica Sets• Move MapQuest to bigger hardware (16GB -> 64GB memory)• Add additional slaves 14
    15. 15. Admin Tips• Slaves are a MUST pre1.8• Use 64 bit version • 32 bit version has 2.5 GB storage limit• Use xfs or ext4• Keep eye on oplog size• Turn off atime & dtime• Consider using getLastError() 15
    16. 16. More Admin Tips• Increase File Descriptor Limits• Do not use kill -9 (pre-1.8)• Consider having a slave on replication delay • -- slavedelay <seconds>• db.runCommand(“logRotate”)• Keep db.<collection>.totalIndexSize() less than RAM• Linux dirty_background_ratio and dirty_ratio (pre 2.6.22) 16
    17. 17. Even More Admin Tips• Omit parenthesis to see command details• 5 Primitives of Mongo • insert, remove, update, find, getMore• Replication is slave polling master process• Master and slaves each have their own oplog• Choose shard key carefully (ex. timestamp) 17
    18. 18. Admin Tools• mongodump-mongorestore • use fsync and lock database to ensure consistent backup• fsync and lock are a must for file system backups (ex LVM)• http://localhost:28017 (server port + 1000)• db.currentOp() 18
    19. 19. More Admin Tools• mongostat• db.printReplicationInfo()• db.serverStatus()• db.<collection>.stats()• Database Profiler• Explain 19
    20. 20. Admin Resources• mongodb.org • Events • Forums• Wordnik Mongo Admin Tools (Github)• Mongo Snippets (Github)• IRC (freenode #mongodb)• Little MongoDB Book 20
    21. 21. More Admin Resources• slideshare (Use Time-Based Search)• GUI Admin Tools • MongoVUE • Others• Kristina Chodorows Blog• Boxed Ice 21
    22. 22. Even More Resources• Follow @MongoQuestion (StackOverflow)• MongoDB on Quora (@q_mongodb)• 10gen Deployment Strategies Slides• Books• Training• 10gen Support• Office Hours in NYC and Redwood City 22
    23. 23. New MongoDB Release• 1.8 (Released March 16) • Single Server Durability (Journaling) • Enhancements to Sharding & Replica Sets • Covered and Sparse Indexes • Tab Completion • Maximum BSON Document: 16MB • 1.8 Features Presentation 23
    24. 24. Future Releases• 2.0 (May/June?) • Better Map-Reduce and Aggregation • Improved Concurrency • Online Compaction • TTL Time-Out Collections• Beyond • Full-Text Search? 24
    25. 25. Thank You!• www.slideshare.net/radiocats• @radiocats on Twitter• www.linkedin.com/in/mdelnegro 25

    ×