Managing a MongoDB Deployment
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Managing a MongoDB Deployment

  • 6,319 views
Uploaded on

Talk by Wordnik's Tony Tam on managing a large scale MongoDB deployment

Talk by Wordnik's Tony Tam on managing a large scale MongoDB deployment

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
6,319
On Slideshare
6,316
From Embeds
3
Number of Embeds
2

Actions

Shares
Downloads
61
Comments
0
Likes
7

Embeds 3

https://www.linkedin.com 2
http://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • IO issues8 EC2 servers => 1 physical
  • Don’t use RS, version 1.0 of feature, plus no supportfor authentication
  • Slaves are smarter
  • Tweets, may want hot (shard on id)Random access, need equi depthRedistributing shards is tricky

Transcript

  • 1. Managing a MongoDB deployment
    Tony Tam @fehguy
  • 2. Areas of discussion
    Many drivers to go non-relational
    Maybe you already are?
    SOA?
    “Yes we should!” Now what?
    Implementation details, best practices
    Depends on what system you choose
    Operational Architecture
    Making your deployment “maintainable”
    Maximizing performance
    Software + Hardware tuning
  • 3. Wordnik Says Yes to MongoDB
    And we’re alive to talk about it
    Our stats
    14 M REST API calls yesterday
    7 ms avg. response
    3 TB of MongoDB data
    Mostly all “un-cached” responses
  • 4. Software Architecture
    Java/Scala/Jetty application servers
    Share (almost) nothing
    All roads in must speak REST
  • 5. Deployment architecture
    Wordnik uses physical servers
  • 6. Operational Constraints
    Fundamental question to answer
    Can we support this?
    You need to consider
    If you want it to scale
    If you have a SLA to support
    If you carry the pager
    Also important if
    You have customers
    You like your job
  • 7. Operational Constraints
    Core items for Wordnik
    Redundancy with master/slave
    Multi-datacenter deployment possible?
    Design for read-only mode
    Control at the driver level
    Blocking
    When is it OK?
    Steps around with MongoDB
    Consistency
    When does it matter?
    Scale-out
    What limits our speed/size?
    Other?
    Some features => opportunity
  • 8. Redundancy
    MongoDB supports MS and Replica Sets
    RS is like MS with auto-failover
    Driver intelligence
    Still one master, elected by peers
    Master
    Slave
    Master
    Master
    Slave
    Slave
    Slave
    Slave
    Slave
    Slave
    Slave
  • 9. Redundancy
    Master/Slave fine for us but…
    Not fine for multi operational datacenters
    resync?
    Your data drives the appropriate solution
    Custom tools to support this
    https://github.com/wordnik/wordnik-oss
    Primary Datacenter
    Hot Datacenter
    Incremental Backup Files
    Master
    Master
    Replay Util
    SCP
  • 10. Redundancy
    Multi-datacenter deployment
    Some data need not be consistent
    Some does
    Replication tricks for master-master
    db.documents
    documents.src == 1
    Master 1
    Master 2
    db.documents
    documents.src == 2
  • 11. Controlling blocking operations
    Important to know what blocks
    Some situations can create global locks
    Know what happens with:
    Adding data
    Deleting data
    Modifying data
    …Then toss in 100 reads/sec
    Only access your DB “the right way”
    Enforce by good software design
    The right way == “right way for your db”
  • 12. Controlling blocking operations
    Wordnik Solution
    MongoDB access via service layer
    Only allow queries you want run
    All others => non-production server, Hadoop
    Smart driver manager
    Slave bias
    $inc:{page_views:1} => $set{page_views:10038}
    Read-only mode
    Hot-remove server from pool
  • 13. Handling eventual consistency
    EC makes things fast
    May create timing issues
    Sometimes need current data
    val db:DB = getReadWriteConnection()
    Avoid this a much as possible
    But if you must…
    Write then call “getLastError”
    Forces commit to DB
    RS allows “replicate to n nodes”
    Forces commit to master, n slaves
  • 14. Scale-out
    What are you scaling?
    Size of data
    Size of indexes
    Operations
    Add slave servers
    Won’t help if size > index/server capacity
    Sharding if done right
    Autoshard solution
    Roll your own
  • 15. Scale-out
    Selecting a shard key
    Can be tricky
    Depends on what you need
  • 16. Keeping it running
    Backups
    Export with “mongodump”
    No incremental backup solution included
    Wordnik tools
    https://github.com/wordnik/wordnik-oss
  • 17. Wordnik OSS tools
    Take our tools-They work!!!
    SnapshotUtil
    Selectively snapshot in BSON
    Index info too!
    IncrementalBackupUtil
    Tail the oplog, stream to disk
    Only the collections you want!
    Compress & rotate
    RestoreUtil
    Recover your snapshots
    Apply indexes yourself
    ReplayUtil
    Apply your Incremental backups
  • 18. What if Scenarios
    One collection gets corrupt?
    Restore it
    Apply all operations to it
    “My top developer dropped a collection!”
    Restore just that one
    Apply operations to it until that POT
    “We got hacked!”
    Restore it all
    Apply operations until that POT
  • 19. Wrap up: We like MongoDB
    See more @ http://developer.wordnik.com
    Questions?