Repl internals

Replication Internals
MongoBerlin 2010-10-4
Mathias Stearn
[email_address]

Replication is Good!
Use it!
mongod --master
mongod --slave --source host:port
30 second demo…

Basic Phases of a Slave
If not empty skip to step 4 (--fastsync)
Note master's position in OpLog
Copy data from master ($snapshot)
Replay OpLog to catch up
Continue watching OpLog for new changes
???
Profit

The OpLog
“ local.oplog.$main” for Master/Slave
Only on Master
“ local.oplog.rs” for ReplicaSets
On all nodes
Uses a capped collection (--oplogSize)
“ Wraps” when full (LRI)
Make sure it's large enough
You can query it like any other collection

OpLog Entries
{ "ts" : { "t" : 1286179802000, "i" : 1 }, "op" : "i", "ns" : "test.foo", "o" : { "_id":1, "a" : 1 } }
Operations:
I : insert
U : update (modified to be idempotent)
D : delete (one per object)
C : command
N : No-op (ignore these)

OpLog demo
Insert and update an object
Look at OpLog
Use local
db.oplog.$main.find({op:{$ne:'n'}})
sort({$natural:-1})
Show data on slave
Add new slave and show data

Replication Optimizations
All of these QueryOptions are used internally
Exaust – Sends all data w/o GetMores
Tailable – Leaves cursor open
AwaitData – Query blocks for data (push)
OplogReplay – Optimizes scan for entries newer than supplied timestamp so we don't need an index

Replica Pairs
Deprecated by ReplSets in 1.6
Don't use!

Replica Sets
Evolution of Replica Pairs
Reflects “lessons learned”
Any number of nodes
All keep a copy of the OpLog
At any time one node is Primay
If Primary fails, remaining nodes elect a new Primary with newest data

Replica Set Gotchas
When Primary fails, short period of time with no Primary
Election requires a majority of nodes up
Works best with 3 failure domains
Writes not commited to quorum can be rolled-back
Use getLastError with “W”
Cascading not support yet (planned)

Questions?
Ask away!
#mongodb on irc.freenode.net
[email_address]
[email_address]
Thanks for coming!

Repl internals

by Dwight Merriman on Oct 12, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

Repl internals — Presentation Transcript