Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MongoDB Drivers
&	

High Availability
A. Jesse Jiryu Davis
Staff Engineer, MongoDB
@jessejiryudavis
Client

Side
• It’s a MongoDB driver specification.
• (It’s not MMS.)
• Written by me.

Advisors: David Golden, Craig Wilson, Jeff Yemin....
Good News
&	

Bad News
Agenda
•Startup
•Steady State
•Crisis
•Resolution
•Practical Applications
•Further Study
Setup
•3-node replica set
•PyMongo 3
client = MongoClient(!
'mongodb://host1,host2/?replicaSet=my-rs')!
primary
host1
secondary
host2
secondary
host3
Python Program
monitor

thread
monitor

thread
MongoClient


{secondary: tru...
ismaster

(from secondary)
{"setName": "rs-name",!
"ismaster": false,!
"secondary": true,!
"hosts": ["host1", "host2", "ho...
{"setName": "rs-name",!
"ismaster": true,!
"secondary": false,!
"hosts": ["host1", "host2", "host3"]}!
ismaster

(from pri...
Updating Topology
Description
Current description
+
ismaster response
→ new description
Updating Topology
Description
primary
host1
secondary
host2
secondary
host3
Python Program
monitor

thread
monitor

thread
monitor

thread
MongoClient

...
client.db.collection.insert({'my': 'document'})!
client = MongoClient(!
'mongodb://host1,host2/?replicaSet=my-rs')!
primary
host1
secondary
host2
secondary
host3
Python Program
monitor

thread
monitor

thread
monitor

thread
MongoClient

...
Steady State
primary
host1
secondary
host2
secondary
host3
monitor

thread
monitor

thread
monitor

thread
MongoClient
Your

Code
inser...
Crisis
primary
host1
secondary
host2
secondary
host3
monitor

thread
monitor

thread
monitor

thread
MongoClient
Your

Code
inser...
primary
host1
secondary
host2
secondary
host3
monitor

thread
monitor

thread
monitor

thread
Your

Code
insert
Connection...
primary
host1
secondary
host2
secondary
host3
monitor

thread
monitor

thread
monitor

thread
Your

Code
insert
✓ ✓
MongoC...
Resolution
primary
host1
secondary
host2
secondary
host3
monitor

thread
monitor

thread
monitor

thread
Your

Code
✓
insert
"Wake up...
So What?
client = MongoClient(!
'mongodb://host1,host2/?replicaSet=my-rs')!
!
try:!
client.admin.command('ismaster')!
except pymong...
Error Handling
#2
def find_my_document():!
try:!
return collection.find_one(query)!
except ConnectionFailure:!
logging.exc...
Error Handling
#2
def insert_doc():!
doc = {'_id': ObjectId(), 'a': 1}!
try:!
collection.insert(doc)!
except ConnectionFai...
#2
Error Handling
collection.update(!
{'_id': 1},!
{'$inc': {'counter': 1}})!
#2
Only idempotent
operations can

be retried
Error Handling
Knobs
#3
•connectTimeoutMS
•serverSelectionTimeoutMS
ConnectionFailure!
monitor

thread
monitor

thread
monitor

thread
primary
host1
secondary
host2
secondary
host3
Your

Cod...
bit.ly/server-discovery
Further Study:


Server Discovery
&	

Monitoring Spec
Upcoming SlideShare
Loading in …5
×

MongoDB Drivers And High Availability: Deep Dive

MongoDB has separate drivers for ten programming languages, and all are responsible for connecting to replica sets or groups of mongos. What algorithms do they use to find servers, connect to them, discover their state, choose which ones to use, and survive failovers? A. Jesse Jiryu Davis, a Staff Engineer on the MongoDB Drivers Team, is the author of the Server Discovery And Monitoring Spec. He will explain the inner workings of MongoDB drivers and their behavior within complex server topologies.

  • Login to see the comments

MongoDB Drivers And High Availability: Deep Dive

  1. 1. MongoDB Drivers & High Availability A. Jesse Jiryu Davis Staff Engineer, MongoDB @jessejiryudavis
  2. 2. Client
 Side
  3. 3. • It’s a MongoDB driver specification. • (It’s not MMS.) • Written by me.
 Advisors: David Golden, Craig Wilson, Jeff Yemin.
 Overseen by Bernie Hackett. Server Discovery & Monitoring Spec
  4. 4. Good News & Bad News
  5. 5. Agenda •Startup •Steady State •Crisis •Resolution •Practical Applications •Further Study
  6. 6. Setup •3-node replica set •PyMongo 3
  7. 7. client = MongoClient(! 'mongodb://host1,host2/?replicaSet=my-rs')!
  8. 8. primary host1 secondary host2 secondary host3 Python Program monitor
 thread monitor
 thread MongoClient 
 {secondary: true,
 hosts: ["host1", "host2", "host3"]}
 "mongodb://host1,host2/?replicaSet=my-rs"
  9. 9. ismaster
 (from secondary) {"setName": "rs-name",! "ismaster": false,! "secondary": true,! "hosts": ["host1", "host2", "host3"]}!
  10. 10. {"setName": "rs-name",! "ismaster": true,! "secondary": false,! "hosts": ["host1", "host2", "host3"]}! ismaster
 (from primary)
  11. 11. Updating Topology Description Current description + ismaster response → new description
  12. 12. Updating Topology Description
  13. 13. primary host1 secondary host2 secondary host3 Python Program monitor
 thread monitor
 thread monitor
 thread MongoClient 
 {secondary: true,
 hosts: ["host1", "host2", "host3"]}
 "new info!" Your
 Code? ✓
  14. 14. client.db.collection.insert({'my': 'document'})! client = MongoClient(! 'mongodb://host1,host2/?replicaSet=my-rs')!
  15. 15. primary host1 secondary host2 secondary host3 Python Program monitor
 thread monitor
 thread monitor
 thread MongoClient 
 {ismaster: true,
 hosts: ["host1", "host2", "host3"]}
 {secondary: true,
 hosts: ["host1", "host2", "host3"]}
 Your
 Code insert ok ✓ ✓ "new info!" insert ok ✓
  16. 16. Steady State
  17. 17. primary host1 secondary host2 secondary host3 monitor
 thread monitor
 thread monitor
 thread MongoClient Your
 Code insert ok insert ok ✓ ✓ ✓ Steady State
  18. 18. Crisis
  19. 19. primary host1 secondary host2 secondary host3 monitor
 thread monitor
 thread monitor
 thread MongoClient Your
 Code insert ok insert ok ✓ ✓ ✓
  20. 20. primary host1 secondary host2 secondary host3 monitor
 thread monitor
 thread monitor
 thread Your
 Code insert ConnectionFailure! insert ✓ ✓ ✓ MongoClient ? Crisis
  21. 21. primary host1 secondary host2 secondary host3 monitor
 thread monitor
 thread monitor
 thread Your
 Code insert ✓ ✓ MongoClient "Wake up!" Crisis ?
  22. 22. Resolution
  23. 23. primary host1 secondary host2 secondary host3 monitor
 thread monitor
 thread monitor
 thread Your
 Code ✓ insert "Wake up!" (every half second) MongoClient primary host2 ✓ insert ok ok Resolution ?
  24. 24. So What?
  25. 25. client = MongoClient(! 'mongodb://host1,host2/?replicaSet=my-rs')! ! try:! client.admin.command('ismaster')! except pymongo.errors.ConnectionFailure as e:! print("Can't connect: %s" % e)! Non-Blocking
 Client Construction #1
  26. 26. Error Handling #2 def find_my_document():! try:! return collection.find_one(query)! except ConnectionFailure:! logging.exception("finding document")! return collection.find_one(query)!
  27. 27. Error Handling #2 def insert_doc():! doc = {'_id': ObjectId(), 'a': 1}! try:! collection.insert(doc)! except ConnectionFailure:! logging.exception("inserting document")! # Try again.! collection.insert(doc)! except DuplicateKeyError:! # Previous try actually succeeded.! pass!
  28. 28. #2 Error Handling collection.update(! {'_id': 1},! {'$inc': {'counter': 1}})!
  29. 29. #2 Only idempotent operations can
 be retried Error Handling
  30. 30. Knobs #3 •connectTimeoutMS •serverSelectionTimeoutMS
  31. 31. ConnectionFailure! monitor
 thread monitor
 thread monitor
 thread primary host1 secondary host2 secondary host3 Your
 Code insert insertMongoClient "Wake up!" wait for
 connectTimeoutMS wait for
 serverSelectionTimeoutMS (30 seconds)
  32. 32. bit.ly/server-discovery Further Study:


Server Discovery & Monitoring Spec

×