Downloaded 46 times
![24
Client Side View
Secondary
host2
Secondary
host3
Primary
host1
Mongo
Client
Monitor
Thread 1
Monitor
Thread 2
{ ismaster : False,
secondary: True,
hosts : [ host1, host2, host3 ] }](https://image.slidesharecdn.com/b2b-webinar-3-replica-sets-1-170202121146/85/Back-to-Basics-Webinar-3-Introduction-to-Replica-Sets-24-320.jpg)
![25
What Does ismaster show?
>>> pprint.pprint( db.command( "ismaster" ))
{u'hosts': [u'JD10Gen-old.local:27017',
u'JD10Gen-old.local:27018',
u'JD10Gen-old.local:27019'],
u'ismaster' : False,
u'secondary': True,
u'setName' : u'replset',
…}
>>>](https://image.slidesharecdn.com/b2b-webinar-3-replica-sets-1-170202121146/85/Back-to-Basics-Webinar-3-Introduction-to-Replica-Sets-25-320.jpg)
![44
What Does This Mean? - Inserts
def insert_with_recovery( collection, doc ) :
doc[ "_id" ] = ObjectId()
try:
collection.insert_one( doc )
except pymongo.errors.ConnectionFailure, e:
logging.info( "Connection error: %s" % e )
collection.insert_one( doc )
except DuplicateKeyError:
pass](https://image.slidesharecdn.com/b2b-webinar-3-replica-sets-1-170202121146/85/Back-to-Basics-Webinar-3-Introduction-to-Replica-Sets-44-320.jpg)
Uploaded byMongoDB
Back to Basics Webinar 3: Introduction to Replica Sets
This document provides an introduction to MongoDB replica sets, which allow for data redundancy and high availability. It discusses how replica sets work, including the replica set life cycle and how applications should handle writes and queries when using a replica set. Specifically, it explains that the MongoDB driver is responsible for server discovery and monitoring, retry logic, and handling topology changes in a replica set to provide a consistent view of the data to applications.
More Related Content
Similar to Back to Basics Webinar 3: Introduction to Replica Sets
![MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]](https://cdn.slidesharecdn.com/ss_thumbnails/315pminfosysfinalsfoversionvocalpart1-200120221508-thumbnail.jpg?width=640&height=640&fit=bounds)
![제 23회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [MBOAX] : ABSA를 활용한 소비자 반응 분석 기반 운영 효율화 대시보드 설계](https://cdn.slidesharecdn.com/ss_thumbnails/3-1boaz23rdconferencemboax-260203102709-9d519923-thumbnail.jpg?width=640&height=640&fit=bounds)
![Hacking-Uncovered-How-People-Get-Hacked-and-How-to-Stay-Safe[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/hacking-uncovered-how-people-get-hacked-and-how-to-stay-safe1-260130170011-4883a9c7-thumbnail.jpg?width=640&height=640&fit=bounds)
Back to Basics Webinar 3: Introduction to Replica Sets
- 2. Back to Basics2017 : Webinar 3 Introduction to Replica Sets Joe Drumgoole Director of Developer Advocacy, EMEA MongoDB @jdrumgoole V1.0
- 3. 3 Summary of Part1 and 2 • Why NoSQL exists • The types of NoSQL database • The key features of MongoDB • How to install MongoDB • How to do the basic CRUD operations • How to create indexes • How to use the explain() plan • MongoDB Compass and MongoDB Atlas
- 4. 4 Agenda • Data durability •The MongoDB Approach – The replica set • Replica Set Life-cycle • How to program when using a replica set
- 5. 5 Next Webinar :Introduction to Sharding • How to build a highly scalable performant cluster • How to remove write bottlenecks • How to select a shard key Thursday, 9-Feb-2016, 11:00 am GMT.
- 6. 6 HighAvailability and DataDurability – Replica Sets SecondarySecondary Primary
- 7.
- 8. 8 Replica Set NodeFailure SecondarySecondary Primary No Heartbeat
- 9.
- 10. 10 New Replica Set– 2 Nodes SecondaryPrimary Heartbeat And New Primary
- 11.
- 12.
- 13.
- 14. 14 Driver Responsibilities https://github.com/mongodb/mongo-python-driver Driver Authentication & Security Python<->BSON Errorhandling & Recovery Wire Protocol Topology Management Connection Pool
- 15.
- 16.
- 17. 17 Write Concerns • Networkacknowledgement • Wait for error • Wait for journal sync • Wait for replication
- 18. 18 Write Concern :0 (Unacknowledged)
- 19. 19 Write Concern :1 (Acknowledged)
- 20. 20 Write Concern :majority
- 21. 21 Driver Responsibilities https://github.com/mongodb/mongo-python-driver Driver Authentication & Security Python<->BSON Errorhandling & Recovery Wire Protocol Topology Management Connection Pool
- 22. 22 Start MongoClient c =MongoClient( "host1, host2", replicaSet="replset" )
- 23. 23 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client MongoClient("host1, host2", replicaSet="replset" )
- 24. 24 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 { ismaster : False, secondary: True, hosts : [ host1, host2, host3 ] }
- 25. 25 What Does ismastershow? >>> pprint.pprint( db.command( "ismaster" )) {u'hosts': [u'JD10Gen-old.local:27017', u'JD10Gen-old.local:27018', u'JD10Gen-old.local:27019'], u'ismaster' : False, u'secondary': True, u'setName' : u'replset', …} >>>
- 26.
- 27.
- 28. 28 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3
- 29. 29 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3 Your Code
- 30. 30 Next Is Insert c= MongoClient( "host1, host2", replicaSet="replset" ) client.db.col.insert_one( { "a" : "b" } )
- 31. 31 Insert Will Block Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3 Your Code Insert
- 32. 32 ismaster response fromHost 1 Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert ismaster
- 33. 33 Now Write CanProceed Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert Insert
- 34. 34 Later Host 3Responds Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code
- 35. 35 Steady State Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code
- 36. 36 Life Intervenes Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code ✖
- 37. 37 Monitor may notdetect Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert ConnectionFailure
- 38. 38 So Retry Secondary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code ✖ Insert
- 39. 39 Check for Primary Secondary host2 Secondary host3 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 40. 40 Host 2 IsPrimary Primary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 41. 41 Steady State Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code
- 42. 42 What Does ThisMean? - Connect import pymongo client = pymongo.MongoClient() try: client.admin.command( "ismaster" ) except pymongo.errors.ConnectionFailure, e : print( "Cannot connect: %s" % e )
- 43. 43 What Does ThisMean? - Queries import pymongo def find_with_recovery( collection, query ) : try: return collection.find_one( query ) except pymongo.errors.ConnectionFailure, e : logging.info( "Connection failure : %s" e ) return collection.find_one( query )
- 44. 44 What Does ThisMean? - Inserts def insert_with_recovery( collection, doc ) : doc[ "_id" ] = ObjectId() try: collection.insert_one( doc ) except pymongo.errors.ConnectionFailure, e: logging.info( "Connection error: %s" % e ) collection.insert_one( doc ) except DuplicateKeyError: pass
- 45. 45 What Does ThisMean? - Updates collection.update( { "_id" : 1 }, { "$inc" : { "counter" : 1 }})
- 46.
- 47. 47 connectTimeoutMS Secondary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread3 Your Code ✖ Insert connectTimeoutMS serverTimeoutMS
- 48. 48 More Reading • Thespec author Jess Jiryu Davis has a collection of links and his better version of this talk https://emptysqua.re/blog/server-discovery-and-monitoring-in-mongodb- drivers/ • The full server discovery and monitoring spec is on GitHub https://github.com/mongodb/specifications/blob/master/source/server- discovery-and-monitoring/server-discovery-and-monitoring.rst
- 49.
Editor's Notes
- #3 Who I am, how long have I been at MongoDB.
- #15 Present a native language interface - converts python types to BSON objects Convert the JSON query language into commands for the database Convert JSON data into BSON data and vice-versa Handles interfacing to different MongoDB topologies Helps recover from server side outages/network errors Manages the client side connection pool The pymongo driver code is on Github (Apache License)
- #22 Present a native language interface - converts python types to BSON objects Convert the JSON query language into commands for the database Convert JSON data into BSON data and vice-versa Handles interfacing to different MongoDB topologies Helps recover from server side outages/network errors Manages the client side connection pool The pymongo driver code is on Github (Apache License)
- #24 Calls i
- #25 Calls i
- #27 State machine, full set of states defined in spec.
- #28 Calls i
- #29 Calls i
- #30 Calls i
- #32 Needs a primary to complete a write.
- #33 Needs a primary to complete a write.
- #34 Needs a primary to complete a write.
- #35 Needs a primary to complete a write.
- #36 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #37 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #38 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #39 Primary is marked as unknown Wakes up all monitor threads to check for a primary.
- #40 Primary is marked as unknown Wakes up all monitor threads to check for a primary every half second.
- #41 Primary is marked as unknown Wakes up all monitor threads to check for a primary every half second.
- #42 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #44 Try once. This will accomdate elections. Other errore should be propagated.
- #45 Try once. This will accomdate elections. Other errore should be propagated.
- #46 Can you afford to over or under count. Operations need to be idempotent. Turn an update into a write of a document, cf EventSourcing. Then aggregate on the server.
- #48 How long should a connection wait before timing out and sleeping for 10 seconds.