Download to read offline
![23
Client Side View
Secondary
host2
Secondary
host3
Primary
host1
Mongo
Client
Monitor
Thread 1
Monitor
Thread 2
{ ismaster : False,
secondary: True,
hosts : [ host1, host2, host3 ] }](https://image.slidesharecdn.com/b2b-webinar-3-replica-sets-1-170518103133/75/Back-to-Basics-German-3-Einfuhrung-in-Replica-Sets-23-2048.jpg)
![24
What Does ismaster show?
>>> pprint.pprint( db.command( "ismaster" ))
{u'hosts': [u'JD10Gen-old.local:27017',
u'JD10Gen-old.local:27018',
u'JD10Gen-old.local:27019'],
u'ismaster' : False,
u'secondary': True,
u'setName' : u'replset',
…}
>>>](https://image.slidesharecdn.com/b2b-webinar-3-replica-sets-1-170518103133/75/Back-to-Basics-German-3-Einfuhrung-in-Replica-Sets-24-2048.jpg)
![43
What Does This Mean? - Inserts
def insert_with_recovery( collection, doc ) :
doc[ "_id" ] = ObjectId()
try:
collection.insert_one( doc )
except pymongo.errors.ConnectionFailure, e:
logging.info( "Connection error: %s" % e )
try:
collection.insert_one( doc )
except DuplicateKeyError:
pass](https://image.slidesharecdn.com/b2b-webinar-3-replica-sets-1-170518103133/75/Back-to-Basics-German-3-Einfuhrung-in-Replica-Sets-43-2048.jpg)
Uploaded byMongoDB
Back to Basics German 3: Einführung in Replica Sets
The document provides an overview of MongoDB's replica sets, explaining their importance for data durability and high availability. It covers the lifecycle of replica sets, client-side interactions, and error handling protocols in development. Additionally, it includes practical examples and references to further reading on server discovery and monitoring in MongoDB.
More Related Content
Similar to Back to Basics German 3: Einführung in Replica Sets
![MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]](https://cdn.slidesharecdn.com/ss_thumbnails/315pminfosysfinalsfoversionvocalpart1-200120221508-thumbnail.jpg?width=640&height=640&fit=bounds)
Back to Basics German 3: Einführung in Replica Sets
- 2. Back to Basics2017 : Webinar 3 Einführung in Replica Sets Benjamin Lorenz Senior Solutions Architect MongoDB Frankfurt
- 3. 3 Zusammenfassung von Teil1 und Teil 2 • Warum NoSQL • Unterschiedliche Typen von NoSQL-Datenbanken • MongoDB: Detaillierte Übersicht • Datenbankkonzepte • Installieren von MongoDB • Erstellen einer einfachen Blogging-Anwendung • Hinzufügen eines Index • Abfrageoptimierung mit explain()
- 4. 4 Ablauf heute • Dauerhaftigkeit(durability) von Daten • Der MongoDB-Ansatz: Das Replica Set • Replica Set-Lebenszyklus • Programmierung mit einem Replica Set
- 5. 5 HighAvailability and DataDurability – Replica Sets SecondarySecondary Primary
- 6.
- 7. 7 Replica Set NodeFailure SecondarySecondary Primary No Heartbeat
- 8.
- 9. 9 New Replica Set– 2 Nodes SecondaryPrimary Heartbeat And New Primary
- 10.
- 11.
- 12.
- 13. 13 Driver Responsibilities https://github.com/mongodb/mongo-python-driver Driver Authentication & Security Python<->BSON Errorhandling & Recovery Wire Protocol Topology Management Connection Pool
- 14.
- 15.
- 16. 16 Write Concerns • Networkacknowledgement • Wait for error • Wait for journal sync • Wait for replication
- 17. 17 Write Concern :0 (Unacknowledged)
- 18. 18 Write Concern :1 (Acknowledged)
- 19. 19 Write Concern :majority
- 20. 20 Driver Responsibilities https://github.com/mongodb/mongo-python-driver Driver Authentication & Security Python<->BSON Errorhandling & Recovery Wire Protocol Topology Management Connection Pool
- 21. 21 Start MongoClient c =MongoClient( "host1, host2", replicaSet="replset" )
- 22. 22 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client MongoClient("host1, host2", replicaSet="replset" )
- 23. 23 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 { ismaster : False, secondary: True, hosts : [ host1, host2, host3 ] }
- 24. 24 What Does ismastershow? >>> pprint.pprint( db.command( "ismaster" )) {u'hosts': [u'JD10Gen-old.local:27017', u'JD10Gen-old.local:27018', u'JD10Gen-old.local:27019'], u'ismaster' : False, u'secondary': True, u'setName' : u'replset', …} >>>
- 25.
- 26.
- 27. 27 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3
- 28. 28 Client Side View Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3 Your Code
- 29. 29 Next Is Insert c= MongoClient( "host1, host2", replicaSet="replset" ) client.db.col.insert_one( { "a" : "b" } )
- 30. 30 Insert Will Block Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3 Your Code Insert
- 31. 31 ismaster response fromHost 1 Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert ismaster
- 32. 32 Now Write CanProceed Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code Insert Insert
- 33. 33 Later Host 3Responds Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code
- 34. 34 Steady State Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code
- 35. 35 Life Intervenes Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code ✖
- 36. 36 Monitor may notdetect Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert ConnectionFailure
- 37. 37 So Retry Secondary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code ✖ Insert
- 38. 38 Check for Primary Secondary host2 Secondary host3 Mongo Client Monitor Thread1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 39. 39 Host 2 IsPrimary Primary host2 Secondary host3 Mongo Client Monitor Thread 1 Monitor Thread 2 Monitor Thread 3 Your Code ✖ Insert
- 40. 40 Steady State Secondary host2 Secondary host3 Primary host1 Mongo Client Monitor Thread 1 Monitor Thread2 Monitor Thread 3 Your Code
- 41. 41 What Does ThisMean? - Connect import pymongo client = pymongo.MongoClient() try: client.admin.command( "ismaster" ) except pymongo.errors.ConnectionFailure, e : print( "Cannot connect: %s" % e )
- 42. 42 What Does ThisMean? - Queries import pymongo def find_with_recovery( collection, query ) : try: return collection.find_one( query ) except pymongo.errors.ConnectionFailure, e : logging.info( "Connection failure : %s" e ) return collection.find_one( query )
- 43. 43 What Does ThisMean? - Inserts def insert_with_recovery( collection, doc ) : doc[ "_id" ] = ObjectId() try: collection.insert_one( doc ) except pymongo.errors.ConnectionFailure, e: logging.info( "Connection error: %s" % e ) try: collection.insert_one( doc ) except DuplicateKeyError: pass
- 44. 44 What Does ThisMean? - Updates collection.update( { "_id" : 1 }, { "$inc" : { "counter" : 1 }})
- 45. 45 More Reading • Thespec author Jess Jiryu Davis has a collection of links and his better version of this talk https://emptysqua.re/blog/server-discovery-and-monitoring-in-mongodb- drivers/ • The full server discovery and monitoring spec is on GitHub https://github.com/mongodb/specifications/blob/master/source/server- discovery-and-monitoring/server-discovery-and-monitoring.rst
- 46.
Editor's Notes
- #3 Who I am, how long have I been at MongoDB.
- #14 Present a native language interface - converts python types to BSON objects Convert the JSON query language into commands for the database Convert JSON data into BSON data and vice-versa Handles interfacing to different MongoDB topologies Helps recover from server side outages/network errors Manages the client side connection pool The pymongo driver code is on Github (Apache License)
- #21 Present a native language interface - converts python types to BSON objects Convert the JSON query language into commands for the database Convert JSON data into BSON data and vice-versa Handles interfacing to different MongoDB topologies Helps recover from server side outages/network errors Manages the client side connection pool The pymongo driver code is on Github (Apache License)
- #23 Calls i
- #24 Calls i
- #26 State machine, full set of states defined in spec.
- #27 Calls i
- #28 Calls i
- #29 Calls i
- #31 Needs a primary to complete a write.
- #32 Needs a primary to complete a write.
- #33 Needs a primary to complete a write.
- #34 Needs a primary to complete a write.
- #35 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #36 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #37 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #38 Primary is marked as unknown Wakes up all monitor threads to check for a primary.
- #39 Primary is marked as unknown Wakes up all monitor threads to check for a primary every half second.
- #40 Primary is marked as unknown Wakes up all monitor threads to check for a primary every half second.
- #41 Each thread wakes every 10 seconds. Runs ismaster, sleeps. We use ismaster to check latency. Keep topology description up to date.
- #43 Try once. This will accomdate elections. Other errore should be propagated.
- #44 Try once. This will accomdate elections. Other errore should be propagated.
- #45 Can you afford to over or under count. Operations need to be idempotent. Turn an update into a write of a document, cf EventSourcing. Then aggregate on the server.