Getting started with replica set in MongoDB


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Getting started with replica set in MongoDB

  1. 1. Getting Started with Replica Set
  2. 2. Why Replication? • How many have faced node failures? • How many have been woken up from sleep to do • a fail-over(s)? • How many have experienced issues due to network latency? • Different uses for data Normal processing Simple analytics
  3. 3. Replicas Lifecycle
  4. 4. Replica Set – Initialize
  5. 5. Replica Set – Failure
  6. 6. Replica Set – Failover
  7. 7. Replica Set – Recovery
  8. 8. Replica Set – Recovered
  9. 9. Replication
  10. 10. Communication Test Step 1. Start all instants Step 2. All members of a replica set must be able to connect to every other member of the set to support replication. E.g. given replica set with three members running on three different machines host1 : 27017 host2 : 27017 host3 : 27017 Check from host1 ./mongo --host host2IP - - port ./mongo --host host3IP - - port Similar way one by one check from host2 and host3.
  11. 11. Step 3. Start all mongod instance by issuing following command mongod –host --port 27017 --replSet rs0 mongod –host --port 27017 --replSet rs0 mongod –host --port 27017 --replSet rs0 Here rs0 is name of replicaSet name. Step 4. open mongo shell and connect to the first mongod instance ./mongo –port 27017 Step 5. Use rs.initiate() to initiate a replica set consisting of the current member and using the default configuration: rs.initiate() Step 6. Display the current replica configuration rs.conf() Step 7. Add two members to the replica set by issuing a sequence of commands similar to the following. rs.add(“") rs.add(“")
  12. 12. Step 8. Check the status of your replica set at any time with the rs.status() operation. Step 9. Using Configuration file create mongodb.conf file by issuing following commands vi /etc.mongodb.conf with following details port = 27017 bind_ip = dbpath = /data/db fork = true ( not working in windows) replSet = rs0 Step 10. Start mongod by following operation ./mongod --config /etc/mongodb.config
  13. 13. Add Members to a Replica Set Requirements 1. An active replica set. 2. A new MongoDB system capable of supporting your dataset, accessible by the active replica set through the network. 3. Deploy MongoDB new instance, specifying name of replica set. 4. Open mongo shell and connect to replica set’s primary. If you don’t know which member is primary, then issue following commands in mongo shell, db.isMaster() 5. In the mongo shell, issue the following command to add the new member to the replica set. rs.add(“") 6. Confirm new member is instance of replica set’s.
  14. 14. Replica set Maintenance and Administration 1. No downtime 2. Rolling Upgrade maintenance Start with secondary Primary Last
  15. 15. Sharding Shard: A single replica set that stores some portion of total sharded cluster data set. Shard Key : In the sharded collection, shard key is the field that MongoDB uses to distribute the document among the member of sharded clusters. Feature of sharding 1. Range-base Data Partitioning MongoDB distribute document among shards base on the shard key. Each chunk is block of document with value that fall within specific range. 2. Automatic Data Volume Distribution sharding system automatic balance data across cluster without in intervention from application layer. Effective automatic sharding depend on well chosen sharding key.
  16. 16. 3. Transparent Query Routing Sharding is completely transparent to the application layer, because all connection goes through mongos. 4. Horizontal Capacity A typical sharded cluster consist of, • 3 config server that store metadata. Metadata map chunks to shard. • More than one replica set or mongod instances. These are the shard. • A number of lightweight process, called mongos.
  17. 17. Sharding
  18. 18. When to use sharding 1. your data set approaches or exceeds the storage capacity of a single node in your system. 2. The size of your system’s active working set will soon exceed the capacity of the maximum amount of RAM for your system. 3. your system has a large amount of write activity, a single MongoDB instance cannot write data fast enough to meet demand, and all other approaches have not reduced contention. if these attribute are not in your system, sharding will add additional complexity to your system without providing much benefits.
  19. 19. Sharding Requirements 1. Three config sever. For development and testing purposes you may deploy a cluster with a single configuration server process, but always use exactly three config servers for redundancy and safety in production. 2. Two or more shard each shard consist of one or more mongod. Typically each shard is a replica sets. 3. One or more mongos instance. 4. Shard key “Shard keys” refer to the field that exists in every document in a collection that MongoDB uses to distribute documents among the shards.
  20. 20. Setting Up Sharding Step 1. Starting the Servers create a config server database directory by issuing following command, $mkdir –p /configsvr/config The config server needs to be started first, because mongos uses it to get its configuration. $./mongod - -dbpath /configsvr/config - - port 10000 Step 2. Starting mongos servers Routing servers don’t even need a data directory, but they need to know where the config server is: $./mongos --port 20000 --configdb localhost:10000 Shard administration is always done through a mongos.
  21. 21. Step 3. Adding shard A shard is just MongoDB instance (or replica set). Start three MongoDB instances using following operations, $./mongod –dbpath /data/sard1 - - port 10001 $./mongod –dbpath /data/sard2 - - port 10002 $./mongod –dbpath /data/sard3 - - port 10003 Now connect monos process using mongo by issuing following commands $ ./mongo localhost:20000/admin Make sure that you are connected to mongos not to mongod. Now you can add shard by following commands, >db.runCommands ( { addShard : “localhost:10001”, allowLocal : true } ) Out put > { "added" : "localhost:10000", "ok" : true }
  22. 22. The "allowLocal" key is necessary only if you are running the shard on localhost. Step 4. similar way add remain shards
  23. 23. Sharding Data 1. MongoDB won’t just distribute every piece of data you’ve ever stored: you have to explicitly turn sharding on at both the database and collection levels. Create database “shardDB”, first we enable sharding for database blog > db.runCommands ( { “enablesharding” : “shardDB” } ) 2. Once you’ve enabled sharding on the database level, you can shard a collection by running the shardcollection command: > db.runCommands ( { “shardcollection” : “” , “key” : { “_id” : 1} } ) 3. User define sharding key you must create index on sharding key if it is not a “_id” field.
  24. 24. Many Mongos You can also run as many mongos processes as you want. One recommended setup is to run a mongos process for every application server. That way, each application server can talk mongos locally. If server goes down, no will be trying to talk with mongos. A Sturdy Shard In production, production each shard should be a replica set. That way individual server can fail without bringing whole shard. To add replica set as a shard, pass its name and a seed to the addshard command. > db.runCommand ( { "addshard" : "foo/“ } )
  25. 25. Sharding Administration 1. Sharding information mostly stored on config server, which can be accessed by any mongos process. 2. Connect to mongos process to access config database. Switch to config DB issue following commands, > db.getCollectionNames() it shows all collection name. You can find list of shards in shards collection > db.shards.find() each shard assigning unique human readable id. 3. Databases The databases collection contain a list of databases that exist on shards and information about that. >db.databases.find()
  26. 26. > db.databases.find() { "_id" : "admin", "partitioned" : false, "primary" : "config" } { "_id" : "foo", "partitioned" : false, "primary" : "shard1" } { "_id" : "x", "partitioned" : false, "primary" : "shard0" } { "_id" : "test", "partitioned" : true, "primary" : "shard0", "sharded" : { "" : { "key" : {“id" : 1}, "unique" : false } } }
  27. 27. 4. Chunks chunks information stored in chunks collection. You can actually see how your data has been divided up across the cluster. > db.chunks.find() { "_id" : "", "lastmod" : { "t" : 1276636243000, "i" : 1 }, "ns" : "", "min" : { "x" : { $minKey : 1 } }, "max" : { "x" : { $maxKey : 1 } } "shard" : "shard0" }
  28. 28. Sharding Commands 1. The printing sharding status give a quick summary of sharded collection. >db.printShardingStatus() 2. Removing a shard Shards can be removed from a cluster with the removeshard command. Removeshard drains all of the chunks on a given shard to the other shards. >db.runCommand({"removeshard" : "localhost:10000"}); As the shard is drained, removeshard will give you the status of how much remains on the shard. >db.runCommand({"removeshard" : "localhost:10000"}); when the shard has finished draining, removeshard shows that the shard has been successfully removed.