2. Why Replication?
• How many have faced node failures?
• How many have been woken up from sleep to do
• a fail-over(s)?
• How many have experienced issues due to
network latency?
• Different uses for data
Normal processing
Simple analytics
10. Communication Test
Step 1. Start all instants
Step 2. All members of a replica set must be able to connect to every other
member of the set to support replication.
E.g. given replica set with three members running on three different
machines
host1 : 27017
host2 : 27017
host3 : 27017
Check from host1
./mongo --host host2IP - - port
./mongo --host host3IP - - port
Similar way one by one check from host2 and host3.
11. Step 3. Start all mongod instance by issuing following command
mongod –host 10.1.1.61 --port 27017 --replSet rs0
mongod –host 10.1.1.62 --port 27017 --replSet rs0
mongod –host 10.1.1.63 --port 27017 --replSet rs0
Here rs0 is name of replicaSet name.
Step 4. open mongo shell and connect to the first mongod instance
./mongo 10.1.1.61 –port 27017
Step 5. Use rs.initiate() to initiate a replica set consisting of the
current member and using the default configuration:
rs.initiate()
Step 6. Display the current replica configuration
rs.conf()
Step 7. Add two members to the replica set by issuing a sequence of
commands similar to the following.
rs.add(“10.1.1.62:27017")
rs.add(“10.1.1.63:27017")
12. Step 8. Check the status of your replica set at any time with the rs.status()
operation.
Step 9. Using Configuration file
create mongodb.conf file by issuing following commands
vi /etc.mongodb.conf with following details
port = 27017
bind_ip = 10.1.1.61
dbpath = /data/db
fork = true ( not working in windows)
replSet = rs0
Step 10. Start mongod by following operation
./mongod --config /etc/mongodb.config
13. Add Members to a Replica Set
Requirements
1. An active replica set.
2. A new MongoDB system capable of supporting your dataset, accessible
by the active replica set through the network.
3. Deploy MongoDB new instance, specifying name of replica set.
4. Open mongo shell and connect to replica set’s primary. If you don’t know
which member is primary, then issue following commands in mongo
shell,
db.isMaster()
5. In the mongo shell, issue the following command to add the new member
to the replica set.
rs.add(“10.1.1.61:27017")
6. Confirm new member is instance of replica set’s.
14. Replica set Maintenance
and Administration
1. No downtime
2. Rolling Upgrade maintenance
Start with secondary
Primary Last
15. Sharding
Shard: A single replica set that stores some portion of total sharded cluster
data set.
Shard Key : In the sharded collection, shard key is the field that MongoDB
uses to distribute the document among the member of sharded
clusters.
Feature of sharding
1. Range-base Data Partitioning
MongoDB distribute document among shards base on the shard key.
Each chunk is block of document with value that fall within specific
range.
2. Automatic Data Volume Distribution
sharding system automatic balance data across cluster without in
intervention from application layer. Effective automatic sharding depend
on well chosen sharding key.
16. 3. Transparent Query Routing
Sharding is completely transparent to the application layer, because
all connection goes through mongos.
4. Horizontal Capacity
A typical sharded cluster consist of,
• 3 config server that store metadata. Metadata map chunks to shard.
• More than one replica set or mongod instances. These are the shard.
• A number of lightweight process, called mongos.
18. When to use sharding
1. your data set approaches or exceeds the storage capacity of a single node
in your system.
2. The size of your system’s active working set will soon exceed the capacity
of the maximum amount of RAM for your system.
3. your system has a large amount of write activity, a single MongoDB
instance cannot write data fast enough to meet demand, and all other
approaches have not reduced contention.
if these attribute are not in your system, sharding will add additional
complexity to your system without providing much benefits.
19. Sharding Requirements
1. Three config sever.
For development and testing purposes you may deploy a cluster with a
single configuration server process, but always use exactly three config
servers for redundancy and safety in production.
2. Two or more shard
each shard consist of one or more mongod. Typically each shard is a
replica sets.
3. One or more mongos instance.
4. Shard key
“Shard keys” refer to the field that exists in every document in a collection
that MongoDB uses to distribute documents among the shards.
20. Setting Up Sharding
Step 1. Starting the Servers
create a config server database directory by issuing following
command,
$mkdir –p /configsvr/config
The config server needs to be started first, because mongos uses it
to get its configuration.
$./mongod - -dbpath /configsvr/config - - port 10000
Step 2. Starting mongos servers
Routing servers don’t even need a data directory, but they need to
know where the config server is:
$./mongos --port 20000 --configdb localhost:10000
Shard administration is always done through a mongos.
21. Step 3. Adding shard
A shard is just MongoDB instance (or replica set).
Start three MongoDB instances using following operations,
$./mongod –dbpath /data/sard1 - - port 10001
$./mongod –dbpath /data/sard2 - - port 10002
$./mongod –dbpath /data/sard3 - - port 10003
Now connect monos process using mongo by issuing following commands
$ ./mongo localhost:20000/admin
Make sure that you are connected to mongos not to mongod.
Now you can add shard by following commands,
>db.runCommands ( { addShard : “localhost:10001”, allowLocal : true } )
Out put > {
"added" : "localhost:10000",
"ok" : true
}
22. The "allowLocal" key is necessary only if you are running the shard
on localhost.
Step 4. similar way add remain shards
23. Sharding Data
1. MongoDB won’t just distribute every piece of data you’ve ever stored:
you have to explicitly turn sharding on at both the database and
collection levels.
Create database “shardDB”, first we enable sharding for database blog
> db.runCommands ( { “enablesharding” : “shardDB” } )
2. Once you’ve enabled sharding on the database level, you can shard a
collection by running the shardcollection command:
> db.runCommands ( { “shardcollection” : “shardDB.blog” ,
“key” : { “_id” : 1} } )
3. User define sharding key
you must create index on sharding key if it is not a “_id” field.
24. Many Mongos
You can also run as many mongos processes as you want. One
recommended setup is to run a mongos process for every application
server. That way, each application server can talk mongos locally. If server
goes down, no will be trying to talk with mongos.
A Sturdy Shard
In production, production each shard should be a replica set. That way
individual server can fail without bringing whole shard. To add replica set
as a shard, pass its name and a seed to the addshard command.
> db.runCommand ( { "addshard" : "foo/10.1.1.61:27017“ } )
25. Sharding Administration
1. Sharding information mostly stored on config server, which can be
accessed by any mongos process.
2. Connect to mongos process to access config database. Switch to config
DB issue following commands,
> db.getCollectionNames()
it shows all collection name.
You can find list of shards in shards collection
> db.shards.find()
each shard assigning unique human readable id.
3. Databases
The databases collection contain a list of databases that exist on shards
and information about that.
>db.databases.find()
27. 4. Chunks
chunks information stored in chunks collection. You can actually see how your data
has been divided up across the cluster.
> db.chunks.find()
{
"_id" : "test.foo-x_MinKey",
"lastmod" : { "t" : 1276636243000, "i" : 1 },
"ns" : "test.foo",
"min" : {
"x" : { $minKey : 1 }
},
"max" : {
"x" : { $maxKey : 1 }
}
"shard" : "shard0"
}
28. Sharding Commands
1. The printing sharding status give a quick summary of sharded collection.
>db.printShardingStatus()
2. Removing a shard
Shards can be removed from a cluster with the removeshard command.
Removeshard drains all of the chunks on a given shard to the other shards.
>db.runCommand({"removeshard" : "localhost:10000"});
As the shard is drained, removeshard will give you the status of how much
remains on the shard.
>db.runCommand({"removeshard" : "localhost:10000"});
when the shard has finished draining, removeshard shows that the shard
has been successfully removed.
