Mongo db部署架构之优先方案
Upcoming SlideShare
Loading in...5
×
 

Mongo db部署架构之优先方案

on

  • 1,430 views

 

Statistics

Views

Total Views
1,430
Slideshare-icon Views on SlideShare
1,430
Embed Views
0

Actions

Likes
1
Downloads
23
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft Word

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mongo db部署架构之优先方案 Mongo db部署架构之优先方案 Document Transcript

    • Mongo 之架构部署 (Replica Sets+Sharding)1、环境要构建一个 MongoDB Sharding Cluster,需要三种角色:•Shard Server: mongod 实例,用于存储实际的数据块。创建人:李航围脖:http://weibo.com/lidaohang§
    • •Config Server: mongod 实例,存储了整个 Cluster Metadata,其中包括 chunk 信息。•Route Server: mongos 实例,前端路由,客户端由此接入,且让整个集群看上去像单一进程 数据库。方案一:192.168.136.14 192.168.136.15 192.168.136.16 192.168.136.26 192.168.136.29Shard1(master) Shard2(master) Shard3(master) Shard1(slave) Shard1(arbiter)Shard2(slave) Shard3(slave) Shard1(slave) Shard3(slave) Shard2(arbiter)Shard3(arbiter) Shard1(arbiter) Shard2(arbiter) Shard2(slave) Shard3 (arbiter)1.节点: S1: 192.168.136..14,192.168.136..26,192.168.136..16,192.168.136.15,192.168.136.29(arbiter)S2: 192.168.136.15,192.168.136.26,192.168.136.14,192.168.136.16,192.168.136.29(arbiter)S3: 192.168.136.16,192.168.136.26,192.168.136.15,192.168.136.14,192.168.136.29(arbiter)c主机 端口信息192.168.136.14 mongod shard1:27017(master) mongod shard2:27018(slave) mongod shard3:27019(arbiter) mongod config:30000 mongs:40000192.168.136.15 mongod shard1:27017(arbiter) mongod shard2:27018(master) mongod shard3:27019(slave) mongod config:30000 mongs:40000192.168.136.16 mongod shard1:27017(slave) mongod shard2:27018(arbiter) mongod shard3:27019(master) mongod config:30000 mongs:40000192.168.136.26 mongod shard1:27017(slave) mongod shard2:27018(slave) mongod shard3:27019(slave)192.168.136.29 mongod shard1:27017(arbiter) mongod shard2:27018(arbiter) mongod shard3:27019(arbiter)创建人:李航围脖:http://weibo.com/lidaohang§
    • 2、安装部署软件准备及目录1.下载 mongodb 程序 curl -O http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-1.8.3.tgz2.解压 mongodb-linux-x86_64-2.0.0.tgz tar zxvf mongodb-linux-x86_64-1.8.3.tgz3.重命名 mongodb-linux-x86_64-2.0.0.tgz 为 mongodb mv mongodb-linux-x86_64-2.0.0 mongodb4.进入 mongodb 目录 cd mongodb5.新建文件夹 data mkdir data mkdir logs配置 Replica Sets,Config Server※配置配置文件 conf# 1.start_mongod Shard1.confshardsvr=trueport=27017dbpath=/data/database/shard1/logpath=/data/logs/shard1.loglogappend=truefork=truereplSet=s1rest=truejournal=true# 2.start_mongod Shard2.confshardsvr=trueport=27018dbpath=/data/database/shard2/logpath=/data/logs/shard2.loglogappend=truefork=truereplSet=s2rest=true创建人:李航围脖:http://weibo.com/lidaohang§
    • journal=true# 3.start_mongod Shard3.confshardsvr=trueport=27019dbpath=/data/database/shard3/logpath=/data/logs/shard3.loglogappend=truefork=truereplSet=s3rest=truejournal=true192.168.136.141.创建相应的文件夹(注意此处的 testadmin 是客户端登陆名)mkdir -p /data/database/shard1mkdir -p /data/database/shard2mkdir -p /data/database/shard3mkdir -p /data/database/configmkdir -p /data/logs2.配置 mongod./mongod --config /mongodb/shard1.conf./mongod --config /mongodb/shard2.conf./mongod --config /mongodb/shard3.conf./mongod --config /mongodb/configsvr.conf3.查看 mongod 的进程是否启动ps aux | grep mongodb | grep -v grep4.初始化 replica sets(此处 ip 是对应的内网 IP)/testadmin/mongodb/bin/mongo --port 27017 config = {_id: s1, members: [{_id: 0,host: 192.168.136.14:27017,priority:5},{_id: 1, host:192.168.136.26:27017,priority:2},{_id: 2, host: 192.168.136.16:27017,priority:.5},{_id: 3, host:192.168.136.15:27017,arbiterOnly: true},{_id: 4, host: 192.168.136.29:27017, arbiterOnly:true}]}创建人:李航围脖:http://weibo.com/lidaohang§
    • rs.initiate(config) rs.status()192.168.136.151.创建相应的文件夹(注意此处的 testadmin 是客户端登陆名)mkdir -p /data/database/shard1mkdir -p /data/database/shard2mkdir -p /data/database/shard3mkdir -p /data/database/configmkdir -p /data/logs2.配置 mongod./mongod --config /mongodb/shard1.conf./mongod --config /mongodb/shard2.conf./mongod --config /mongodb/shard3.conf./mongod --config /mongodb/configsvr.conf3.查看 mongod 的进程是否启动ps aux | grep mongodb | grep -v grep4.初始化 replica sets/testadmin/mongodb/bin/mongo --port 27018 config={_id: s2, members: [{_id: 0, host: 192.168.136.15:27018,priority:5},{_id: 1, host:192.168.136.26:27018,priority:2},{_id: 2, host: 192.168.136.14:27018,priority:.5},{_id: 3, host:192.168.136.16:27018, arbiterOnly: true},{_id: 4, host: 192.168.136.29:27018, arbiterOnly:true}]} rs.initiate(config)192.168.136.161.创建相应的文件夹(注意此处的 testadmin 是客户端登陆名)mkdir -p /data/shard1mkdir -p /data/shard2mkdir -p /data/shard3mkdir -p /data/configmkdir -p /data/logs创建人:李航围脖:http://weibo.com/lidaohang§
    • 2.配置 mongod./mongod --config /mongodb/shard1.conf./mongod --config /mongodb/shard2.conf./mongod --config /mongodb/shard3.conf./mongod --config /mongodb/configsvr.conf3.查看 mongod 的进程是否启动ps aux | grep mongodb | grep -v grep4.初始化 replica sets/testadmin/mongodb/bin/mongo --port 27019 config={_id: s3,members: [{_id: 0, host: 192.168.136.16:27019,priority:5},{_id: 1, host:192.168.136.26:27019,priority:2},{_id: 2, host: 192.168.136.15:27019,priority:.5},{_id: 3, host:192.168.136.14:27019, arbiterOnly: true},{_id: 4, host: 192.168.136.29:27019, arbiterOnly:true}]}rs.status()192.168.136.26,192.168.136.29 分别执行如下:1.创建相应的文件夹(注意此处的 testadmin 是客户端登陆名)mkdir -p /data/shard1mkdir -p /data/shard2mkdir -p /data/shard3mkdir -p /data/configmkdir -p /data/logs2.配置 mongod./mongod --config /mongodb/shard1.conf./mongod --config /mongodb/shard2.conf./mongod --config /mongodb/shard3.conf3.查看 mongod 的进程是否启动ps aux | grep mongodb | grep -v grep配置 Mongos(在每一台机子上建立路由)创建人:李航围脖:http://weibo.com/lidaohang§
    • /mongodb/bin/./mongos --fork --port 40000 --logpath /data/logs/mongos.log --configdb192.168.136.14:30000,192.168.136.15:30000,192.168.136.16:30000添加分片1 连接任意一台,其他无需这样操作:/home/testadmin/bin/mongo --port 40000use admindb.runCommand({addshard:s1/192.168.136.14:27017,192.168.136.26:27017,192.168.136.16:27017})db.runCommand({addshard:s2/192.168.136.15:27018,192.168.136.26:27018,192.168.136.14:27018})db.runCommand({addshard:s3/192.168.136.16:27019,192.168.136.26:27019,192.168.136.15:27019})db.runCommand({ listshards:1 })db.runCommand({ enablesharding:weibo })db.runCommand({shardcollection:weibo.test, key:{_id:1},unique:true}) printShardingStatus()db.data.stats();创建人:李航围脖:http://weibo.com/lidaohang§
    • 3、用户认证1.注意 1.9.1 之前复制集不支持用户认证,只能通过 keyFile 密匙文件,幸好这几天 2.0 正式版出来了,很多功能问题都已解决。呵呵注意:用户验证,启动 mongod 必须添加--auth#设置用户名和密码>use test>db.addUser(mongo,456123);>db.auth(mongo,456123)>db.system.users.find() --查看该用户是否添加成功>db.system.users.remove(mongo,456123)>mongo 数据库 -u mongo -p注意:启动时必须添加--auth 用户权限才会生效,第一次配置完成后,没效果就重启下。4、鸭梨大了怎么办?添加服务器,怎么样添加呢?1.如果读得鸭梨大了,则添加加 slave 节点,分散读得鸭梨。 启动后,在 primary 节点中添加节点 如:rs.add("10.168.0.100:27017") 当我们看到变为 secondary 后,就一切正常。2.如果写的鸭梨打了,则可以添加一组 shard 节点分散写的鸭梨。 如:如上所述启动 mongod,添加即可。五、备份恢复策略增量备份(添加延迟备份节点)1、利用另外一台 secondary 机器传送数据2、在新机器上中配置文件中添加 fastsync=true 配置(当需要从有数据中启动节点,那么必创建人:李航围脖:http://weibo.com/lidaohang§
    • 须加上 fastsync=true ,否则启动会报错,如果是从主库直接同步所有数据,那么就不需要添加这个参数)3、启动后,在 primary 节点中添加节点如:rs.add("10.168.0.102:27017") 当我们看到变为 secondary 后,就一切正常,可以正常提供线上服务了4、通过 rs.conf()命令查看现在的节点信息(需要 admin 库密码权限)5、rs.remove("10.168.0.102:27017")删除节点6、添加 arbiter 节点:rs.addArb("10.73.24.171:19003")7、添加延时备份机器:rs.add({_id:5,host:"10.168.0.102:27017",priority:0,slaveDelay:300});rs.add({_id:5,host:"10.168.0.102:27018",priority:0,slaveDelay:300});rs.add({_id:5,host:"10.168.0.102:27019",priority:0,slaveDelay:300});注意:slaveDelay 单位秒.8、出现这个错误时:replSet error RS102 too stale to catch up,我们可以db.printReplicationInfo()查看主库、从库的 oplog 信息;利用延迟备份节点恢复数据 1. 先把延迟备份节点的数据,备份到各节点的 master 机子上。如: #./mongodump -h 192.168.136.14:27017 -d weibo -o /data/mongoback/ #./mongodump -h 192.168.136.15:27018 -d weibo -o /data/mongoback/ #./mongodump -h 192.168.136.16:27019 -d weibo -o /data/mongoback/ 2. 把备份的数据导入到个节点的 master 上。如: 建议先修复下,压缩空间 db.repairDatabase();修复数据(和压缩(删除数据)空间) ./mongorestore -h 127.0.0.1:27017 --directoryperdb /data/mongoback --drop --indexesLast ./mongorestore -h 127.0.0.1:27018 --directoryperdb /data/mongoback --drop --indexesLast ./mongorestore -h 127.0.0.1:27019 --directoryperdb /data/mongoback --drop --indexesLast全量备份(添加延迟备份节点)1. 写好脚本定期凌晨备份数据,如: ./mongodump -h 10.168.0.187:40000 -d weibo -o /data/mongoback/2. 恢复数据创建人:李航围脖:http://weibo.com/lidaohang§
    • 3. 建议先修复下,压缩空间4. db.repairDatabase();修复数据(和压缩(删除数据)空间) ./mongorestore -h 10.168.0.187:40000 --directoryperdb /data/mongoback --drop --indexesLast六、 其他问题 ~1.如果启动不成功,则尝试修复.如: ./mongod --port 27017 --repair --dbpath /data/database/shard1/2 如果 master 节点 kill 了,则起来之后通过 rs.stepDown(100)让出 master 的位置。3.其他问题请随时欢迎提出,联系我(李航),围脖:http://weibo.com/lidaohang~创建人:李航围脖:http://weibo.com/lidaohang§