MongoDB分片集群部署详解

2023-08-23 12:02:05 331

一、环境说明

1、我们prod环境MongoDB的集群架构是做的分片集群的部署，但是目前我们没有分片，即所有数据都在一个分片上，后期如果数量大，需要分配，集群随时可以分片，对业务方透明

2、各个角色的部署情况

角色	IP	端口	复制集名称
mongos	172.21.244.101，172.21.244.102，172.21.244.94	27000	无
configserver	172.21.244.101，172.21.244.102，172.21.244.94	27100	repl_configsvr
存储节点（shard）	172.21.244.101，172.21.244.102，172.21.244.94	27101	shard1

3、MongoDB版本

mongos>db.version()
4.0.4-62-g7e345a7

二、基础信息准备

0、系统优化

echo"never">/sys/kernel/mm/transparent_hugepage/enabled
echo"never">/sys/kernel/mm/transparent_hugepage/defrag

1、下载MongoDB二进制文件

cd/chj/app
wgetops.chehejia.com:9090/pkg/chj_mongodb_4.0.4.tar.gz
tar-zxvfchj_mongodb_4.0.4.tar.gz

2、相关目录建立

#建立base目录
mkdir/chj/data/mongodb/chj_db
#把MongoDB二进制文件移动到base目录下的bin文件夹
mvchj_mongodb_4.0.4/bin/chj/data/mongodb/chj_db/bin
#建立认证文件目录
mkdir/chj/data/mongodb/chj_db/auth
#建立配置文件目录
mkdir/chj/data/mongodb/chj_db/conf
#建立configserver的data和日志目录
mkdir/chj/data/mongodb/chj_db/config/data-p
mkdir/chj/data/mongodb/chj_db/config/log
#建立mongos的日志目录
mkdir/chj/data/mongodb/chj_db/mongos/log-p
#建立数据节点data和日志目录
mkdir/chj/data/mongodb/chj_db/shard1/data-p
mkdir/chj/data/mongodb/chj_db/shard1/log

3、相关配置文件编写

A、mongos的配置文件编写

vim/chj/data/mongodb/chj_db/conf/mongos.conf
systemLog:
destination:file
logAppend:true
path:/chj/data/mongodb/chj_db/mongos/log/mongos.log

processManagement:
fork:true#forkandruninbackground
pidFilePath:/chj/data/mongodb/chj_db/mongos/log/mongos.pid#locationofpidfile
timeZoneInfo:/usr/share/zoneinfo

net:
port:27000
bindIpAll:true
maxIncomingConnections:1000
unixDomainSocket:
enabled:true
pathPrefix:/chj/data/mongodb/chj_db/mongos/log
filePermissions:0700

security:
keyFile:/chj/data/mongodb/chj_db/auth/keyfile.key
#authorization:enabled

#replication:

sharding:
configDB:repl_configsvr/172.21.244.101:27100,172.21.244.102:27100,172.21.244.94:27100

B、configserver的配置文件编写

vim/chj/data/mongodb/chj_db/conf/config.conf
systemLog:
destination:file
logAppend:true
path:/chj/data/mongodb/chj_db/config/log/congigsrv.log

storage:
dbPath:/chj/data/mongodb/chj_db/config/data
journal:
enabled:true
wiredTiger:
engineConfig:
directoryForIndexes:true

processManagement:
fork:true#forkandruninbackground
pidFilePath:/chj/data/mongodb/chj_db/config/log/configsrv.pid#locationofpidfile
timeZoneInfo:/usr/share/zoneinfo

net:
port:27100
bindIp:0.0.0.0#Enter0.0.0.0,::tobindtoallIPv4andIPv6addressesor,alternatively,usethenet.bindIpAllsetting.
#bindIpAll:true
maxIncomingConnections:1000
unixDomainSocket:
enabled:true
pathPrefix:/chj/data/mongodb/chj_db/config/data
filePermissions:0700

security:
keyFile:/chj/data/mongodb/chj_db/auth/keyfile.key
authorization:enabled

replication:
replSetName:repl_configsvr
sharding:
clusterRole:configsvr

C、存储节点的配置文件编写

vim/chj/data/mongodb/chj_db/conf/shard1.conf
systemLog:
destination:file
logAppend:true
path:/chj/data/mongodb/chj_db/shard1/log/shard1.log

storage:
dbPath:/chj/data/mongodb/chj_db/shard1/data
journal:
enabled:true
wiredTiger:
engineConfig:
directoryForIndexes:true

processManagement:
fork:true#forkandruninbackground
pidFilePath:/chj/data/mongodb/chj_db/shard1/log/shard1.pid#locationofpidfile
timeZoneInfo:/usr/share/zoneinfo

net:
port:27101
bindIp:0.0.0.0#Enter0.0.0.0,::tobindtoallIPv4andIPv6addressesor,alternatively,usethenet.bindIpAllsetting.
#bindIpAll:true
maxIncomingConnections:1000
unixDomainSocket:
enabled:true
pathPrefix:/chj/data/mongodb/chj_db/shard1/data
filePermissions:0700

security:
keyFile:/chj/data/mongodb/chj_db/auth/keyfile.key
authorization:enabled

replication:
replSetName:shard1
sharding:
clusterRole:shardsvr

4、生产key认证文件

echo"chj123456">/chj/data/mongodb/chj_db/auth/keyfile.key
#设置文件的权限为400，不然服务无法启动
chmod400/chj/data/mongodb/chj_db/auth/keyfile.key

三、集群初始化

1、启动configserver服务

/chj/data/mongodb/chj_db/bin/mongod-f/chj/data/mongodb/chj_db/conf/config.conf

2、初始化configserver集群

#登录其中一个configserver节点
/chj/data/mongodb/chj_db/bin/mongo--port27100
#配置集群
config={_id:"repl_configsvr",members:[{_id:0,host:"172.21.244.101:27100"},{_id:1,host:"172.21.244.102:27100"},{_id:2,host:"172.21.244.94:27100"}]}
#初始化集群
rs.initiate(config)
PS：
结果输出如下，说明集群初始化成功，可以通过rs.status()命令查看集群状态
{
"ok":1,
"$gleStats":{
"lastOpTime":Timestamp(1557538260,1),
"electionId":ObjectId("000000000000000000000000")
},
"lastCommittedOpTime":Timestamp(0,0)
}

3、启动存储节点服务

/chj/data/mongodb/chj_db/bin/mongod-f/chj/data/mongodb/chj_db/conf/shard1.conf

4、初始化存储集群

#登录你希望是主节点的服务器
/chj/data/mongodb/chj_db/bin/mongo--port27101
#配置集群
config={_id:"shard1",members:[{_id:0,host:"172.21.244.101:27101"},{_id:1,host:"172.21.244.102:27101"},{_id:2,host:"172.21.244.94:27101",arbiterOnly:true}]}
#初始化集群
rs.initiate(config)
PS：
结果输出如下，说明集群初始化成功，可以通过rs.status()命令查看集群状态
{"ok":1}

5、添加存储集群的管理账号

登录主节点

/chj/data/mongodb/chj_db/bin/mongo--port27101
useadmin
db.createUser(
{
user:"root",
pwd:"123456",
roles:[{role:"root",db:"admin"}]
}
)

6、启动mongos服务

/chj/data/mongodb/chj_db/bin/mongos-f/chj/data/mongodb/chj_db/conf/mongos.conf

7、添加configserver的管理账号

登录任意一个mongos节点

/chj/data/mongodb/chj_db/bin/mongo--port27000
useadmin
db.createUser(
{
user:"root",
pwd:"123456",
roles:[{role:"root",db:"admin"}]
}
)

8、把存储节点添加到mongos

登录任意一个mongos节点（如果是在上一步的窗口，需要退出重新登录）

/chj/data/mongodb/chj_db/bin/mongo--port27000
useadmin
db.auth('root','123456')
#添加分片
sh.addShard('shard1/172.21.244.101:27101,172.21.244.102:27101,172.21.244.94:27101')
#查看分片状态
sh.status()

四、交付业务方

1、建立应用账号

登录任意一个mongos节点
/chj/data/mongodb/chj_db/bin/mongo--port27000
useadmin
db.auth('root','123456')
#切到业务数据库
usechj_db
#建立读写账号
db.createUser(
{
user:"chj_db_rw",
pwd:"123456",
roles:[
{role:"readWrite",db:"chj_db"},
{role:"dbOwner",db:"chj_db"}
]
}
)
#建立只读账号(根据业务需求确认是否需要)
db.createUser(
{
user:"chj_db_r",
pwd:"123456",
roles:[{role:"read",db:"chj_db"}]
}
)

2、交付开发人员信息

连接地址：172.21.244.101:27000，172.21.244.102:27000，172.21.244.94:27000
库名：chj_db
账号：chj_db_rw
密码：123456

五、数据库启用分片

如果后期业务量大，需要开启分片，配置如下

#指定需要分片的数据库
mongos>sh.enableSharding("chj_db")
{
"ok":1,
"operationTime":Timestamp(1557546835,3),
"$clusterTime":{
"clusterTime":Timestamp(1557546835,3),
"signature":{
"hash":BinData(0,"bkrrr8Kxrr9j9udrDc/hURHld38="),
"keyId":NumberLong("6689575940508352541")
}
}
}
#在chj_db数据库和users集合中创建了name和age为升序的片键
mongos>sh.shardCollection("chj_db.users",{name:1,age:1})
{
"collectionsharded":"chj_db.users",
"collectionUUID":UUID("59c0b99f-efff-4132-b489-f6c7e3d98f42"),
"ok":1,
"operationTime":Timestamp(1557546861,12),
"$clusterTime":{
"clusterTime":Timestamp(1557546861,12),
"signature":{
"hash":BinData(0,"UBB1A/YODnmXwG5eAhgNLcKVzug="),
"keyId":NumberLong("6689575940508352541")
}
}
}
#查看分片情况
mongos>sh.status()
---ShardingStatus---
shardingversion:{
"_id":1,
"minCompatibleVersion":5,
"currentVersion":6,
"clusterId":ObjectId("5cd625e0da695346d740f749")
}
shards:
{"_id":"shard1","host":"shard1/172.21.244.101:27101,172.21.244.102:27101","state":1}
activemongoses:
"4.0.4-62-g7e345a7":3
autosplit:
Currentlyenabled:yes
balancer:
Currentlyenabled:yes
Currentlyrunning:no
Failedbalancerroundsinlast5attempts:0
MigrationResultsforthelast24hours:
Norecentmigrations
databases:
{"_id":"chj_db","primary":"shard1","partitioned":true,"version":{"uuid":UUID("82088bc7-7b98-4033-843d-7058d8d959f6"),"lastMod":1}}
chj_db.users
shardkey:{"name":1,"age":1}
unique:false
balancing:true
chunks:
shard11
{"name":{"$minKey":1},"age":{"$minKey":1}}-->>{"name":{"$maxKey":1},"age":{"$maxKey":1}}on:shard1Timestamp(1,0)
{"_id":"config","primary":"config","partitioned":true}
config.system.sessions
shardkey:{"_id":1}
unique:false
balancing:true
chunks:
shard11
{"_id":{"$minKey":1}}-->>{"_id":{"$maxKey":1}}on:shard1Timestamp(1,0)

以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持毛票票。

MongoDB分片集群部署详解

热门推荐

随机推荐