MongoDB数据库sharding集群增加和删除节点

作者：贺娥岚761 | 来源：互联网 | 2014-05-28 16:53

MongoDB的Auto-Sharding能够做到：1当各Sharding间负载和数据分布不平衡时，自动rebalancing2简单方便的添加和删除节点3自动故障转移(autofailover)4可扩展至上千台节点如何增加shard节点，在之前的shard集群配置过程使用过。当向一个sharding

MongoDB的Auto-Sharding能够做到：

1 当各Sharding间负载和数据分布不平衡时，自动rebalancing

2 简单方便的添加和删除节点

3 自动故障转移(auto failover)

4 可扩展至上千台节点

如何增加shard节点，在之前的shard集群配置过程使用过。当向一个sharding集群添加新的节点，mongodb 会将在其他节点的数据chunk迁移到新的节点上面。以便达到均分数据的目的，这也算是负载均衡吧。

添加之前：

mongos> db.printShardingStatus()

--- Sharding Status ---

sharding version: { "_id" : 1, "version" : 3 }

shards:

{ "_id" : "shard0000", "host" : "10.250.7.225:27018" }

{ "_id" : "shard0001", "host" : "10.250.7.249:27019" }

{ "_id" : "shard0002", "host" : "10.250.7.241:27020" }

databases:

{ "_id" : "admin", "partitioned" : false, "primary" : "config" }

{ "_id" : "test", "partitioned" : true, "primary" : "shard0000" }

test.momo chunks:

shard0000 30

shard0001 26

shard0002 24

too many chunks to print, use verbose if you want to force print

........省略.......

Noet：对于由于chunks数量过大，而显示“too many chunks to print, use verbose if you want to force print”，可以使用如下方式查看：

printShardingStatus(db.getSisterDB("config"),1);

在admin 数据库操作

mongos> use admin

switched to db admin

mongos> db.runCommand({addshard:"10.250.7.225:27019"})

{ "shardAdded" : "shard0003", "ok" : 1 }

这里添加很短时间就返回结果，但是后台要花一定的时间来做数据 chunk的迁移，从其他shard节点迁移到新的节点上面.

mongos> db.runCommand({ listShards : 1});

{

"shards" : [

{

"_id" : "shard0000",

"host" : "10.250.7.225:27018"

{

"_id" : "shard0001",

"host" : "10.250.7.249:27019"

{

"_id" : "shard0002",

"host" : "10.250.7.241:27020"

{

"_id" : "shard0003",

"host" : "10.250.7.225:27019"

}

"ok" : 1

}

过一段时间再看：已经做了数据的平均分布了。

mongos> printShardingStatus(db.getSisterDB("config"),1);

--- Sharding Status ---

sharding version: { "_id" : 1, "version" : 3 }

shards:

{ "_id" : "shard0000", "host" : "10.250.7.225:27018" }

{ "_id" : "shard0001", "host" : "10.250.7.249:27019" }

{ "_id" : "shard0002", "host" : "10.250.7.241:27020" }

{ "_id" : "shard0003", "host" : "10.250.7.225:27019" }

databases:

{ "_id" : "admin", "partitioned" : false, "primary" : "config" }

{ "_id" : "test", "partitioned" : true, "primary" : "shard0000" }

test.momo chunks:

shard0003 16

shard0001 21

shard0000 21

shard0002 23

{ "id" : { $minKey : 1 } } -->> { "id" : 0 } on : shard0003 { "t" : 28000, "i" : 0 }

{ "id" : 0 } -->> { "id" : 5236 } on : shard0003 { "t" : 33000, "i" : 0 }

{ "id" : 5236 } -->> { "id" : 11595 } on : shard0003 { "t" : 35000, "i" : 0 }

{ "id" : 11595 } -->> { "id" : 17346 } on : shard0003 { "t" : 37000, "i" : 0 }

{ "id" : 17346 } -->> { "id" : 23191 } on : shard0003 { "t" : 40000, "i" : 0 }

{ "id" : 23191 } -->> { "id" : 31929 } on : shard0003 { "t" : 43000, "i" : 0 }

.....省略部分....

{ "id" : 930108 } -->> { "id" : 948575 } on : shard0002 { "t" : 21000, "i" : 7 }

{ "id" : 948575 } -->> { "id" : 957995 } on : shard0002 { "t" : 27000, "i" : 42 }

{ "id" : 957995 } -->> { "id" : 969212 } on : shard0002 { "t" : 27000, "i" : 43 }

{ "id" : 969212 } -->> { "id" : 983794 } on : shard0002 { "t" : 25000, "i" : 6 }

{ "id" : 983794 } -->> { "id" : 999997 } on : shard0002 { "t" : 25000, "i" : 7 }

{ "id" : 999997 } -->> { "id" : { $maxKey : 1 } } on : shard0002 { "t" : 11000, "i" : 3 }

test.yql chunks:

shard0003 1

shard0000 1

shard0002 1

shard0001 1

{ "_id" : { $minKey : 1 } } -->> { "_id" : ObjectId("4eb298b3adbd9673afee95e3") } on : shard0003 { "t" : 5000, "i" : 0 }

{ "_id" : ObjectId("4eb298b3adbd9673afee95e3") } -->> { "_id" : ObjectId("4eb2a64640643e5bb60072f7") } on : shard0000 { "t" : 4000, "i" : 1 }

{ "_id" : ObjectId("4eb2a64640643e5bb60072f7") } -->> { "_id" : ObjectId("4eb2a65340643e5bb600e084") } on : shard0002 { "t" : 3000, "i" : 1 }

{ "_id" : ObjectId("4eb2a65340643e5bb600e084") } -->> { "_id" : { $maxKey : 1 } } on : shard0001 { "t" : 5000, "i" : 1 }

{ "_id" : "mongos", "partitioned" : false, "primary" : "shard0000" }

附上日志记录：

##启动信息

Sat Nov 5 17:41:23 [initandlisten] MongoDB starting : pid=11807 port=27019 dbpath=/opt/mongodata/r2 64-bit host=rac1

Sat Nov 5 17:41:23 [initandlisten] db version v2.0.1, pdfile version 4.5

Sat Nov 5 17:41:23 [initandlisten] git version: 3a5cf0e2134a830d38d2d1aae7e88cac31bdd684

Sat Nov 5 17:41:23 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41

Sat Nov 5 17:41:23 [initandlisten] options: { dbpath: "/opt/mongodata/r2", logappend: true, logpath: "/opt/mongodata/r1/27019.log", port: 27019, shardsvr: true }

Sat Nov 5 17:41:23 [initandlisten] journal dir=/opt/mongodata/r2/journal

Sat Nov 5 17:41:23 [initandlisten] recover : no journal files present, no recovery needed

Sat Nov 5 17:41:23 [initandlisten] waiting for connections on port 27019

Sat Nov 5 17:41:23 [websvr] admin web console waiting for connections on port 28019

###连接其他节点，并复制数据

Sat Nov 5 17:41:53 [initandlisten] connection accepted from 10.250.7.220:46807 #1

Sat Nov 5 17:42:03 [initandlisten] connection accepted from 10.250.7.225:57578 #2

Sat Nov 5 17:42:03 [FileAllocator] allocating new datafile /opt/mongodata/r2/test.ns, filling with zeroes...

Sat Nov 5 17:42:03 [FileAllocator] creating directory /opt/mongodata/r2/_tmp

Sat Nov 5 17:42:03 [FileAllocator] done allocating datafile /opt/mongodata/r2/test.ns, size: 16MB, took 0.1 secs

Sat Nov 5 17:42:03 [FileAllocator] allocating new datafile /opt/mongodata/r2/test.0, filling with zeroes...

Sat Nov 5 17:42:06 [FileAllocator] done allocating datafile /opt/mongodata/r2/test.0, size: 64MB, took 3.143 secs

Sat Nov 5 17:42:06 [migrateThread] build index test.momo { _id: 1 }

Sat Nov 5 17:42:06 [migrateThread] build index done 0 records 0 secs

Sat Nov 5 17:42:06 [migrateThread] info: creating collection test.momo on add index

Sat Nov 5 17:42:06 [migrateThread] build index test.momo { id: 1.0 }

Sat Nov 5 17:42:06 [migrateThread] build index done 0 records 0 secs

Sat Nov 5 17:42:06 [FileAllocator] allocating new datafile /opt/mongodata/r2/test.1, filling with zeroes...

Sat Nov 5 17:42:07 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: MinKey } -> { id: 0.0 }

Sat Nov 5 17:42:07 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: MinKey } -> { id: 0.0 }

Sat Nov 5 17:42:07 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: MinKey } -> { id: 0.0 }

Sat Nov 5 17:42:07 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: MinKey } -> { id: 0.0 }

Sat Nov 5 17:42:07 [migrateThread] about to log metadata event: { _id: "rac1-2011-11-05T09:42:07-0", server: "rac1", clientAddr: "", time: new Date(1320486127651), wh

at: "moveChunk.to", ns: "test.momo", details: { min: { id: MinKey }, max: { id: 0.0 }, step1: 3271, step2: 217, step3: 0, step4: 0, step5: 520 } }

Sat Nov 5 17:42:07 [migrateThread] SyncClusterConnection connecting to [rac1:28001]

Sat Nov 5 17:42:07 [migrateThread] SyncClusterConnection connecting to [rac2:28002]

Sat Nov 5 17:42:07 [migrateThread] SyncClusterConnection connecting to [rac3:28003]

Sat Nov 5 17:42:07 [FileAllocator] done allocating datafile /opt/mongodata/r2/test.1, size: 128MB, took 1.011 secs

Sat Nov 5 17:42:13 [initandlisten] connection accepted from 10.250.7.249:40392 #3

Sat Nov 5 17:42:13 [migrateThread] build index test.yql { _id: 1 }

Sat Nov 5 17:42:13 [migrateThread] build index done 0 records 0.001 secs

Sat Nov 5 17:42:13 [migrateThread] info: creating collection test.yql on add index

Sat Nov 5 17:42:13 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.yql' { _id: MinKey } -> { _id: ObjectId('4eb298b3adbd9673afee95e3') }

Sat Nov 5 17:42:13 [migrateThread] migrate commit flushed to journal for 'test.yql' { _id: MinKey } -> { _id: ObjectId('4eb298b3adbd9673afee95e3') }

Sat Nov 5 17:42:14 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.yql' { _id: MinKey } -> { _id: ObjectId('4eb298b3adbd9673afee95e3') }

Sat Nov 5 17:42:14 [migrateThread] migrate commit flushed to journal for 'test.yql' { _id: MinKey } -> { _id: ObjectId('4eb298b3adbd9673afee95e3') }

Sat Nov 5 17:42:14 [migrateThread] about to log metadata event: { _id: "rac1-2011-11-05T09:42:14-1", server: "rac1", clientAddr: "", time: new Date(1320486134775), wh

at: "moveChunk.to", ns: "test.yql", details: { min: { _id: MinKey }, max: { _id: ObjectId('4eb298b3adbd9673afee95e3') }, step1: 5, step2: 0, step3: 0, step4: 0, step5:

1006 } }

Sat Nov 5 17:42:16 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: 102100 } -> { id: 120602 }

Sat Nov 5 17:42:16 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: 102100 } -> { id: 120602 }

Sat Nov 5 17:42:17 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: 102100 } -> { id: 120602 }

Sat Nov 5 17:42:17 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: 102100 } -> { id: 120602 }

Sat Nov 5 17:42:17 [migrateThread] about to log metadata event: { _id: "rac1-2011-11-05T09:42:17-2", server: "rac1", clientAddr: "", time: new Date(1320486137351), wh

at: "moveChunk.to", ns: "test.momo", details: { min: { id: 102100 }, max: { id: 120602 }, step1: 0, step2: 0, step3: 1573, step4: 0, step5: 479 } }

Sat Nov 5 17:42:20 [conn2] end connection 10.250.7.225:57578

Sat Nov 5 17:42:21 [initandlisten] connection accepted from 10.250.7.220:46814 #4

Sat Nov 5 17:42:21 [conn4] warning: bad serverID set in setShardVersion and none in info: EOO

Sat Nov 5 18:06:47 [initandlisten] connection accepted from 10.250.7.225:13612 #6

Sat Nov 5 18:06:47 [migrateThread] Socket say send() errno:32 Broken pipe 10.250.7.225:27018

Sat Nov 5 18:06:47 [migrateThread] about to log metadata event: { _id: "rac1-2011-11-05T10:06:47-3", server: "rac1", clientAddr: "", time: new Date(1320487607530), wh

at: "moveChunk.to", ns: "test.momo", details: { min: { id: 120602 }, max: { id: 132858 }, note: "aborted" } }

Sat Nov 5 18:06:47 [migrateThread] not logging config change: rac1-2011-11-05T10:06:47-3 SyncClusterConnection::insert prepare failed: 9001 socket exception [2] serve

r [127.0.0.1:28001] rac1:28001:{}

Sat Nov 5 18:07:00 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: 120602 } -> { id: 132858 }

Sat Nov 5 18:07:00 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: 120602 } -> { id: 132858 }

Sat Nov 5 18:07:01 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: 120602 } -> { id: 132858 }

Sat Nov 5 18:07:01 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: 120602 } -> { id: 132858 }

Sat Nov 5 18:07:01 [migrateThread] about to log metadata event: { _id: "rac1-2011-11-05T10:07:01-4", server: "rac1", clientAddr: "", time: new Date(1320487621150), wh

at: "moveChunk.to", ns: "test.momo", details: { min: { id: 120602 }, max: { id: 132858 }, step1: 0, step2: 0, step3: 1121, step4: 0, step5: 886 } }

Sat Nov 5 18:07:01 [migrateThread] SyncClusterConnection connecting to [rac1:28001]

Sat Nov 5 18:07:01 [migrateThread] SyncClusterConnection connecting to [rac2:28002]

Sat Nov 5 18:07:01 [migrateThread] SyncClusterConnection connecting to [rac3:28003]

Sat Nov 5 18:07:17 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: 142178 } -> { id: 154425 }

Sat Nov 5 18:07:18 [migrateThread] migrate commit succeeded flushing to secondaries for 'test.momo' { id: 142178 } -> { id: 154425 }

Sat Nov 5 18:07:18 [migrateThread] migrate commit flushed to journal for 'test.momo' { id: 142178 } -> { id: 154425 }

Sat Nov 5 18:07:18 [migrateThread] about to log metadata event: { _id: "rac1-2011-11-05T10:07:18-6", server: "rac1", clientAddr: "", time: new Date(1320487638676), wh

at: "moveChunk.to", ns: "test.momo", details: { min: { id: 142178 }, max: { id: 154425 }, step1: 0, step2: 0, step3: 1108, step4: 0, step5: 940 } }

.....省略部分.....

Sat Nov 5 18:09:23 [clientcursormon] mem (MB) res:55 virt:413 mapped:80

Sat Nov 5 18:12:21 [conn1] command admin.$cmd command: { writebacklisten: ObjectId('4eb4e43618ed672581e26201') } ntoreturn:1 reslen:44 300012ms

Sat Nov 5 18:14:24 [clientcursormon] mem (MB) res:55 virt:413 mapped:80

Sat Nov 5 18:17:21 [conn1] command admin.$cmd command: { writebacklisten: ObjectId('4eb4e43618ed672581e26201') } ntoreturn:1 reslen:44 300012ms

Sat Nov 5 18:19:24 [clientcursormon] mem (MB) res:55 virt:413 mapped:80

二删除节点

集群对于删除节点，也会将被删除节点上的数据迁移到其他的节点上面。

db.runCommand({ listShards : 1});

mongos> db.runCommand({removeshard:"10.250.7.225:27018"})

{

"msg" : "draining started successfully",

"state" : "started",

"shard" : "shard0000",

"ok" : 1

}

mongos> db.runCommand({ listShards : 1});

{

"shards" : [

{

"_id" : "shard0001",

"host" : "10.250.7.249:27019"

{

"_id" : "shard0002",

"host" : "10.250.7.241:27020"

{

"_id" : "shard0003",

"host" : "10.250.7.225:27019"

{

"_id" : "shard0000",

"draining" : true, --正在迁移数据

"host" : "10.250.7.225:27018"

}

"ok" : 1

}

mongos>

删除之后：

mongos> db.printShardingStatus()

--- Sharding Status ---

sharding version: { "_id" : 1, "version" : 3 }

shards:

{ "_id" : "shard0000", "draining" : true, "host" : "10.250.7.225:27018" }

{ "_id" : "shard0001", "host" : "10.250.7.249:27019" }

{ "_id" : "shard0002", "host" : "10.250.7.241:27020" }

{ "_id" : "shard0003", "host" : "10.250.7.225:27019" }

databases:

{ "_id" : "admin", "partitioned" : false, "primary" : "config" }

{ "_id" : "test", "partitioned" : true, "primary" : "shard0000" }

test.momo chunks:

shard0003 27

shard0001 28

shard0002 27

too many chunks to print, use verbose if you want to force print

test.yql chunks:

shard0003 1

shard0001 2

shard0002 1

{ "_id" : { $minKey : 1 } } -->> { "_id" : ObjectId("4eb298b3adbd9673afee95e3") } on : shard0003 { "t" : 5000, "i" : 0 }

{ "_id" : ObjectId("4eb298b3adbd9673afee95e3") } -->> { "_id" : ObjectId("4eb2a64640643e5bb60072f7") } on : shard0001 { "t" : 6000, "i" : 0 }

{ "_id" : ObjectId("4eb2a64640643e5bb60072f7") } -->> { "_id" : ObjectId("4eb2a65340643e5bb600e084") } on : shard0002 { "t" : 3000, "i" : 1 }

{ "_id" : ObjectId("4eb2a65340643e5bb600e084") } -->> { "_id" : { $maxKey : 1 } } on : shard0001 { "t" : 5000, "i" : 1 }

{ "_id" : "mongos", "partitioned" : false, "primary" : "shard0000" }

mongos>

推荐阅读

linux
技术变现之道：从日常工作中挖掘潜力

本文探讨了如何在日常工作中通过优化效率和深入研究核心技术，将技术和知识转化为实际收益。文章结合个人经验，分享了提高工作效率、掌握高价值技能以及选择合适工作环境的方法，帮助读者更好地实现技术变现。 ... [详细]

蜡笔小新 2024-12-24 15:21:23
get
告别传统文件传输，迎接新一代高效工具Croc

在现代网络环境中，两台计算机之间的文件传输需求日益增长。传统的FTP和SSH方式虽然有效，但其配置复杂、步骤繁琐，难以满足快速且安全的传输需求。本文将介绍一种基于Go语言开发的新一代文件传输工具——Croc，它不仅简化了操作流程，还提供了强大的加密和跨平台支持。 ... [详细]

蜡笔小新 2024-12-26 16:16:06
install
基于Node.js、Express、MongoDB和Socket.io的实时聊天应用开发

本文详细介绍了使用Node.js、Express、MongoDB和Socket.io构建的实时聊天应用程序。涵盖项目结构、技术栈选择及关键依赖项的配置。 ... [详细]

蜡笔小新 2024-12-22 15:31:28
function
Mongoose 5.12.10 发布：MongoDB 异步对象模型工具的新特性与修复

Mongoose 是一款专为异步环境设计的 MongoDB 对象模型工具，支持 Promise 和回调函数。最新版本 Mongoose 5.12.10 带来了多项修复和改进，包括查询选项中的默认值设置、嵌入式判别器填充、以及 TypeScript 定义文件的优化。 ... [详细]

蜡笔小新 2024-12-20 18:54:14
uri
Docker的安全基准

nsitionalENhttp:www.w3.orgTRxhtml1DTDxhtml1-transitional.dtd ... [详细]

蜡笔小新 2024-12-28 13:00:24
uri
资源推荐 | TensorFlow官方中文教程助力英语非母语者学习

来源：机器之心。本文详细介绍了TensorFlow官方提供的中文版教程和指南，帮助开发者更好地理解和应用这一强大的开源机器学习平台。 ... [详细]

蜡笔小新 2024-12-28 09:00:51
list
Handling Null Object Encoding in OAuth 1.0a API Implementation

Explore a common issue encountered when implementing an OAuth 1.0a API, specifically the inability to encode null objects and how to resolve it. ... [详细]

蜡笔小新 2024-12-28 08:54:34
function
网络链路质量监控：Smokeping部署与配置

本文详细介绍了如何在Linux系统上安装和配置Smokeping，以实现对网络链路质量的实时监控。通过详细的步骤和必要的依赖包安装，确保用户能够顺利完成部署并优化其网络性能监控。 ... [详细]

蜡笔小新 2024-12-27 19:31:05
function
Python 的 10 个开发技巧！太实用了

1.如何在运行状态查看源代码？查看函数的源代码，我们通常会使用IDE来完成。比如在PyCharm中，你可以Ctrl+鼠标点击进入函数的源代码。那如果没有IDE呢？当我们想使用一个函 ... [详细]

蜡笔小新 2024-12-27 18:36:54
function
开源软件：新时代的商业机遇与挑战

在哈佛大学商学院举行的Cyberposium大会上，专家们深入探讨了开源软件的崛起及其对企业市场的影响。会议指出，开源软件不仅为企业提供了新的增长机会，还促进了软件质量的提升和创新。 ... [详细]

蜡笔小新 2024-12-27 14:49:56
string
扫描线三巨头 hdu1928hdu 1255 hdu 1542 [POJ 1151]

学习链接：http:blog.csdn.netlwt36articledetails48908031学习扫描线主要学习的是一种扫描的思想，后期可以求解很 ... [详细]

蜡笔小新 2024-12-26 20:04:36
string
SQL基础入门与数据库体系概述

本文介绍了数据库体系的基础知识，涵盖关系型数据库（如MySQL）和非关系型数据库（如MongoDB）的基本操作及高级功能。通过三个阶段的学习路径——基础、优化和部署，帮助读者全面掌握数据库的使用和管理。 ... [详细]

蜡笔小新 2024-12-22 19:17:15
uri
深入解析Spring Cloud微服务架构与分布式系统实战

本文详细介绍了Spring Cloud在微服务架构和分布式系统中的应用，结合实际案例和最新技术，帮助读者全面掌握微服务的实现与优化。 ... [详细]

蜡笔小新 2024-12-19 16:02:50
const
ThinkPHP6多数据库部署指南

本文将详细介绍如何在ThinkPHP6框架中实现多数据库的部署，包括读写分离的策略，以及如何通过负载均衡和MySQL同步技术优化数据库性能。 ... [详细]

蜡笔小新 2024-12-17 18:59:28
install
使用LVS与ldirectord实现高可用负载均衡

本文介绍了如何通过LVS（Linux Virtual Server）结合ldirectord工具来实现服务器的健康检查及负载均衡功能。环境设置包括一个LVS节点和两个真实服务器节点，通过配置ldirectord进行健康状态监测，确保系统的高可用性。 ... [详细]

蜡笔小新 2024-12-17 17:30:12

贺娥岚761

这个家伙很懒，什么也没留下！

Tags | 热门标签

RankList | 热门文章