qfs集群安装之一metaserver安装

下载程序

下载centos 6的版本：

https://s3.amazonaws.com/quantcast-qfs/qfs-centos-6.7-master-x86_64.tgz

然后解压

Meta Server

负责保存所有chunk的位置，协调chunk的复制和恢复。

配置文件

需要创建一个配置文件，文件名为meta_server.prp，这是简要配置

# port used by clients to connect to the metaservermetaServer.clientPort = 20000

# port used by chunk servers to connect to the metaserver
metaServer.chunkServerPort = 30000

# chunk placement groups by IP address or first three octets
# metaServer.rackPrefixes = 

# create new file system if no transaction logs or checkpoints are found
metaServer.createEmptyFs = 1

# location to write transaction logs
metaServer.logDir = /your-path/qfs_logs

# location to write checkpoints, this needs be pruned periodically
metaServer.cpDir = /your-path/qfs_checkpoint

# unique cluster id
metaServer.clusterKey = dean-qfs

完整配置从这里下载。

注意完整配置文件分成两部分，一部分是静态配置，必须重启服务才生效，一部份是动态配置，可以通过发SIGHUP信号给meta server进程生效。两部分中间有分割说明

# ==============================================================================# The parameters below this line can be changed at runtime by editing the# configuration file and sending meta server process HUP signal.# Note that to restore parameter to default at run time the default value must# be explicitly specified in the configuration file. In other words commenting# out the parameter will not have any effect until restart.

创建qfs用户

创建qfs用户，adduser qfs，然后将之前配置的所有目录都改成qfs为owner。

初始化

切换到qfs用户，运行下面的命令

/your-path/qfs/bin/metaserver -c /your-path/qfs_conf/meta_server.prp

遇到错误：

10-13-2016 02:06:13.083 ERROR - (UserAndGroup.cc:648) getpwent duplicate user id: 0 name: a cur: root group: 0 cur: 010-13-2016 02:06:13.083 ERROR - (UserAndGroup.cc:648) getpwent duplicate user id: 0 name: b cur: root group: 0 cur: 0

这是个意外，服务器上存在a和b两个帐号，使用0为uid，和root一样。

这样不能直接删除用户，需要(用root帐号)先到/etc/passwd中将a和b的uid改成两个新值，然后再用userdel 命令删除a和b

现在再试运行一次命令

继续报错：

10-13-2016 02:10:20.687 ERROR - (TcpSocket.cc:575) 0.0.0.0:30000 Address already in use 9810-13-2016 02:10:20.687 ERROR - (Acceptor.cc:102) failed to bind to:  30000 error: Address already in use 98

30000端口被占用了。换一个，修改配置文件为40000。再来。 这次过了。

Loading key metaServer.clientPort with value 20000Loading key metaServer.chunkServerPort with value 40000Loading key metaServer.rackPrefixes with value Loading key metaServer.createEmptyFs with value 1Loading key metaServer.logDir with value /your-path/qfs_logsLoading key metaServer.cpDir with value /data/slot0/qfs_checkpointLoading key metaServer.clusterKey with value dean-qfsLoading key metaServer.clusterKey with value dean-qfs10-13-2016 02:15:47.171 INFO - (nofilelimit.cc:82) max # of open files: 60000010-13-2016 02:15:47.173 INFO - (metaserver_main.cc:460) meta server client listner:  2000010-13-2016 02:15:47.173 INFO - (metaserver_main.cc:479) meta server chunk server listener:  4000010-13-2016 02:15:47.173 INFO - (metaserver_main.cc:488) path->fid cache disabled10-13-2016 02:15:47.173 INFO - (metaserver_main.cc:351) min chunk servers that should connect: 110-13-2016 02:15:47.173 INFO - (metaserver_main.cc:359) min. # of replicas per file: 110-13-2016 02:15:47.173 INFO - (metaserver_main.cc:520) hard limits: open files: 600000 chunk servers: 4096 clients: 59564810-13-2016 02:15:47.173 INFO - (LayoutManager.cc:1728) rack: prefix:  id: 1010-13-2016 02:15:47.173 INFO - (LayoutManager.cc:2060) max. response size: 268435456 minIoBufferBytesToProcessRequest: 27892121610-13-2016 02:15:47.173 INFO - (metaserver_main.cc:541) creating empty file system10-13-2016 02:15:47.173 INFO - (metaserver_main.cc:706) creating new empty file system10-13-2016 02:15:47.173 INFO - (metaserver_main.cc:729) replaying logs10-13-2016 02:15:47.173 INFO - (metaserver_main.cc:747) updating space utilization

启动服务

将刚才的命令去掉 -c，保存为脚本文件，然后运行脚本：(-c只能用一次，以后集群正常运转的时候使用-c会丢失数据）

$ ./start-qfs-meta-server.shLoading key metaServer.clientPort with value 20000Loading key metaServer.chunkServerPort with value 40000Loading key metaServer.rackPrefixes with value Loading key metaServer.createEmptyFs with value 1Loading key metaServer.logDir with value /your-path/qfs_logsLoading key metaServer.cpDir with value /data/slot0/qfs_checkpointLoading key metaServer.clusterKey with value dean-qfsLoading key metaServer.clusterKey with value dean-qfs10-13-2016 02:18:26.080 INFO - (nofilelimit.cc:82) max # of open files: 60000010-13-2016 02:18:26.081 INFO - (metaserver_main.cc:460) meta server client listner:  2000010-13-2016 02:18:26.081 INFO - (metaserver_main.cc:479) meta server chunk server listener:  4000010-13-2016 02:18:26.081 INFO - (metaserver_main.cc:488) path->fid cache disabled10-13-2016 02:18:26.081 INFO - (metaserver_main.cc:351) min chunk servers that should connect: 110-13-2016 02:18:26.081 INFO - (metaserver_main.cc:359) min. # of replicas per file: 110-13-2016 02:18:26.081 INFO - (metaserver_main.cc:520) hard limits: open files: 600000 chunk servers: 4096 clients: 59564810-13-2016 02:18:26.081 INFO - (LayoutManager.cc:1728) rack: prefix:  id: 1010-13-2016 02:18:26.081 INFO - (LayoutManager.cc:2060) max. response size: 268435456 minIoBufferBytesToProcessRequest: 27892121610-13-2016 02:18:26.081 INFO - (metaserver_main.cc:541) starting metaserver10-13-2016 02:18:26.082 INFO - (Restorer.cc:99) restoring from checkpoint of 2016-10-12T18:15:47.174078Z10-13-2016 02:18:26.082 INFO - (Replay.cc:73) open log file: /your-path/qfs_logs/log.010-13-2016 02:18:26.082 INFO - (Restorer.cc:99) restoring from checkpoint of 2016-10-12T18:15:47.174088Z10-13-2016 02:18:26.082 INFO - (metaserver_main.cc:729) replaying logs10-13-2016 02:18:26.082 INFO - (Replay.cc:73) open log file: /your-path/qfs_logs/log.010-13-2016 02:18:26.082 INFO - (Replay.cc:750) log time: 2016-10-12T18:15:47.174020Z10-13-2016 02:18:26.082 INFO - (metaserver_main.cc:747) updating space utilization10-13-2016 02:18:26.082 INFO - (Logger.cc:124) log append: int base: 16 file: /your-path/qfs_logs/log.010-13-2016 02:18:26.082 INFO - (metaserver_main.cc:548) start servicing10-13-2016 02:18:27.083 INFO - (NetDispatch.cc:667) ===request=counters: 1476296307083673 9998 3999 1 0 9 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 9 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 010-13-2016 02:18:27.083 INFO - (NetDispatch.cc:684) ===rusage=self: utime: 9998 stime: 3999 maxrss: 6444 ixrss: 0 idrss: 0 isrss: 0 minflt: 1716 majflt: 0 inblock: 0 oublock: 16 msgsnd: 0 msgrcv: 0 nsignals: 0 nvscw: 18 nivscw: 4 10-13-2016 02:18:27.083 INFO - (NetDispatch.cc:689) ===rusage=chidren: utime: 0 stime: 0 maxrss: 0 ixrss: 0 idrss: 0 isrss: 0 minflt: 0 majflt: 0 inblock: 0 oublock: 0 msgsnd: 0 msgrcv: 0 nsignals: 0 nvscw: 0 nivscw: 0 10-13-2016 02:18:27.083 INFO - (LayoutManager.cc:9391) Initiating a replication check of all chunks

服务器启动了。现在可以修改脚本，加上nohup，保证一直运行。

#!/bin/bash#-c can be used once, do not use it when have impoartand data in cluster#/your-path/qfs/bin/metaserver -c /your-path/qfs_conf/meta_server.prp nohup /your-path/qfs/bin/metaserver /your-path/qfs_conf/meta_server.prp &

注意观察nohup.out的内容，都是INFO就算正常。

chunk server的事下篇再说吧。

官方文档

https://github.com/quantcast/qfs/wiki/Deployment-Guide

参考meta server章节