8月14日更新glance配置。
glance 使用swift作为后端配置非常简单
修改/etc/glance/glance-api.conf
#default_store = file
default_store = swift
swift_store_auth_address = http://192.168.4.87:5000/v2.0/
swift_store_user = service:swift
swift_store_key = zhoubing
swift_store_create_container_on_put = True
以上配置需要根据环境的实际情况填写。(配置项在文件中都有,只需要修改,不需要添加).
Swift模块OpenStack Object Storage (Swift) 是用来创建冗余的、可扩展的对象存储(引擎)的开源软件。 swift使用标准化的服务器存储 PB 级的访问数据。它并不是文件系统或者实时的数据存储系统 ,而是一个用于存储永久类型的静态数据的长期存储系统 ,这些数据可以检索、调整、必要时进行更新。最适合存储的数据类型的首要例子是虚拟机镜像,图片存储,邮件存储,存档备份。因为没有中心单元或者主控结点,swift提供了更强的扩展性、冗余和持久性。
使用主机信息:
|----+----------------+-------------+-----------+--------------|
| ID | Name | OS | IP | Used for |
|----+----------------+-------------+-----------+--------------|
| 1 | swift-proxy | ubuntu12.04 | 192.168.4.87 | Proxy Node |
| 2 | swift-storage1 | ubuntu12.04 | 192.168.4.88| Storage Node |
| 3 | swift-storage2 | ubuntu12.04 | 192.168.4.89| Storage Node |
| 4 | swift-storage3 | ubuntu12.04 | 192.168.4.90| Storage Node |
|----+----------------+-------------+-----------+--------------|
1 安装配置存储节点(Storage Node)l 安装需要的软件包
apt-get update
apt-get install swift swift-account swift-container swift-object swift-doc xfsprogs
l 创建loopback设备(假设为sdb1)分出一个分区然后格式化
fdisk /dev/sdb
mkfs.xfs -i size=1024 /dev/sdb1
上面的方法是官方文档上的方法,也是推荐的方法,我使用的是创建回环设备来存
dd if=/dev/zero of=/srv/swift-node bs=1024
count=0 seek=1000000
mkfs.xfs -i size=1024 /srv/swift-node
l 更新fstab文件
cat >> /etc/fstab <
/srv/swift-node /swift/node xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
EOF
l 挂载
mkdir /swift/node
mount /swift/node
chown -R swift:swift /swift 注意
没有赋权限,会导致创建失败
l 创建swift的配置文件
cat >/etc/swift/swift.conf <
[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = `od -t x8 -N 8 -A n
EOF
Note: 这个文件所有的存储节点和代理节点共用,可以用scp拷贝到其它节点
l 安装rsync
apt-get install rsync
l 创建rsync配置文件
cat > /etc/rsyncd.conf << EOF
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 192.168.4.88
[account]
max connections = 2
path = /swift/node
read only = false
lock file = /var/lock/account.lock
[container]
max connections = 2
path = /swift/node
read only = false
lock file = /var/lock/container.lock
[object]
max connections = 2
path = /swift/node
read only = false
lock file = /var/lock/object.lock
EOF
l 启用rsync
perl -pi -e 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/' /etc/default/rsync
l 启动rsync
service rsync start
l 配置account服务
cat > /etc/swift/account-server.conf << EOF
[DEFAULT]
devices = /swift/node
mount_check = false
bind_ip =
bind_port = 6012
workers = 2
log_facility = LOG_LOCAL3
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:swift#account
[account-replicator]
[account-auditor]
[account-reaper]
EOF
l 配置container服务
cat > /etc/swift/container-server.conf << EOF
[DEFAULT]
devices = /swift/node
mount_check = false
bind_ip =
bind_port = 6011
workers = 2
log_facility = LOG_LOCAL2
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:swift#container
[container-replicator]
[container-updater]
[container-auditor]
[container-sync]
EOF
l 配置object服务
cat > /etc/swift/object-server.conf << EOF
[DEFAULT]
devices = /swift/node/
mount_check = false
bind_ip =
bind_port = 6010
workers = 2
log_facility = LOG_LOCAL1
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:swift#object
[object-replicator]
[object-updater]
[object-auditor]
[object-expirer]
EOF
l 创建swift日志配置文件
cat > /etc/rsyslog.d/10-swift.conf << EOF
local1,local2,local3.* /var/log/swift/all.log
local1.* /var/log/swift/object.log
local2.* /var/log/swift/container.log
local3.* /var/log/swift/account.log
EOF
l 创建日志目录
mkdir -p /var/log/swift
chown -R syslog.adm /var/log/swift
l 重启日志服务进程
service rsyslog restart
l 安装需要的软件包
apt-get update
apt-get install swift swift-proxy swift-doc memcached
l 修改memcached配置文件
perl -pi -e "s/-l 127.0.0.1/-l 192.168.4.87/" /etc/memcached.conf
l 重启memcached服务
service memcached restart
l 从存储节点拷贝swift配置文件
mkdir /etc/swift
scp root@192.168.4.88:/etc/swift/swift.conf /etc/swift
l 创建proxy-server配置文件
cat > /etc/swift/proxy-server.conf < EOF
[DEFAULT]
bind_port = 8080
bind_ip = 192.168.4.87
user = swift
log_level = DEBUG
log_facility = LOG_LOCAL1
[pipeline:main]
pipeline = healthcheck cache authtoken keystone proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
[filter:keystone]
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
operator_roles = admin, swiftoperator
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
delay_auth_decision = 1
auth_port = 35357
auth_host = 192.168.4.87
auth_protocol = http
service_protocol = http
service_host = 192.168.4.87
service_port = 5000
admin_tenant_name = service
admin_user = swift
admin_password = zhoubing
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
memcache_servers = 192.168.4.87:11211
EOF
l 创建swift日志配置文件
cat > /etc/rsyslog.d/10-swift.conf << EOF
local1.* /var/log/swift/proxy.log
l 创建日志目录
mkdir -p /var/log/swift
chown -R syslog.adm /var/log/swift
l 重启日志服务进程
service rsyslog restart
l 首先创建环
cd /etc/swift
swift-ring-builder account.builder create 18 2 1
swift-ring-builder container.builder create 18 2 1
swift-ring-builder object.builder create 18 2 1
Note: 后面的数字第一个为存储分区数,第二个为备份次数,备份次数不能小于ZONE数目,第三个为延迟删除的时间,以小时为单位
向环里添加存储设备我前面配置了两台存储节点:
|---------+------+-----------+-----------+------+--------|
| Storage | Zone | IP | Server | Port | Device |
|---------+------+-----------+-----------+------+--------|
| 1 | 1 | 192.168.4.88| account | 6012 | sdb1 |
| | | | container | 6011 | sdb1 |
| | | | object | 6010 | sdb1 |
|---------+------+-----------+-----------+------+--------|
| 2 | 2 | 192.168.4.89 | account | 6012 | sdb1 |
| | | | container | 6011 | sdb1 |
| | | | object | 6010 | sdb1 |
|---------+------+-----------+-----------+------+--------|
cd /etc/swift
swift-ring-builder account.builder add z1-192.168.4.88:6012/sdb1 100
swift-ring-builder container.builder add z1-192.168.4.88:6011/sdb1 100
swift-ring-builder object.builder add z1-192.168.4.88:6010/sdb1 100
swift-ring-builder account.builder add z2-192.168.4.89:6012/sdb1 100
swift-ring-builder container.builder add z2-192.168.4.89:6011/sdb1 100
swift-ring-builder object.builder add z2-192.168.4.89:6010/sdb1 100
Note: IP,端口,设备和前面的配置对应
l 检验每个环里的条目
cd /etc/swift
swift-ring-builder account.builder
swift-ring-builder container.builder
swift-ring-builder object.builder
应该得到类似这样的输出
account.builder, build version 1
262144 partitions, 1 replicas, 1 zones, 1 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
0 1 192.168.4.88 6012 sdb1 100.00 262144 0.00
0 1 192.168.4.89 6012 sdb1 100.00 262144 0.00
l 平衡环
cd /etc/swift
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance
Note: 这个过程可能需要一些时间
Note: 这三条命令会产生以gz结尾的文件,这些文件就是所有节点要用到的环文件,可以用scp拷贝到其它各节点
Note: 如果创建环的时候指定的副本数大于环的数量,平衡环的时候可能会报类似下面的错误
...
4, in _initial_balance
while available_devs[index]['zone'] in other_zones:
IndexError: list index out of range
l 分发新生成的环文件
scp /etc/swift/*.gz root@192.168.4.88:/etc/swift/
scp /etc/swift/*.gz root@192.168.4.89:/etc/swift/
l 启动各节点相关的服务
# 代理节点
swift-init proxy start
# 存储节点
swift-init all start