Ubuntu12.04系统中安装配置OpenStackFolsom云平台（GRE模式）

作者：mobiledu2502872577 | 来源：互联网 | 2014-05-27 20:14

OpenstackFolsom安装比较复杂，尤其是Quantum部分，新的内容很多。Quantum的租户网络有两种模式：GRE和VLAN模式，这两种方式配置有很大的区别，一个明显的区别就是控制节点，Vlan模式2块网卡，GRE模式需要3块网卡。这篇文档就是采用GRE模式，控制节点需要3

Openstack Folsom 安装比较复杂，尤其是Quantum部分，新的内容很多。Quantum的租户网络有两种模式：GRE和VLAN模式，这两种方式配置有很大的区别，一个明显的区别就是控制节点，Vlan模式2块网卡，GRE模式需要3块网卡。这篇文档就是采用GRE模式，控制节点需要3块网卡

英文原文

https://github.com/jedipunkz/openstack_folsom_deploy

https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/blob/stable/GRE/OpenStack_Folsom_Install_Guide_WebVersion.rst

我基本会参考原文做翻译，下面的和原文差异的地方

我的网络会会原文不同, 文档我会在真实的环境下验证，由于每个人的网络基本都是不一样，所以文档和IP相关的地方，我基本采用变量，比较灵活的方式，大家可以用sed命令实现修改。很多朋友安装不成功，基本都是因为更换IP，导致有地方没修改，所以提供sed的命令修改。

对原文提供的两个keystone导入数据的脚本做了细微的修改，主要是采用变量，让他更加灵活。

mysql直接采用IP访问，而不是localhost

keystone的token采用随机生成，而不是password

文档修改记录

2012年9月8日，文档还在草稿中。

2012年9月11日：完成文档大部分内容，目前quantum的安装包有冲突。等待上游修复。

2012年9月21日：基本完成控制节点的安装，登陆dashboard，创建网络。目前dashboard还是需要http:/ip/horizon 访问。

2012年10月11日: 把控制节点安装好。目前Folsom的源，算是已经正式发布。已经成功登陆dashbaord。

2012年10月12日：加上计算节点，不过目前还是有问题。看不到新加入的计算节点。

2012年10月15日: 基本调试通过.成功创建了第一个虚拟机,不过目前网络还是不通,无法访问.vnc还是有问题.

2012年10月16日：根据Essex版本的vnc设置进行调整，目前vnc已经可以工作。已经给作者反馈。目前发现metadata不工作，密钥没有注入到虚拟机里。调整了一下nova.conf 文件。

2012年11月2日：目前quantum的网络还是无法正常工作。需要更多的时间去了解和学习。

2012年11月27日：终于可以实现访问虚拟机，不过目前虚拟机还无法访问外网。正在调试中，希望这个星期能完成整个文档。

2012年11月29日：经过多次重复安装，基本已经实现虚拟机的访问。不过目前虚拟机还是无法访问外部网络，估计还是quantum的bug，今天也是Folsom发布第一个补丁包，希望可以在ubuntu集成补丁包后，修复所有相关的bug。目前文档已经基本可用。

2013年1月5日：作者的原文也做了很多调整，我根据调整也校对了一遍文档，改进了一些地方。目前原文已经把控制节点和网络节点分开，这样更有利于理解，不过调整太大，我就不修改。目前就剩下一个主要问题，虚拟机无法访问外部的网络。

2013年1月17日:源已经更新到 Folsom 2012.2.1, 发现确实修复了几个明显的bug,也顺便调整了一下文档. 不过虚拟机无法访问外网的问题,还是没有解决.这个确实很郁闷.

	控制节点（3块网卡）	计算节点（2块网卡）
管理网络（eth0）	10.1.199.53/24	10.1.199.6/24
VMs Networks with OVS in tunnel mode	10.0.0.3/24	10.0.0.4/24
Public Bridge	不需要设置IP
hostname	controller	compute1
服务	MySQL RabbitMQ Nova Glance Keystone Quantum	kvm quantum nova-compute

要求

控制节点一定需要3块网卡，计算节点2块网卡，如果测试迁移，那么需要2台计算节点

机器支持kvm，可以通过运行命令 kvm-ok 检测是否支持

全部的命令都是在root下运行

操作系统

安装ubuntu 12.04.1 Server版本，最小化安装，只需要安装SSH server就可以。Cinder 需要一个单独的分区或者硬盘。

源

目前Folsom进入ubuntu 12.04的官方的源，不过需要你手工添加。源的官方说明

cat <>/etc/apt/sources.list
deb http: "rem">//ubuntu-cloud.archive.canonical.com/ubuntu precise-proposed/folsom main
deb http: "rem">//ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/folsom main
EOF

运行下面命令

apt-get install ubuntu-cloud-keyring
apt-get update && apt-get -y dist-upgrade
Hostname设置 # cat /etc/hostname
controller
# cat /etc/hosts
127.0.0.1       localhost
10.1.199.53      controller.chenshake.com        controller
10.1.199.6      compute1.chenshake.com  compute1
# hostname
controller
# hostname -f
controller.chenshake.com
网络

直接设置 /etc/network/interface

root@node53:~# cat /etc/network/interfaces
# This file describes network interfaces avaiulable on your system
# and how to activate them. For more information, see interfaces(5).
# Modified by convert_static.sh.
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 10.1.199.53
hwaddress ether 00:25:90:2d:7a:42
netmask 255.255.255.0
network 10.1.199.0
gateway 10.1.199.1
dns-search chenshake.com
dns-nameservers 8.8.8.8
# VMs Networks with OVS in tunnel mode
auto eth1
    iface eth1 inet static
    address 10.0.0.3
    netmask 255.255.255.0
# Public Bridge
auto eth2
    iface eth2 inet manual
    up ifconfig $IFACE 0.0.0.0 up
    up ip link set $IFACE promisc on
    down ip link set $IFACE promisc off
    down ifconfig $IFACE down

重启服务

/etc/init.d/networking restart

设置IP转发

sed -i -r "str">'s/^\s*#(net\.ipv4\.ip_forward=1.*)/\1/' /etc/sysctl.conf
echo 1 > /proc/sys/net/ipv4/ip_forward

检查修改结果

# sysctl -p
net.ipv4.ip_forward = 1

修改完这些，重启机器

查看当前机器的路由

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
"kwrd">default         10.1.199.1      0.0.0.0         UG    100    0        0 eth0
10.0.0.0        *               255.255.255.0   U     0      0        0 eth1
10.1.199.0      *               255.255.255.0   U     0      0        0 eth0
NTP服务器

编辑 /etc/ntp.conf ，在 server ntp.ubuntu.com 下添加两行

server ntp.ubuntu.com
server 127.127.1.0
fudge 127.127.1.0 stratum 10

或者直接运行下面命令

sed -i 's/server ntp.ubuntu.com/server ntp.ubuntu.com\nserver 127.127.1.0\nfudge 127.127.1.0 stratum 10/g' /etc/ntp.conf

重启NTP服务

service ntp restart
环境变量
cat >/root/novarc < export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=password
export MYSQL_PASS=password
export SERVICE_PASSWORD=password
export RABBIT_PASSWORD=password
export FIXED_RANGE=10.0.0.0/24
export FLOATING_RANGE=$(/sbin/ifconfig eth0 | awk "str">'/inet addr/ {print $2}' | cut -f2 -d "str">":" | awk -F "." "str">'{print $1"."$2"."$3}').224/27
export OS_AUTH_URL= "str">"http://localhost:5000/v2.0/"
export SERVICE_ENDPOINT= "str">"http://localhost:35357/v2.0"
export SERVICE_TOKEN=$(openssl rand -hex 10)
export MASTER= "str">"$(/sbin/ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")"
export LOCAL_IP= "str">"$(/sbin/ifconfig eth1 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")"
export OS_TEST_TENANT=bank
export OS_TEST_USER=chenshake
export OS_TEST_NET=bank_net
export OS_TEST_ROUTER=bank_router
export OS_TEST_SUBNET=10.10.10.0/24
EOF

你可以根据你的需要调整用户的密码。

source novarc
echo "source novarc">>.bashrc
Mysql

下面是我们需要用到的数据库

数据库	用户	密码
mysql	root	password
nova	nova	password
keystone	keystone	password
glance	glance	password
cinder	cinder	password
quantum	quantum	password

安装

设置自动安装，无需输入密码

cat < mysql-server-5.5 mysql-server/root_password password $MYSQL_PASS
mysql-server-5.5 mysql-server/root_password_again password $MYSQL_PASS
mysql-server-5.5 mysql-server/start_on_boot boolean "kwrd">true
MYSQL_PRESEED

安装mysql

apt-get -y install mysql-server python-mysqldb curl
设置

运行远程访问mysql

sed -i "str">'s/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf

重启服务

service mysql restart
创建数据库
mysql -uroot -p$MYSQL_PASS < CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO "str">'nova'@ "str">'%' IDENTIFIED BY "str">'$MYSQL_PASS';
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO "str">'glance'@ "str">'%' IDENTIFIED BY "str">'$MYSQL_PASS';
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO "str">'keystone'@ "str">'%'IDENTIFIED BY "str">'$MYSQL_PASS';
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO "str">'cinder'@ "str">'%'IDENTIFIED BY "str">'$MYSQL_PASS';
CREATE DATABASE quantum;
GRANT ALL PRIVILEGES ON quantum.* TO "str">'quantum'@ "str">'%'IDENTIFIED BY "str">'$MYSQL_PASS';
FLUSH PRIVILEGES;
EOF
RabbitMQ 安装
apt-get -y install rabbitmq-server
设置

修改默认密码

我们把默认密码 guest，改成password

rabbitmqctl change_password guest $RABBIT_PASSWORD
Keystone 安装
apt-get -y install keystone
配置

编辑 /etc/keystone/keystone.conf

[DEFAULT]
admin_token = d111cf2d97251a9e0422
bind_host = 0.0.0.0
public_port = 5000
admin_port = 35357
compute_port = 8774
verbose = True
debug = True
log_file = keystone.log
log_dir = /var/log/keystone
log_config = /etc/keystone/logging.conf
[sql]
connection = mysql: "rem">//keystone:password@10.1.199.53:3306/keystone
idle_timeout = 200

或者直接运行下面脚本

sed -i -e "str">" s/# admin_token = ADMIN/admin_token = $SERVICE_TOKEN/g; s/# bind_host = 0.0.0.0/bind_host = 0.0.0.0/g; s/# public_port = 5000/public_port = 5000/g; s/# admin_port = 35357/admin_port = 35357/g; s/# compute_port = 8774/compute_port = 8774/g; s/# verbose = True/verbose = True/g; s/# idle_timeout/idle_timeout/g" /etc/keystone/keystone.conf

使用mysql数据库

sed -i '/connection = .*/{s|sqlite:///.*|mysql://'"keystone"':'"$MYSQL_PASS"'@'"$MASTER"'/keystone|g}' /etc/keystone/keystone.conf

重启服务和初始化数据库

service keystone restart
keystone-manage db_sync

导入keystone数据

keystone-data.sh

wget http: "rem">//www.chenshake.com/wp-content/uploads/2012/09/keystone-data.sh_.txt
mv keystone-data.sh_.txt keystone-data.sh
bash keystone-data.sh

导入endpoint

keystone-endpoints.sh

wget http: "rem">//www.chenshake.com/wp-content/uploads/2012/09/keystone-endpoints.sh_.txt
mv keystone-endpoints.sh_.txt keystone-endpoints.sh
bash keystone-endpoints.sh

测试

使用curl测试

curl -d "str">'{"auth": {"tenantName": "admin", "passwordCredentials":{"username": "admin", "password": "password"}}}' -H "Content-type:application/json" http://$MASTER:35357/v2.0/tokens | python -mjson.tool

查看log

grep ERROR /var/log/keystone/keystone.log
ps -ef | grep -i keystone-all
Glance 安装
apt-get -y install glance
配置

编辑/etc/glance/glance-api.conf 和 /etc/glance/glance-registry.conf ,两个文件，都是修改4个地方

sql_connection = mysql: "rem">//glance:password@10.1.199.53/glance
admin_tenant_name = service
admin_user = glance
admin_password = password

或者直接运行下面脚本实现

sed -i -e "str">" s/%SERVICE_TENANT_NAME%/service/g; s/%SERVICE_USER%/glance/g; s/%SERVICE_PASSWORD%/$SERVICE_PASSWORD/g; " /etc/glance/glance-api.conf /etc/glance/glance-registry.conf
sed -i "str">'/sql_connection = .*/{s|sqlite:///.*|mysql://'"glance"':'"$MYSQL_PASS"'@'"$MASTER"'/glance|g}' /etc/glance/glance-registry.conf /etc/glance/glance-api.conf

编辑 /etc/glance/glance-api.conf

#notifier_strategy = noop
notifier_strategy = rabbit
#rabbit_password = guest
rabbit_password = password

运行下面命令进行修改

sed -i "str">" s/notifier_strategy = noop/notifier_strategy = rabbit/g;s/rabbit_password = guest/rabbit_password = $RABBIT_PASSWORD/g;" /etc/glance/glance-api.conf

运行下面命令

cat <>/etc/glance/glance-api.conf
flavor = keystone+cachemanagement
EOF
cat <>/etc/glance/glance-registry.conf
flavor = keystone
EOF

重启服务

service glance-api restart && service glance-registry restart

同步数据库

glance-manage db_sync

下载Image

我们下载CirrOS的image作为测试使用，只有10M。如果是ubuntu官方的image，220M，并且ubuntu官方的image，都是需要使用密钥登陆。

CirrOS

下载image

wget https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img

上传image

glance image-create --name=cirros-0.3.0-x86_64 -- "kwrd">public --container-format=bare \
--disk-format=qcow2 < /root/cirros-0.3.0-x86_64-disk.img

Cirros，是可以使用用户名和密码登陆，也可以使用密钥登陆

user:cirros
password:cubswin:)

Ubuntu官方image

下载image

wget http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img

上传image

glance image-create --name= "str">"Ubuntu 12.04 cloudimg amd64" -- "kwrd">public \
--container-format=ovf --disk-format=qcow2 < /root/precise-server-cloudimg-amd64-disk1.img

user：ubuntu

只能使用密钥登陆。

测试

查看image

glance image-list

查看image详细信息

glance image-show 12e2b864-9601-4506-b19d-3f663c0b2e15
Open-vSwitch 安装
apt-get install -y openvswitch-switch
配置

设置网络

ovs-vsctl add-br br-int
ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex eth2
ip link set up br-ex

大家可以通过下面命令来查看你创建的效果, 具体的用途,正在学习中.

ovs-vsct -h
ovs-vsctl list-br
ovs-vsctl show

查看结果

# ovs-vsctl list-br
br-ex
br-int
# ovs-vsctl show
89742cb3-5d15-4150-a278-a4054ab9c219
    Bridge br-int
        Port br-int
            Interface br-int
                type: internal
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "eth2"
            Interface "eth2"
    ovs_version: "1.4.0+build0"
Quantum 安装
apt-get -y install quantum-server python-cliff \
quantum-plugin-openvswitch-agent \
quantum-l3-agent quantum-dhcp-agent python-pyparsing
配置

编辑 /etc/quantum/quantum.conf

auth_strategy = keystone
fake_rabbit = False
rabbit_host = 10.1.199.53
rabbit_password = password

或者运行下面命令

sed -i -e "str">" s/# auth_strategy/auth_strategy/g; s/# fake_rabbit/fake_rabbit/g; s/# rabbit_host = localhost/rabbit_host = $MASTER/g; s/# rabbit_password = guest/rabbit_password = $RABBIT_PASSWORD/g" /etc/quantum/quantum.conf

编辑 /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

sql_connection = mysql: "rem">//quantum:password@10.1.199.53:3306/quantum
[OVS]
tenant_network_type = gre
enable_tunneling = True
tunnel_id_ranges = 1:1000
integration_bridge = br-int
tunnel_bridge = br-tun
local_ip = 10.0.0.3

或者运行下面命令

sed -i -e "str">" s/# Example: tenant_network_type = gre/tenant_network_type = gre/g; s/# Default: enable_tunneling = False/enable_tunneling = True/g; s/# Example: tunnel_id_ranges = 1:1000/tunnel_id_ranges = 1:1000/g; s/# Default: integration_bridge = br-int/integration_bridge = br-int/g; s/# Default: tunnel_bridge = br-tun/tunnel_bridge = br-tun/g; s/# Default: local_ip =/local_ip = $LOCAL_IP/g" /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

修改数据库

sed -i "str">'/sql_connection = .*/{s|sqlite:///.*|mysql://'"quantum"':'"password"'@'"$MASTER"'/quantum|g}' /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

编辑 /etc/quantum/l3_agent.ini 和 /etc/quantum/api-paste.ini

[DEFAULT]
admin_tenant_name = service
admin_user = quantum
admin_password = password

或者运行下面命令

sed -i -e "str">" s/%SERVICE_TENANT_NAME%/service/g; s/%SERVICE_USER%/quantum/g; s/%SERVICE_PASSWORD%/$SERVICE_PASSWORD/g; " /etc/quantum/l3_agent.ini
sed -i -e "str">" s/%SERVICE_TENANT_NAME%/service/g; s/%SERVICE_USER%/quantum/g; s/%SERVICE_PASSWORD%/$SERVICE_PASSWORD/g; " /etc/quantum/api-paste.ini

编辑 /etc/quantum/l3_agent.ini

debug = True
use_namespaces = False
metadata_ip = 10.1.199.53

或者运行下面命令

sed -i -e "str">" s/# debug = True/debug = True/g; s/# use_namespaces = True/use_namespaces = False/g; s/# metadata_ip =/metadata_ip = $MASTER/g" /etc/quantum/l3_agent.ini

编辑 /etc/quantum/dhcp_agent.ini

use_namespaces = False

或者运行命令

sed -i -e "str">" s/# use_namespaces = True/use_namespaces = False/g; " /etc/quantum/dhcp_agent.ini

重启服务

service quantum-server restart
service quantum-plugin-openvswitch-agent restart
service quantum-dhcp-agent restart
service quantum-l3-agent restart

对于Openstack别的组件，都是需要创建数据库的过程。不过quantum，你只需要重启quantum-server服务，他就会自动创建相关的表。

为demo用户创建网络

利用提供的脚本，为用户 Demo创建一个网络

wget http: "rem">//www.chenshake.com/wp-content/uploads/2012/09/quantum-networking.sh_.txt
mv quantum-networking.sh_.txt quantum-networking.sh

需要对脚本进行一些修改。

##############################################################
### Public Network ###########################################
##############################################################
# Provider Router Information - what name should
# this provider have "kwrd">in Quantum?
PROV_ROUTER_NAME="provider-router"
# Name of External Network (Don't change it!)
EXT_NET_NAME="ext_net"
# External Network addressing - our official
# Internet IP address space
EXT_NET_CIDR="10.1.199.0/24"
EXT_NET_LEN=${EXT_NET_CIDR#*/}
# External bridge that we have configured
# into l3_agent.ini (Don't change it!)
EXT_NET_BRIDGE=br-ex
# IP of external bridge (br-ex) - "kwrd">this node's
# IP "kwrd">in our official Internet IP address space:
EXT_GW_IP="10.1.199.13"
# IP of the Public Network Gateway - The
# default GW "kwrd">in our official Internet IP address space:
EXT_NET_GATEWAY="10.1.199.1"
# Floating IP range
POOL_FLOATING_START= "str">"10.1.199.130" # First "kwrd">public IP to be used "kwrd">for VMs
POOL_FLOATING_END= "str">"10.1.199.150" # Last "kwrd">public IP to be used "kwrd">for VMs
###############################################################

上面最让人困惑的设置就是：EXT_GW_IP, 这其实是控制节点的eth2的IP地址，不过这个IP地址，不是通过/etc/network/interface 设置，而是通过这个脚本设置。运行完脚本，你就可以ping通这个IP。

改完后，运行脚本

root@node53:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
"kwrd">default         10.1.199.1      0.0.0.0         UG    100    0        0 eth0
10.0.0.0        *               255.255.255.0   U     0      0        0 eth1
10.1.199.0      *               255.255.255.0   U     0      0        0 eth0
root@node53:~# bash quantum-networking.sh
Added "kwrd">interface to router f69ecf3d-d476-433a-82a6-de20614b9d32
Created a new subnet:
+------------------+--------------------------------------------------+
| Field            | Value                                            |
+------------------+--------------------------------------------------+
| allocation_pools | { "str">"start": "10.1.199.130", "end": "10.1.199.150"} |
| cidr             | 10.1.199.0/24                                    |
| dns_nameservers  |                                                  |
| enable_dhcp      | False                                            |
| gateway_ip       | 10.1.199.1                                       |
| host_routes      |                                                  |
| id               | ef65a5bd-39d2-496f-a042-7234b5b8956e             |
| ip_version       | 4                                                |
| name             |                                                  |
| network_id       | 92c05466-6f80-4e5f-bbc3-59987df8d489             |
| tenant_id        | ab38cf34ab0a4a9995c84a53044a2269                 |
+------------------+--------------------------------------------------+
Set gateway "kwrd">for router f69ecf3d-d476-433a-82a6-de20614b9d32
root@node53:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
"kwrd">default         10.1.199.1      0.0.0.0         UG    100    0        0 eth0
10.0.0.0        *               255.255.255.0   U     0      0        0 eth1
10.1.199.0      *               255.255.255.0   U     0      0        0 eth0
10.1.199.0      *               255.255.255.0   U     0      0        0 br-ex
10.5.5.0        *               255.255.255.0   U     0      0        0 tap218751e0-6d

修改/etc/quantum/l3_agent.ini，修改 router 和 external network. 我直接使用下面命令修改。

router=$(quantum router-list | awk "str">'/provider-router/ {print $2}')
ext_net=$(quantum net-list | awk "str">'/ext_net/ {print $2}')
sed -i -e "str">" s/# router_id =/router_id = $router/g; s/# gateway_external_network_id =/gateway_external_network_id = $ext_net/g;" /etc/quantum/l3_agent.ini

这个时候，需要重启l3agent服务

service quantum-l3-agent restart

这时候你查看路由表

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
"kwrd">default         10.1.199.1      0.0.0.0         UG    0      0        0 eth0
"kwrd">default         10.1.199.1      0.0.0.0         UG    100    0        0 eth0
10.0.0.0        *               255.255.255.0   U     0      0        0 eth1
10.1.199.0      *               255.255.255.0   U     0      0        0 eth0
10.1.199.0      *               255.255.255.0   U     0      0        0 br-ex
10.1.199.0      *               255.255.255.0   U     0      0        0 qg-fba8f518-45
10.5.5.0        *               255.255.255.0   U     0      0        0 tap218751e0-6d
10.5.5.0        *               255.255.255.0   U     0      0        0 qr-61f55d19-9e

查看IP

可以看到很多信息。

root@node53:~# ip addr show
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:25:90:2d:7a:18 brd ff:ff:ff:ff:ff:ff
    inet 10.1.199.53/24 brd 10.1.199.255 scope global eth0
    inet6 fe80::225:90ff:fe2d:7a18/64 scope link
       valid_lft forever preferred_lft forever
3: eth1:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:25:90:2d:7a:19 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.3/24 brd 10.0.0.255 scope global eth1
    inet6 fe80::225:90ff:fe2d:7a19/64 scope link
       valid_lft forever preferred_lft forever
4: eth2:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:25:90:3b:23:c8 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::225:90ff:fe3b:23c8/64 scope link
       valid_lft forever preferred_lft forever
5: eth3:  mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 00:25:90:3b:23:c9 brd ff:ff:ff:ff:ff:ff
6: br- "kwrd">int:  mtu 1500 qdisc noop state DOWN
    link/ether ba:03:3e:6c:9b:42 brd ff:ff:ff:ff:ff:ff
7: br-ex:  mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 00:25:90:3b:23:c8 brd ff:ff:ff:ff:ff:ff
    inet 10.1.199.13/24 scope global br-ex
8: br-tun:  mtu 1500 qdisc noop state DOWN
    link/ether ba:43:40:98:d2:4a brd ff:ff:ff:ff:ff:ff
9: tapd84b2276-bc:  mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:a2:88:0e brd ff:ff:ff:ff:ff:ff
    inet 10.5.5.2/24 brd 10.5.5.255 scope global tapd84b2276-bc
    inet6 fe80::f816:3eff:fea2:880e/64 scope link
       valid_lft forever preferred_lft forever
10: qr-7afa9a7d-be:  mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:8e:9c:48 brd ff:ff:ff:ff:ff:ff
    inet 10.5.5.1/24 brd 10.5.5.255 scope global qr-7afa9a7d-be
    inet6 fe80::f816:3eff:fe8e:9c48/64 scope link
       valid_lft forever preferred_lft forever
11: qg-2a8f838e-06:  mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:b9:75:b8 brd ff:ff:ff:ff:ff:ff
    inet 10.1.199.130/24 brd 10.1.199.255 scope global qg-2a8f838e-06
    inet6 fe80::f816:3eff:feb9:75b8/64 scope link
       valid_lft forever preferred_lft forever
Cinder 安装
apt-get install -y cinder-api cinder-scheduler cinder-volume iscsitarget \
open-iscsi iscsitarget-dkms python-cinderclient
配置

分区

我的硬盘专门一个分区给volume使用

umount /dev/sda5
pvcreate /dev/sda5
vgcreate cinder-volumes /dev/sda5

去掉开机挂载

sed -i '/nova-volume/s/^/#/' /etc/fstab

iscsi

sed -i 's/false/true/g' /etc/ "kwrd">default/iscsitarget
service iscsitarget restart
service open-iscsi restart

编辑 /etc/cinder/cinder.conf ,直接运行下面命令就可以.

cat >/etc/cinder/cinder.conf < [DEFAULT]
rootwrap_config = /etc/cinder/rootwrap.conf
sql_connection = mysql: "rem">//cinder:$MYSQL_PASS@$MASTER:3306/cinder
iscsi_helper = ietadm
volume_group = cinder-volumes
rabbit_password= $RABBIT_PASSWORD
logdir=/var/log/cinder
verbose=true
auth_strategy = keystone
EOF

编辑 /etc/cinder/api-paste.ini

admin_tenant_name = service
admin_user = cinder
admin_password = password

或者用下面命令

sed -i -e "str">" s/%SERVICE_TENANT_NAME%/service/g; s/%SERVICE_USER%/cinder/g; s/%SERVICE_PASSWORD%/$SERVICE_PASSWORD/g; " /etc/cinder/api-paste.ini

同步数据库

cinder-manage db sync

重启服务

service cinder-api restart
service cinder-scheduler restart
service cinder-volume restart
Nova 安装
apt-get -y install nova-api nova-cert nova-common \
nova-scheduler python-nova python-novaclient nova-consoleauth novnc nova-novncproxy
配置

编辑 /etc/nova/api-paste.ini

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
auth_host = 10.1.199.53
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = password
signing_dirname = /tmp/keystone-signing-nova

或者直接运行命令

sed -i -e "str">" s/127.0.0.1/$MASTER/g; s/%SERVICE_TENANT_NAME%/service/g; s/%SERVICE_USER%/nova/g; s/%SERVICE_PASSWORD%/$SERVICE_PASSWORD/g; " /etc/nova/api-paste.ini

创建 /etc/nova/nova.conf 文件，直接copy下面的命令，运行就可以。

cat >/etc/nova/nova.conf < [DEFAULT]
# MySQL Connection #
sql_cOnnection=mysql: "rem">//nova:$MYSQL_PASS@$MASTER/nova
# nova-scheduler #
rabbit_host=$MASTER
rabbit_password=$RABBIT_PASSWORD
scheduler_driver=nova.scheduler.simple.SimpleScheduler
#compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
# nova-api #
cc_host=$MASTER
auth_strategy=keystone
s3_host=$MASTER
ec2_host=$MASTER
nova_url=http://$MASTER:8774/v1.1/
ec2_url=http://$MASTER:8773/services/Cloud
keystone_ec2_url=http: "rem">//$MASTER:5000/v2.0/ec2tokens
api_paste_cOnfig=/etc/nova/api-paste.ini
allow_admin_api=true
use_deprecated_auth=false
ec2_private_dns_show_ip=True
dmz_cidr=169.254.169.254/32
ec2_dmz_host=169.254.169.254
metadata_host=$MASTER
metadata_listen=0.0.0.0
enabled_apis=ec2,osapi_compute,metadata
# Networking #
network_api_class=nova.network.quantumv2.api.API
quantum_url=http://$MASTER:9696
quantum_auth_strategy=keystone
quantum_admin_tenant_name=service
quantum_admin_username=quantum
quantum_admin_password=$SERVICE_PASSWORD
quantum_admin_auth_url=http: "rem">//$MASTER:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
# Compute #
#compute_driver=libvirt.LibvirtDriver
# Cinder #
volume_api_class=nova.volume.cinder.API
# Glance #
glance_api_servers=$MASTER:9292
image_service=nova.image.glance.GlanceImageService
# novnc #
novnc_enable=true
novncproxy_base_url=http: "rem">//$MASTER:6080/vnc_auto.html
vncserver_proxyclient_address=$MASTER
vncserver_listen=$MASTER
# Misc #
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
#root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
rootwrap_cOnfig=/etc/nova/rootwrap.conf
#verbose=true
verbose=false
EOF

同步数据库

nova-manage db sync

重启服务

service nova-api restart
service nova-cert restart
service nova-consoleauth restart
service nova-scheduler restart
service nova-novncproxy restart
Horizon 安装
apt-get -y install apache2 libapache2-mod-wsgi openstack-dashboard memcached python-memcache

删除ubuntu自带的theme 编辑 /etc/openstack-dashboard/local_settings.py

#Comment these lines
#Enable the Ubuntu theme "kwrd">if it is present.
#try:
# from ubuntu_theme import *
#except ImportError:
# pass

或者运行下面命令

sed -i "str">'150,153s/^/#/' /etc/openstack-dashboard/local_settings.py

Reload 服务

service apache2 restart; service memcached restart

访问

http://10.1.199.53/horizon
user：admin
pass：password
或者
user:demo
pass:password

看一下中文的Dashboard，由于在控制节点没有安装计算服务，所以你是无法创建虚拟机。

操作系统

操作系统最小化安装，ssh server就可以。

源

添加Folsom源

运行下面命令

apt-get install ubuntu-cloud-keyring
apt-get update && apt-get -y dist-upgrade
网络
# cat /etc/network/interfaces
# This file describes network interfaces avaiulable on your system
# and how to activate them. For more information, see interfaces(5).
# Modified by convert_static.sh.
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 10.1.199.6
hwaddress ether 00:25:90:2d:7a:42
netmask 255.255.255.0
network 10.1.199.0
gateway 10.1.199.1
dns-search chenshake.com
dns-nameservers 8.8.8.8
# VMs Networks with OVS in tunnel mode
auto eth1
    iface eth1 inet static
    address 10.0.0.4
    netmask 255.255.255.0

重启网络

/etc/init.d/networking restart

IP转发

sed -i -r "str">'s/^\s*#(net\.ipv4\.ip_forward=1.*)/\1/' /etc/sysctl.conf
echo 1 > /proc/sys/net/ipv4/ip_forward
环境变量
cat >/root/novarc < export CONTROLLER_IP=10.1.199.53
export MASTER= "str">"$(/sbin/ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")"
export LOCAL_IP= "str">"$(/sbin/ifconfig eth1 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")"
EOF

你根据你的情况，调整控制节点的IP

source novarc
echo "source novarc">>.bashrc
NTP
apt-get -y install ntp

设置

编辑 /etc/ntp.conf, 指向控制节点

server 10.1.199.53

或者运行命令

sed -i -e "str">" s/server ntp.ubuntu.com/server $CONTROLLER_IP/g" /etc/ntp.conf

重启服务

service ntp restart
Hypervisor
apt-get install -y kvm libvirt-bin pm-utils

编辑 /etc/libvirt/qemu.conf ，添加下面内容

cgroup_device_acl = [
    "/dev/null", "str">"/dev/full", "/dev/zero",
    "/dev/random", "str">"/dev/urandom",
    "/dev/ptmx", "str">"/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "str">"/dev/hpet","/dev/net/tun",
]

或者运行命令：这个地方用命令修改有点复杂，还没找到太好的办法。

cat <>/etc/libvirt/qemu.conf
cgroup_device_acl = [
    "/dev/null", "str">"/dev/full", "/dev/zero",
    "/dev/random", "str">"/dev/urandom",
    "/dev/ptmx", "str">"/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "str">"/dev/hpet","/dev/net/tun",
]
EOF

删除默认 virtual bridge

virsh net-destroy default
virsh net-undefine default

允许迁移

编辑 /etc/libvirt/libvirtd.conf, 去掉这三行的注释

listen_tls = 0
listen_tcp = 1
auth_tcp = "none"

或者运行下面命令

sed -i "str">'/#listen_tls/s/#listen_tls/listen_tls/; /#listen_tcp/s/#listen_tcp/listen_tcp/; /#auth_tcp/s/#auth_tcp/auth_tcp/; /auth_tcp/s/sasl/none/' /etc/libvirt/libvirtd.conf

编辑 /etc/init/libvirt-bin.conf

env libvirtd_opts="-d -l"

或者使用命令

sed -i "str">'/env libvirtd_opts/s/-d/-d -l/' /etc/init/libvirt-bin.conf

编辑 /etc/default/libvirt-bin

libvirtd_opts="-d -l"

或者使用命令

sed -i "str">'/libvirtd_opts/s/-d/-d -l/' /etc/ "kwrd">default/libvirt-bin

重启服务

service libvirt-bin restart
Open-vSwitch
apt-get install -y openvswitch-switch

配置bridge

ovs-vsctl add-br br-int
Quantum
apt-get -y install quantum-plugin-openvswitch-agent

编辑 /etc/quantum/quantum.conf , 修改和控制节点一样，直接从控制直接复制过来

scp root@$CONTROLLER_IP:/etc/quantum/quantum.conf /etc/quantum/quantum.conf

编辑 /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini，

可以从控制节点copy过来,只需要修改local_IP就可以.

scp root@$CONTROLLER_IP:/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini

修改 local_ip

sed -i "str">'s/^local_ip.*$/local_ip = '$LOCAL_IP "str">'/g' /etc/quantum/plugins/openvswitch//ovs_quantum_plugin.ini

重启服务

service openvswitch-switch restart
service quantum-plugin-openvswitch-agent restart
Nova
apt-get -y install nova-compute-kvm novnc nova-novncproxy nova-api-metadata

编辑 /etc/nova/api-paste.ini

或者运行下面命令,直接从控制节点复制过来就可以。

scp root@$CONTROLLER_IP:/etc/nova/api-paste.ini /etc/nova/

编辑 /etc/nova/nova-compute.conf

[DEFAULT]
libvirt_type=kvm
libvirt_ovs_bridge=br-int
libvirt_vif_type=ethernet
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
libvirt_use_virtio_for_bridges=True

或者运行下面命令

cat > /etc/nova/nova-compute.conf < [DEFAULT]
libvirt_type=kvm
libvirt_ovs_bridge=br-int
libvirt_vif_type=ethernet
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
libvirt_use_virtio_for_bridges=True
EOF

编辑 /etc/nova/nova.conf。我们可以从控制节点copy过来修改

scp root@$CONTROLLER_IP:/etc/nova/nova.conf /etc/nova/nova.conf

修改下面内容

metadata_host=10.1.199.6
enabled_apis=metadata
# Compute #
compute_driver=libvirt.LibvirtDriver
# novnc #
novnc_enable=true
novncproxy_base_url=http: "rem">//10.1.199.53:6080/vnc_auto.html
vncserver_proxyclient_address=10.1.199.6
vncserver_listen=10.1.199.6

可以使用命令

sed -i "str">"/metadata_host/s/$CONTROLLER_IP/$MASTER/; s/^enabled_apis.*$/enabled_apis=metadata/g; s/#compute_driver/compute_driver/g; /vncserver_proxyclient_address/s/$CONTROLLER_IP/$MASTER/; /vncserver_listen/s/$CONTROLLER_IP/$MASTER/" /etc/nova/nova.conf

重启服务

service nova-novncproxy restart
service nova-compute restart
service nova-api-metadata restart

这个时候,你就可以通过命令看到计算节点和控制节点。

快照22

这个操作是控制节点运行, 由于quantum的脚本，已经为demo的用户创建了一个网络，所以我们就直接用demo的用户操作。

cat > /root/demo << EOF
export OS_USERNAME=admin
export OS_TENANT_NAME=demo
export OS_PASSWORD=password
export OS_AUTH_URL=http: "rem">//127.0.0.1:35357/v2.0/
export PS1="[\u@\h \W(demo)]\$ "
EOF

运行

. demo

创建虚拟机，你需要根据情况调整image id

nova keypair-add oskey > oskey.priv
chmod 600 oskey.priv
nova flavor-list
nova image-list
nova boot --flavor 2 --key_name oskey --image ea3ffba1-065e-483f-bfe2-c84184ee76be test1
nova secgroup-add-rule "kwrd">default tcp 22 22 0.0.0.0/0
nova secgroup-add-rule "kwrd">default icmp -1 -1 0.0.0.0/0

这个时候虚拟机就创建好了。查看虚拟机的Ip

这个时候就可以ssh到虚拟机

ssh -i oskey.priv ubuntu@10.5.5.3

设置floating IP