部署Cloudstack3.0.5+KVM+CentOS6.5云计算平台

来自http://hongxin.blog.51cto.com/2464891/1176018
--基本网络域

规划：Cloudstack 3.0.5+CentOS6.3 x86_64+KVM平台

主机名称	management.example.com	agent-1.example.com
ip地址	192.168.1.250	192.168.1.251
用途	cloud-management+NFS	KVM+Agent

第一部分、cloudstack management 服务器的安装配置

1、CPU虚拟化支持验证（KVM的要求）

[root@management ~]# grep  vmx  /proc/cpuinfo         

flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid

-----否则，cpu不支持或是主板BIOS的虚拟化支持没有打开。另外KVM需要linux是x86_64架构的。

2、配置网卡（本示例中使用的eth1，因为eth0网卡有问题）

[root@management ~]# vi  /etc/sysconfig/network-scripts/ifcfg-eth1

3、上面关闭了NM_CONTROLLED，开启network服务

[root@management ~]#chkconfig network on

[root@management ~]#service network restart

4.修改/etc/yum.conf文件，使yum保存缓存

[root@management ~]# vi  /etc/yum.conf

keepcache=1

5.修改主机名称并验证

[root@management ~]# vi  /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=management.example.com

~

[root@management ~]# vi  /etc/hosts                (规划时有多少主机IP对应的写多少)

192.168.1.250  management.example.com

192.168.1.251  agent-1.example.com

[root@management ~]# hostname  --fqdn

Management.example.com

注意：如果此时查询时没有返回正确的长格式名称，可能需要重启linux服务器。

6、重启network服务

[root@management ~]# service network restart

7、安装NTP服务并配置

[root@management ~]# yum  install ntp  -y

[root@management ~]# chkconfig   ntpd  on

[root@management ~]# vi /etc/ntp.conf

      #server 0.centos.pool.ntp.org              22行首加“#”，注释改行

     #server 1.centos.pool.ntp.org               23行首加“#”，注释改行

     #server 2.centos.pool.ntp.org               24行首加“#”，注释改行

      server  127.127.1.0     # local clock       35行去掉行首“#”

      fudge   127.127.1.0 stratum 10            36行去掉行首“#”

[root@management ~]# service  ntpd  restart

Shutting down ntpd: [  OK  ]

Starting ntpd: [  OK  ]

8、清空防火墙

[root@management ~]# iptables  -F

[root@management ~]# service  iptables  save

iptables: Saving firewall rules to /etc/sysconfig/iptables: [  OK  ]

9、关闭SElinux

[root@management ~]# vi  /etc/selinux/config

SELINUX=permissive

[root@management ~]# setenforce 0

[root@management ~]# getenforce

Permissive

10、上传cloudstack 安装包和kvm用的系统模板

[root@management ~]# ls

acton-systemvm-02062012.qcow2.bz2  Desktop    install.log         Pictures   Videos

anaconda-ks.cfg                    Documents  install.log.syslog  Public

CloudStack-3.0.5-1-rhel6.2.tar.gz  Downloads  Music               Templates

11、安装mysql数据库

[root@management ~]# tar  -zxvf  CloudStack-3.0.5-1-rhel6.2.tar.gz

[root@management ~]# cd  CloudStack-3.0.5-1-rhel6.2

[root@management CloudStack-3.0.5-1-rhel6.2]# ls

6.2  install.sh  oss  repodata  scripts

[root@management CloudStack-3.0.5-1-rhel6.2]# ./install.sh

Setting up the temporary repository...

Cleaning Yum cache...

Loaded plugins: fastestmirror, refresh-packagekit, security

Cleaning repos: base cloud-temp extras updates

6 metadata files removed

Welcome to the CloudStack Installer.  What would you like to do?

        NOTE:   For installing KVM agent, please setup EPEL yum repo first;

For installing CloudStack on RHEL6.x, please setup distribution yum repo either from ISO or from your registeration account.

    M) Install the Management Server  

    A) Install the Agent

    B) Install BareMetal Agent

    S) Install the Usage Monitor

    D) Install the database server (from distribution's repo)     

    L) Install the MySQL 5.1.58 (only for CentOS5.x, Rhel6.x naturally has higher version MySql)

    Q) Quit

> D                               #（安装mysql数据库）

12、配置mysql服务

[root@management CloudStack-3.0.5-1-rhel6.2]# vi   /etc/my.cnf

在[mysqld] 字段下添加：

innodb_rollback_on_timeout=1

innodb_lock_wait_timeout=600

max_cOnnections=350

log-bin=mysql-bin

binlog-format = 'ROW'

13、重启mysql服务

[root@management CloudStack-3.0.5-1-rhel6.2]# service  mysqld  restart

Stopping mysqld:  [  OK  ]

Starting mysqld:  [  OK  ]

[root@management CloudStack-3.0.5-1-rhel6.2]# chkconfig mysqld on

14、安装cloudstack management 端( rpm包： cloud-client)

[root@management CloudStack-3.0.5-1-rhel6.2]# ./install.sh

Setting up the temporary repository...

Cleaning Yum cache...

Loaded plugins: fastestmirror, refresh-packagekit, security

Cleaning repos: base cloud-temp extras updates

7 metadata files removed

Welcome to the CloudStack Installer.  What would you like to do?

        NOTE:   For installing KVM agent, please setup EPEL yum repo first;

For installing CloudStack on RHEL6.x, please setup distribution yum repo either from ISO or from your registeration account.

3.We detect you already have MySql server installed, you can bypass mysql install chapter in CloudStack installation guide.

        Or you can use E) to remove current mysql then re-run install.sh selecting D) to reinstall if you think existing MySql server has some trouble.

        For MySql downloaded from community, the script may not be able to detect it.

    M) Install the Management Server  

    A) Install the Agent

    B) Install BareMetal Agent

    S) Install the Usage Monitor

    E) Remove the MySQL server (will not remove the MySQL databases)

    Q) Quit

> M                               #（安装management server）

15、cloudstack数据库初始化

注意：本示例中并没有设置mysql的root密码

[root@management CloudStack-3.0.5-1-rhel6.2]# cloud-setup-databases   cloud:cloud@localhost  --deploy-as=root: 

Mysql user name:cloud                                                           [ OK ]

Mysql user password:cloud                                                       [ OK ]

Mysql server ip:localhost                                                       [ OK ]

Mysql server port:3306                                                          [ OK ]

Mysql root user name:root                                                       [ OK ]

Mysql root user password:                                                       [ OK ]

Checking Cloud database files ...                                               [ OK ]

Checking local machine hostname ...                                             [ OK ]

Checking SELinux setup ...                                                      [ OK ]

Detected local IP address as 192.168.1.250, will use as cluster management server node IP[ OK ]

Preparing /etc/cloud/management/db.properties                                   [ OK ]

Applying /usr/share/cloud/setup/create-database.sql                             [ OK ]

Applying /usr/share/cloud/setup/create-schema.sql                               [ OK ]

Applying /usr/share/cloud/setup/create-database-premium.sql                     [ OK ]

Applying /usr/share/cloud/setup/create-schema-premium.sql                       [ OK ]

Applying /usr/share/cloud/setup/server-setup.sql                                [ OK ]

Applying /usr/share/cloud/setup/templates.sql                                   [ OK ]

Applying /usr/share/cloud/setup/create-index-fk.sql                             [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_db.sql                    [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_schema.sql                [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_multipart.sql             [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_index.sql                 [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_multipart_alter.sql       [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_bucketpolicy.sql          [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_policy_alter.sql          [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_offering.sql              [ OK ]

Applying /usr/share/cloud/setup/bridge/db/cloudbridge_offering_alter.sql        [ OK ]

Processing encryption ...                                                       [ OK ]

Finalizing setup ...                                                            [ OK ]

CloudStack has successfully initialized database, you can check your database configuration in /etc/cloud/management/db.properties

16、初始化cloud-management服务

[root@management CloudStack-3.0.5-1-rhel6.2]# cloud-setup-management

Starting to configure CloudStack Management Server:

Configure sudoers ...         [OK]

Configure Firewall ...        [OK]

Configure CloudStack Management Server ...[OK]

CloudStack Management Server setup is Done!

17、查看cloud-management服务状态

[root@management CloudStack-3.0.5-1-rhel6.2]# service   cloud-management  status

cloud-management (pid  30417) is running...

18、安装并配置NFS服务

[root@management CloudStack-3.0.5-1-rhel6.2]# yum  install nfs-utils  -y

19、产看当前磁盘使用情况(可以注意到/home目录的空间最大，此处我们用于nfs共享)

[root@management CloudStack-3.0.5-1-rhel6.2]# df  -h

Filesystem            Size  Used Avail Use% Mounted on

/dev/mapper/vg_management-lv_root

50G  4.1G   43G   9% /

tmpfs                  16G  260K   16G   1% /dev/shm

/dev/sda1             485M   37M  423M   9% /boot

/dev/mapper/vg_management-lv_home

1.3T  198M  1.3T   1% /home

20、/home目录权限如下

[root@management CloudStack-3.0.5-1-rhel6.2]# ll -d  /home/

drwxr-xr-x. 3 root root 4096 Sep 23  2011 /home/

21、建立主存储和二级存储目录

[root@management CloudStack-3.0.5-1-rhel6.2]# mkdir  -p  /home/storage/primary

[root@management CloudStack-3.0.5-1-rhel6.2]# mkdir  -p  /home/storage/secondary

----注意：本示例中的磁盘划分不是很合理，最好是能为/storage单独做个分区

22、配置nfs的共享目录

[root@management CloudStack-3.0.5-1-rhel6.2]# vi  /etc/exports

添加如下两行：

/home/storage/primary   *(rw,async,no_root_squash)

/home/storage/secondary  *(rw,async,no_root_squash)

~

22、NFSv4要求所有客户端的域设置匹配

[root@management CloudStack-3.0.5-1-rhel6.2]# vi  /etc/idmapd.conf

Domain = example.com                       #（本示例中域名是 example.com）

23、配置nfs

[root@management CloudStack-3.0.5-1-rhel6.2]# vi  /etc/sysconfig/nfs

取消下面各行行首的“#”，取消注释

LOCKD_TCPPORT=32803

LOCKD_UDPPORT=32769

MOUNTD_PORT=892

RQUOTAD_PORT=875

STATD_PORT=662

STATD_OUTGOING_PORT=2020

24、nfs服务的启动

[root@management CloudStack-3.0.5-1-rhel6.2]# chkconfig  rpcbind  on

[root@management CloudStack-3.0.5-1-rhel6.2]# service  rpcbind  restart

Stopping rpcbind: [  OK  ]

Starting rpcbind: [  OK  ]

[root@management CloudStack-3.0.5-1-rhel6.2]# chkconfig  nfs  on

[root@management CloudStack-3.0.5-1-rhel6.2]# service  nfs  restart

[root@management CloudStack-3.0.5-1-rhel6.2]# showmount  -e  192.168.1.250

Export list for 192.168.1.250:

/home/storage/secondary  *

/home/storage/primary   *

25、挂在二级存储

[root@management CloudStack-3.0.5-1-rhel6.2]# mount  -t nfs 192.168.1.250:/home/storage/secondary   /opt

26、上传kvm系统模板

[root@management CloudStack-3.0.5-1-rhel6.2]# /usr/lib64/cloud/agent/scripts/storage/secondary/cloud-install-sys-tmplt -m  /opt   -f  /root/acton-systemvm-02062012.qcow2.bz2 -h  kvm  -F

返回结果：

Successfully installed system VM template /root/acton-systemvm-02062012.qcow2.bz2 to /opt/template/tmpl/1/3/

27、卸载二级存储

[root@management CloudStack-3.0.5-1-rhel6.2]# umount  /opt

28、重启cloud-management服务

[root@management CloudStack-3.0.5-1-rhel6.2]# service  cloud-management  restart

Stopping cloud-management:[  OK  ]

Starting cloud-management: [  OK  ]

29、在浏览器（最好使用火狐，IE下访问有BUG）输入 http：//serverIP：8080/client

默认账户：admin

密码：password

30、本示例中选中“我以前使用过cloudstack，跳过该指南”

注意：此时我们还没有安装agent端

31、进入界面后的全局概览

32、选中“基础架构”用以查看当前状态

、选中“全局设置”

、搜索关键字“secstorage”并修改如下所示的参数值

、设置二级存储的NFS所在网段并确认

36、选中“close”

37、搜索关键字“cidr”并查看管理端serverIP是否在“management.netword.cidr”的网段内

38、搜索关键字“expunge”，并修改 “expunge.delay”和“expunge.interval”两个的值

39、选项值设置完成

40、在management 服务器端重启cloud-management服务

[root@management ~]# service  cloud-management  restart

Stopping cloud-management:[  OK  ]

Starting cloud-management: [  OK  ]

第二部分、cloud  agent端的安装配置

1、验证agent服务器支持虚拟化指令集

[root@agent-1 ~]#  grep  vmx  /proc/cpuinfo

flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm arat xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid

2、配置网卡

[root@agent-1 ~]# vi  /etc/sysconfig/network-scripts/ifcfg-em1

注意：NM_COnTROLLED=no 

3、启动network服务

[root@agent-1 ~]# chkconfig   network  on

[root@agent-1 ~]# service  network  restart

4.修改主机名称

[root@agent-1 ~]# chkconfig   network  on

[root@agent-1 ~]# vi  /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=agent-1.example.com

[root@agent-1 ~]# hostname  agent-1.example.com

5.配置/etc/hosts名称解析文件

[root@agent-1 ~]# vi  /etc/hosts

添加如下内容：

192.168.1.251  agent-1.example.com

~

6、验证主机的长格式名称

[root@agent-1 ~]# hostname  --fqdn

agent-1.example.com

7、验证ntp服务是否安装

[root@agent-1 ~]# rpm  -qa  ntp

ntp-4.2.4p8-2.el6.centos.x86_64

8、配置ntp时间服务

[root@agent-1 ~]# grep  -v  "^#"   /etc/ntp.conf

driftfile /var/lib/ntp/drift

restrict default kod nomodify notrap nopeer noquery

restrict -6 default kod nomodify notrap nopeer noquery

restrict 127.0.0.1

restrict -6 ::1

includefile /etc/ntp/crypto/pw

keys /etc/ntp/keys

server 192.168.1.250

9、启动ntp服务

[root@agent-1 ~]# chkconfig  ntpd  on

[root@agent-1 ~]# service  ntpd  restart

Shutting down ntpd: [  OK  ]

Starting ntpd: [  OK  ]

10、清空防火墙

[root@agent-1 ~]# iptables  -F

[root@agent-1 ~]# service  iptables  save

iptables: Saving firewall rules to /etc/sysconfig/iptables: [  OK  ]

11、关闭SElinux

[root@agent-1 ~]# vi  /etc/selinux/config

SELINUX=permissive

[root@agent-1 ~]# setenforce  0

12、安装nfs工具包

[root@agent-1 ~]# yum  install  nfs-utils   -y

13、设置nfs的域

[root@agent-1 ~]# vi  /etc/idmapd.conf

Domain = example.com

14、验证nfs共享存储

[root@agent-1 ~]# showmount  -e  192.168.1.250

Export list for 192.168.1.250:

/home/storage/secondary *

/home/storage/primary   *

15、上传cloudstack安装包到服务器

[root@agent-1 ~]# ls

anaconda-ks.cfg                    Documents    install.log.syslog  Public

CloudStack-3.0.5-1-rhel6.2.tar.gz  Downloads    Music               Templates

Desktop                            install.log  Pictures            Videos

16、解压安装包

[root@agent-1 ~]# tar  -zxvf  CloudStack-3.0.5-1-rhel6.2.tar.gz

17、进入安装目录

[root@agent-1 ~]# cd CloudStack-3.0.5-1-rhel6.2

[root@agent-1 CloudStack-3.0.5-1-rhel6.2]# ls

6.2  install.sh  oss  repodata  scripts

18、安装agent客户端

[root@agent-1 CloudStack-3.0.5-1-rhel6.2]# ./install.sh

Setting up the temporary repository...

Cleaning Yum cache...

Loaded plugins: fastestmirror, refresh-packagekit, security

Cleaning repos: base cloud-temp extras updates

6 metadata files removed

Welcome to the CloudStack Installer.  What would you like to do?

        NOTE:   For installing KVM agent, please setup EPEL yum repo first;

For installing CloudStack on RHEL6.x, please setup distribution yum repo either from ISO or from your registeration account.

M) Install the Management Server  

    A) Install the Agent

    B) Install BareMetal Agent

    S) Install the Usage Monitor

    D) Install the database server (from distribution's repo)     

    L) Install the MySQL 5.1.58 (only for CentOS5.x, Rhel6.x naturally has higher version MySql)

Q) Quit

> A

返回结果：

Complete!

Agent installation is completed, please add the host from management server

Done

19、安装virt-manager并设置qemu文件

[root@agent-1 ~]# yum  install  virt-manager  -y

[root@agent-1 ~]# vi   /etc/libvirt/qemu.conf

打开改行的注释：

vnc_listen = "0.0.0.0"

20、配置cgconfig.conf文件并启动cgconfig服务

[root@agent-1 cloud]# vi  /etc/cgconfig.conf

添加如下内容：

group virt {

cpu {

cpu.shares = 9216;

}

}

[root@agent-1 ~]#  service   cgconfig  start

Starting cgconfig service: [确定]

[root@agent-1 ~]# chkconfig  cgconfig  on

21、重启libvirtd服务

[root@agent-1 ~]# service  libvirtd  restart

Stopping libvirtd daemon: [FAILED]

Starting libvirtd daemon: [  OK  ]

-------千万注意：不要在agent端启动cloud-agent服务。我们通过management端添加区域主机来调用cloud-agent服务。

第三部分、创建基本zone

、http://serverIP:8080/client   ----选中“基础架构”----“（区域）查看全部”

默认账号：admin

    密码：password

、选中“添加区域”

、选择“基本”----“next”

、设置区域信息和虚拟机管理程序，然后 “next”

、默认即可，“next”

、设置提供点的信息，选中“next”

、设置来宾网络，然后“next”

、设置群集名称，然后“next”

、填写主机信息，然后“next”

、设置主存储信息，然后“next”

、设置二级存储信息，然后“next”

、选中“launch  zone”

、选中“是”确认启动该区域

、“基础架构”----“区域” 

、选中“基础架构”----“（系统VM）查看全部”

、大约5---8分钟，系统虚拟机启动成功

----至此kvm+cloudstack平台搭建完成