本次部署使用3个节点：
- Master节点，10.0.2.15
- Node1节点，10.0.2.4 docker0网桥默认地址手动配置为：172.16.10.1/24
- Node2节点，10.0.2.5 docker0网桥默认地址手动配置为：172.16.20.1/24
使用二进制文件方式安装k8s集群的方法。部署的是Kubernetes v1.9.8。
https://storage.googleapis.com/kubernetes-release/release/v1.9.8/kubernetes-server-linux-amd64.tar.gz
我们使用CentOS7.5，注意在每个节点上处理一下防火墙策略：
firewall-cmd --permanent --zOne=public --add-rich-rule="rule family="ipv4" source address="10.0.2.0/24" accept"
firewall-cmd --reload
一、使用二进制方式部署k8s的Master节点
1、etcd服务
从https://github.com/coreos/etcd/releases下载etcd二进制文件，将etcd和etcdctl文件复制到/usr/bin目录。
配置systemd服务文件：/usr/lib/systemd/system/etcd.service
[[email protected] system]# more etcd.service
[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
EnvirOnmentFile=-/etc/etcd/etcd.conf
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd
Restart=on-failure
[Install]
WantedBy=multi-user.target
配置和启动etcd服务：
[[email protected] ~]# ls
anaconda-ks.cfg etcd-v3.3.5-linux-amd64 etcd-v3.3.5-linux-amd64.tar.gz
[[email protected] ~]# cd etcd-v3.3.5-linux-amd64
[[email protected] etcd-v3.3.5-linux-amd64]# ls
Documentation etcd etcdctl README-etcdctl.md README.md READMEv2-etcdctl.md
[[email protected] etcd-v3.3.5-linux-amd64]# mv etcd etcdctl /usr/bin
[[email protected] etcd-v3.3.5-linux-amd64]# systemctl daemon-reload
[[email protected] etcd-v3.3.5-linux-amd64]# systemctl enable etcd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
[[email protected] etcd-v3.3.5-linux-amd64]# mkdir -p /var/lib/etcd/
[[email protected] etcd-v3.3.5-linux-amd64]# systemctl start etcd.service
[[email protected] etcd-v3.3.5-linux-amd64]# etcdctl cluster-health
member 8e9e05c52164694d is healthy: got healthy result from http://localhost:2379
cluster is healthy
[[email protected] etcd-v3.3.5-linux-amd64]#
2、kube-apiserver服务
将解压后kube-apiserver、kube-controller-manager、kube-scheduler以及管理要使用的kubectl二进制命令文件放到/usr/bin目录，即完成这几个服务的安装。
接下来对api服务进行必要的配置。
配置systemd服务文件：/usr/lib/systemd/system/kube-apiserver.service
# more /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=etcd.service
Wants=etcd.service
[Service]
EnvirOnmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS
Restart=on-failure
Type=notify
[Install]
WantedBy=multi-user.target
配置文件：
# mkdir -p /etc/kubernetes
# more /etc/kubernetes/apiserver
KUBE_API_ARGS="--storage-backend=etcd3 --etcd-servers=http://127.0.0.1:2379 --insecure-bind-address=0.0.0.0 --insecure-port=8080 --service-cluster-ip-range=10.10.10.0/24 --service-node-port-range=1-65535 --admission-cOntrol=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,DefaultStorageClass,ResourceQuota --logtostderr=true --log-dir=/var/log/kubernetes --v=2"
3、kube-controller-manager服务
kube-controller-manager服务依赖于kube-apiserver服务：
配置systemd服务文件：/usr/lib/systemd/system/kube-controller-manager.service
# more /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvirOnmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
[[email protected] system]# more /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvirOnmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
配置文件：
# more /etc/kubernetes/controller-manager
KUBE_CONTROLLER_MANAGER_ARGS="--master=http://10.0.2.15:8080 --logtostderr=true --log-dir=/var/log/kubernetes --v=2"
4、kube-scheduler服务
kube-scheduler服务也依赖于kube-apiserver服务。
配置systemd服务文件：/usr/lib/systemd/system/kube-scheduler.service
# more /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=kube-apiserver.service
Requires=kube-apiserver.service
[Service]
EnvirOnmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
配置文件：
# cat /etc/kubernetes/scheduler
KUBE_SCHEDULER_ARGS="--master=http://10.0.2.15:8080 --logtostderr=true --log-dir=/var/log/kubernetes --v=2"
5、完成以上配置后，按顺序启动这些服务
systemctl daemon-reload
systemctl enable kube-apiserver.service
systemctl start kube-apiserver.service
systemctl enable kube-controller-manager.service
systemctl start kube-controller-manager.service
systemctl enable kube-scheduler.service
systemctl start kube-scheduler.service
检查每个服务的健康状态：
systemctl status kube-apiserver.service
systemctl status kube-controller-manager.service
systemctl status kube-scheduler.service
如果有服务出现异常，可执行journalctl -xe -u kube-scheduler查看一个指定的服务的事件信息日志。
二、使用二进制方式部署k8s的Node节点
在Node节点上，以同样的方式把从压缩包中解压出的二进制文件kubelet kube-proxy放到/usr/bin目录中。
1、安装docker ce
先安装依赖工具包：
yum install -y yum-utils device-mapper-persistent-data lvm2
安装docker yum源：
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
安装docker引擎：
yum install docker-ce.x86_64
执行docker info查看安装结果。当前安装的是docker-ce-18.03.1.ce-1.el7.centos.x86_64.rpm。
将docker0网桥的默认地址配置为我们指定的：172.16.10.1
# pwd
/usr/lib/systemd/system
[[email protected] system]# more docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd --bip=172.16.10.1/24
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
# restart the docker process if it exits prematurely
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
注：--bip=172.16.10.1/24，将docker0网桥默认使用的网络地址变更为我们指定的地址。
启动docker服务：
systemctl daemon-reload
systemctl start docker
查看node1的网络地址和路由信息：
# ip a
1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:9f:89:14 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.4/24 brd 10.0.2.255 scope global noprefixroute dynamic enp0s3
valid_lft 774sec preferred_lft 774sec
inet6 fe80::a00:27ff:fe9f:8914/64 scope link
valid_lft forever preferred_lft forever
3: docker0: mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:22:bc:7a:ea brd ff:ff:ff:ff:ff:ff
inet 172.16.10.1/24 brd 172.16.10.255 scope global docker0
valid_lft forever preferred_lft forever
[[email protected] system]# ip r
default via 10.0.2.1 dev enp0s3 proto dhcp metric 100
10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.4 metric 100
172.16.10.0/24 dev docker0 proto kernel scope link src 172.16.10.1
2、kubelet服务
配置systemd服务文件：/usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service
[Service]
WorkingDirectory=/var/lib/kubelet
EnvirOnmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
KillMode=process
[Install]
WantedBy=multi-user.target
配置文件：
# cd /etc/kubernetes
# more kubelet
KUBELET_ARGS="--address=10.0.2.4 --port=10250 --hostname-override=10.0.2.4 --allow-privileged=false --kubecOnfig=/etc/kubernetes/kubelet.kubeconfig --cluster-dns
=10.10.10.2 --cluster-domain=cluster.local --fail-swap-on=false --logtostderr=true --log-dir=/var/log/kubernetes --v=4"
用于kubelet连接Master Apiserver的配置文件：
# more kubelet.kubeconfig apiVersion: v1
kind: Config
clusters:
- cluster:
server: http://10.0.2.15:8080
name: local
contexts:
- context:
cluster: local
name: local
current-context: local
启动kubelet服务：
mkdir -p /var/lib/kubelet
systemctl daemon-reload
systemctl enable kubelet
systemctl start kubelet
systemctl status kubelet
3、kube-proxy服务
kube-proxy服务依赖于network服务。
配置systemd服务文件：/usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.service
Requires=network.service
[Service]
EnvirOnmentFile=/etc/kubernetes/proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536
KillMode=process
[Install]
WantedBy=multi-user.target
配置文件：
# cat /etc/kubernetes/proxy
KUBE_PROXY_ARGS="--master=http://10.0.2.15:8080 --hostname-override=10.0.2.4 --logtostderr=true --log-dir=/var/log/kubernetes --v=4"
启动kube-proxy服务：
systemctl daemon-reload
systemctl enable kube-proxy
systemctl start kube-proxy
systemctl status kube-proxy
4、按照前面相同的步骤配置node2节点
唯一的不同之处是在指定docker0网桥的默认地址时使用下面的网段和地址：
ExecStart=/usr/bin/dockerd --bip=172.16.20.1/24
注：请在配置过程中将使用节点ip地址的地方也替换为node2的实际ip地址10.0.2.5
三、对搭建完成的k8s集群进行一些验证测试
在Master上查看k8s各项服务状态
[[email protected] bin]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
10.0.2.4 Ready 28m v1.9.8
10.0.2.5 Ready 1m v1.9.8
[[email protected] bin]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.10.10.1 443/TCP 3h
[[email protected] bin]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}
京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有 