Linux——K8s集群监控

K8s环境

1.下载dashboard的yaml文件

[root&＃64;master ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml
--2020-11-20 20:43:00-- https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml
正在解析主机 raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.76.133
正在连接 raw.githubusercontent.com (raw.githubusercontent.com)|151.101.76.133|:443... 已连接。
已发出 HTTP 请求&＃xff0c;正在等待回应... 200 OK
长度&＃xff1a;7552 (7.4K) [text/plain]
正在保存至: “recommended.yaml”100%[&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;>] 7,552 6.05KB/s 用时 1.2s 2020-11-20 20:43:03 (6.05 KB/s) - 已保存 “recommended.yaml” [7552/7552])
[root&＃64;master ~]# ls
recommended.yaml


2.更改配置文件使暴露端口

[root&＃64;master ~]# vim recommended.yaml
......
kind: Service
apiVersion: v1
metadata:labels:k8s-app: kubernetes-dashboardname: kubernetes-dashboardnamespace: kubernetes-dashboard
spec:type: NodePort #添加此项ports:- port: 443targetPort: 8443nodePort: 30066 #添加selector:k8s-app: kubernetes-dashboard
......
[root&＃64;master ~]# kubectl apply -f recommended.yaml


3.查看SVC已暴露端口

PS&＃xff1a;是基于https的访问哦

[root&＃64;master ~]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.96.74.95 8000/TCP 73s
kubernetes-dashboard NodePort 10.96.18.208 443:30066/TCP 73s


这个页面&＃xff0c;可以看到登录dashboard有两种方式

4.基于token的方法登录

4.1 创建一个dashboard的管理用户

[root&＃64;master ~]# kubectl create serviceaccount dashboard-admin -n kube-system
serviceaccount/dashboard-admin created


4.2 绑定用户为集群管理用户

[root&＃64;master ~]# kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole&＃61;cluster-admin --serviceaccount&＃61;kube-system:dashboard-admin
clusterrolebinding.rbac.authorization.k8s.io/dashboard-cluster-admin created


4.3 获取token

[root&＃64;master ~]# kubectl get secrets -n kube-system | grep dashboard-admin
dashboard-admin-token-scbsf kubernetes.io/service-account-token 3 16s
[root&＃64;master ~]# kubectl describe secrets -n kube-system dashboard-admin-token-scbsf
Name: dashboard-admin-token-scbsf
Namespace: kube-system
Labels:
Annotations: kubernetes.io/service-account.name: dashboard-adminkubernetes.io/service-account.uid: 35d3a0d0-3928-4005-b5a1-a779e864329fType: kubernetes.io/service-account-tokenData
&＃61;&＃61;&＃61;&＃61;
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tc2Nic2YiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMzVkM2EwZDAtMzkyOC00MDA1LWI1YTEtYTc3OWU4NjQzMjlmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.VAWSBvU9sGdDV6l7ZEvXF4MRo7kY-YLTdrJWes-oy8haFji1ld35fZM7-qDrDntnZw-u1hOo4e1-BL3lyKXnuIoP8mTED8h6_IYHK6JZAixaTLMTkWBRlODaRdy8nH6j7gzcBU6-Kv9ubeeiutCXGJSjyfMY9qPcOnzzPHQTzSdCO1H67KMdkfGKJcFd0ZQoBMnNk49aUxxix9vBofpX7r3s00hJmw8Bmd_WBzGiHRVC1Q88l8uQ5sAeGBBoR7fS87NB9ZS-NM6XYMaQ_wTC_iK4KltSBdA5ZxyieWMh5SF1FScae-IBvwUbTE3jPW4N2LcvyJS3FYhlnX343sywvQ
ca.crt: 1025 bytes
namespace: 11 bytes


4.4 登录

PS&＃xff1a;如果是使用的旧版本的dashboard&＃xff0c;使用谷歌浏览器登录&＃xff0c;可能是不成功的&＃xff0c;需要换成其他的浏览器&＃xff0c;比如&＃xff1a;火狐。

5.基于kubeconfig配置文件的方式登录

5.1 获取token

[root&＃64;master ~]# kubectl get secrets -n kube-system | grep dashboard-admin
dashboard-admin-token-scbsf kubernetes.io/service-account-token 3 16s
[root&＃64;master ~]# kubectl describe secrets -n kube-system dashboard-admin-token-scbsf
Name: dashboard-admin-token-scbsf
Namespace: kube-system
Labels:
Annotations: kubernetes.io/service-account.name: dashboard-adminkubernetes.io/service-account.uid: 35d3a0d0-3928-4005-b5a1-a779e864329fType: kubernetes.io/service-account-tokenData
&＃61;&＃61;&＃61;&＃61;
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tc2Nic2YiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMzVkM2EwZDAtMzkyOC00MDA1LWI1YTEtYTc3OWU4NjQzMjlmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.VAWSBvU9sGdDV6l7ZEvXF4MRo7kY-YLTdrJWes-oy8haFji1ld35fZM7-qDrDntnZw-u1hOo4e1-BL3lyKXnuIoP8mTED8h6_IYHK6JZAixaTLMTkWBRlODaRdy8nH6j7gzcBU6-Kv9ubeeiutCXGJSjyfMY9qPcOnzzPHQTzSdCO1H67KMdkfGKJcFd0ZQoBMnNk49aUxxix9vBofpX7r3s00hJmw8Bmd_WBzGiHRVC1Q88l8uQ5sAeGBBoR7fS87NB9ZS-NM6XYMaQ_wTC_iK4KltSBdA5ZxyieWMh5SF1FScae-IBvwUbTE3jPW4N2LcvyJS3FYhlnX343sywvQ
ca.crt: 1025 bytes
namespace: 11 bytes


5.2 生产kubeconfig配置文件

[root&＃64;master ~]# DASH_TOKEN&＃61;$(kubectl get secrets -n kube-system dashboard-admin-token-scbsf -o jsonpath&＃61;{.data.token} | base64 -d)
[root&＃64;master ~]# kubectl config set-cluster kubernetes --server&＃61;192.168.1.40:6443 --kubeconfig&＃61;/root/.dashboard-admin.conf
Cluster "kubernetes" set.
[root&＃64;master ~]# kubectl config set-credentials dashboard-admin --token&＃61;$DASH_TOKEN --kubeconfig&＃61;/root/.dashboard-admin.conf
User "dashboard-admin" set.
[root&＃64;master ~]# kubectl config set-context dashboard-admin&＃64;kubernetes --cluster&＃61;kubernetes --user&＃61;dashboard-admin --kubeconfig&＃61;/root/.dashboard-admin.conf
Context "dashboard-admin&＃64;kubernetes" created.
[root&＃64;master ~]# kubectl config use-context dashboard-admin&＃64;kubernetes --kubeconfig&＃61;/root/.dashboard-admin.conf
Switched to context "dashboard-admin&＃64;kubernetes".


5.3 将生成的/root/.dashboard-admin.conf的配置文件&＃xff0c;导出并做保存。

PS&＃xff1a;根据自己的环境导出到相应的位置

[root&＃64;master ~]# scp -rp .dashboard-admin.conf node01:
.dashboard-admin.conf 100% 1199 465.9KB/s 00:00


5.4 从浏览器选择kubeconfig的登录方式&＃xff0c;然后导入配置文件即可.

1. 下载项目

[root&＃64;master ~]# wget https://github.com/prometheus-operator/kube-prometheus/archive/v0.3.0.tar.gz
[root&＃64;master ~]# tar zxf v0.3.0.tar.gz


2.更改配置文件

[root&＃64;master ~]# cd kube-prometheus-0.3.0/manifests/
[root&＃64;master manifests]#
[root&＃64;master manifests]# vim grafana-service.yaml
apiVersion: v1
kind: Service
metadata:labels:app: grafananame: grafananamespace: monitoring
spec:type: NodePortports:- name: httpport: 3000targetPort: httpnodePort: 31001selector:app: grafana
[root&＃64;master manifests]# vim prometheus-service.yaml
apiVersion: v1
kind: Service
metadata:labels:prometheus: k8sname: prometheus-k8snamespace: monitoring
spec:type: NodePortports:- name: webport: 9090targetPort: webnodePort: 31002selector:app: prometheusprometheus: k8ssessionAffinity: ClientIP
[root&＃64;master manifests]# vim alertmanager-service.yaml
[root&＃64;master manifests]# cat alertmanager-service.yaml
apiVersion: v1
kind: Service
metadata:labels:alertmanager: mainname: alertmanager-mainnamespace: monitoring
spec:type: NodePortports:- name: webport: 9093targetPort: webnodePort: 31003selector:alertmanager: mainapp: alertmanagersessionAffinity: ClientIP


3.基础环境配置

[root&＃64;master manifests]# pwd
/root/kube-prometheus-0.3.0/manifests
[root&＃64;master manifests]# kubectl apply -f setup/
namespace/monitoring created
customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com created
clusterrole.rbac.authorization.k8s.io/prometheus-operator created
clusterrolebinding.rbac.authorization.k8s.io/prometheus-operator created
deployment.apps/prometheus-operator created
service/prometheus-operator created
serviceaccount/prometheus-operator created


4.运行主yaml文件

PS&＃xff1a;运行主yaml文件。有可能因为目录内yaml文件过多&＃xff0c;一次不能全部运行&＃xff0c;所以运行的时候&＃xff0c;多运行两遍

[root&＃64;master kube-prometheus-0.3.0]# pwd
/root/kube-prometheus-0.3.0
[root&＃64;master kube-prometheus-0.3.0]# kubectl apply -f manifests/
[root&＃64;master kube-prometheus-0.3.0]# kubectl get pod -n monitoring
NAME READY STATUS RESTARTS AGE
alertmanager-main-0 2/2 Running 0 16m
alertmanager-main-1 2/2 Running 0 16m
alertmanager-main-2 2/2 Running 0 16m
grafana-77978cbbdc-6ddcc 1/1 Running 0 19m
kube-state-metrics-7f6d7b46b4-gts5w 3/3 Running 0 19m
node-exporter-h779k 2/2 Running 0 19m
node-exporter-pk2dv 2/2 Running 0 19m
node-exporter-vgqtk 2/2 Running 0 19m
prometheus-adapter-68698bc948-qnsb9 1/1 Running 5 19m
prometheus-k8s-0 3/3 Running 0 16m
prometheus-k8s-1 3/3 Running 0 16m
prometheus-operator-6685db5c6-r5srv 1/1 Running 0 21m[root&＃64;master kube-prometheus-0.3.0]# kubectl top node #查看资源使用情况
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master 226m 11% 1512Mi 87%
node01 89m 4% 1360Mi 79%
node02 86m 4% 1327Mi 77%


5.登录

账户&＃xff1a;admin 密码&＃xff1a;admin

6.导入监控模板