kubernetes之Ingress controller前言&＃xff1a;
traefik
Traefik是一个用Golang开发的轻量级的Http反向代理和负载均衡器。由于可以自动配置和刷新backend节点&＃xff0c;目前可以被绝大部分容器平台支持&＃xff0c;例如Kubernetes&＃xff0c;Swarm&＃xff0c;Rancher等。由于traefik会实时与Kubernetes API交互,所以对于Service的节点变化&＃xff0c;traefik的反应会更加迅速。总体来说traefik可以在Kubernetes中完美的运行.
Nginx-Ingress-Controller
Nginx-Ingress-Controller对于绝大多数刚刚接触k8s的人来说都比较熟悉&＃xff0c;一个对外暴露service的7层反向代理。目前最新代号0.9.0-beta.15&＃xff0c;可见目前nginx-ingress-control仍然处于beta版本。不过接触过的人还是明白nginx-ingress-control强大的Annotate配置&＃xff0c;可以为service提供丰富的个性化配置&＃xff0c;这点对于traefik来说是目前还无法打到的地步。
部署&＃xff1a;
要使用 traefik&＃xff0c;我们同样需要部署 traefik 的 Pod&＃xff0c;由于我们演示的集群中只有 master 节点有外网网卡&＃xff0c;所以我们这里只有 master 这一个边缘节点&＃xff0c;我们将 traefik 部署到该节点上即可。首先&＃xff0c;为安全起见我们这里使用 RBAC 安全认证方式&＃xff1a;(rbac.yaml)&＃xff1a;
vim traefik-rbac.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:name: traefik-ingress-controllernamespace: kube-ops
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:name: traefik-ingress-controller
rules:- apiGroups:- ""resources:- services- endpoints- secretsverbs:- get- list- watch- apiGroups:- extensionsresources:- ingressesverbs:- get- list- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:name: traefik-ingress-controller
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: traefik-ingress-controller
subjects:
- kind: ServiceAccountname: traefik-ingress-controllernamespace: kube-ops
kubectl apply -f traefik-rbac.yaml
[root&＃64;kubemaster traefik]# kubectl get ClusterRole -n kube-ops|grep traefik
traefik-ingress-controller 11m
[root&＃64;kubemaster traefik]# kubectl get ClusterRoleBinding -n kube-ops|grep traefik
traefik-ingress-controller 2m36s
[root&＃64;kubemaster traefik]# kubectl get sa -n kube-ops
NAME SECRETS AGE
default 1 44h
prometheus 1 14h
traefik-ingress-controller 1 11m
[root&＃64;kubemaster traefik]#
可以查看到SA、ClusterRole和ClusterRoleBinding资源
vim traefik-deployment.yaml
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:name: traefik-ingress-controllernamespace: kube-opslabels:k8s-app: traefik-ingress-lb
spec:replicas: 1selector:matchLabels:k8s-app: traefik-ingress-lbtemplate:metadata:labels:k8s-app: traefik-ingress-lbname: traefik-ingress-lbspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 60containers:- image: traefikname: traefik-ingress-lbports:- name: httpcontainerPort: 80hostPort: 80- name: admincontainerPort: 8080args:- --api- --kubernetes- --logLevel&＃61;INFO
---
kind: Service
apiVersion: v1
metadata:name: traefik-ingress-servicenamespace: kube-ops
spec:selector:k8s-app: traefik-ingress-lbports:- protocol: TCPport: 80name: web- protocol: TCPport: 8080name: admintype: NodePort
此处在containerPort里面的字段hostPort指定了&＃xff0c;此容器的端口直接映射到宿主机的80端口&＃xff0c;在创建Ingress资源之前&＃xff0c;我们先需要创建一个演示的web应用
我开始部署一个测试的app应用&＃xff0c;vim traefik-backend-app.yaml 部署了一个deployment和service,然后测试访问.这里我们部署的应用只能通过ClusterIP访问&＃xff0c;而且ClusterIP只能是K8S集群内部才能访问的。如果需要从宿主机的外部访问到这个app应用&＃xff0c;就需要把Service修改成NodePort的类型。加入有上百个应用在一个宿主机上面运行&＃xff0c;那么修改成NodePort的类型的Service&＃xff0c;一个宿主机的Iptables防火墙需要增加上百条策略&＃xff0c;而且每一个宿主机都需要这样操作&＃xff0c;势必会带来管理上的不便。这也就是为什么会产生Ingress资源的原因。客户访问k8s集群里面的web应用的流程应该是首先访问到公司的外部SLB设备&＃xff08;可以是硬件的负载均衡器比如F5等,也可以是软件比如LVS等。然后在从外部的LB设备到k8s集群的Ingress Controller。Ingress Controller就是k8s集群的访问入口&＃xff0c;相当于nginx服务器一样。Ingress Controller既可以支持https协议&＃xff0c;也可以通过虚拟主机或者URL映射的方式调用后端的upstream服务器。后端的upstream服务器就是真正运行的Pod.所以k8s集群只需要将Ingress Controller映射出去即可&＃xff1b;
[root&＃64;kubemaster traefik]# kubectl get pods -n kube-ops --- 现在我们开始创建一个Ingress对象资源,vim traefik-ingress.yaml apiVersion: extensions/v1beta1 现在我们开始在自己的电脑的hosts文件上面增加A记录&＃xff0c;域名对应的IP地址就是运行traefik-ingress-controller的k8s node机器。由于我这边有两个node节点都运行了traefik-ingress-controller&＃xff0c;所以绑定了连个地址 10.83.32.146 myapp.maimaiti.cn 浏览器页面访问http://myapp.maimaiti.cn&＃xff0c;输出的结果是 apiVersion: v1 然后开始重新修改一下Ingress资源的配置&＃xff0c;将tomcat应用对应一个域名tomcat.maimaiti.cn来访问 apiVersion: extensions/v1beta1 现在我们开始在自己的电脑的hosts文件上面增加A记录&＃xff0c;域名对应的IP地址就是运行traefik-ingress-controller的k8s node机器。由于我这边有两个node节点都运行了traefik-ingress-controller&＃xff0c;所以绑定了连个地址 10.83.32.146 myapp.maimaiti.cn tomcat.maimaiti.cn 2. traefik Ingress Controll https认证配置 defaultEntryPoints &＃61; ["http","https"] 配置文件主要包含了https接口访问的证书位置和prometheus的监控配置&＃xff0c;接下来创建自签名证书 openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt 接下来需要修改traefik Ingress Controll的deployment的配置&＃xff0c;增加上读取configmap和secret的参数&＃xff0c;并暴露443端口提供https的访问 --- 接下来需要修改Ingress资源的配置&＃xff0c;增加上https访问 apiVersion: extensions/v1beta1 现在就可以用https访问tomcat和app
NAME READY STATUS RESTARTS AGE
myapp-deploy-6b56d98b6b-65jc9 1/1 Running 0 7m30s
myapp-deploy-6b56d98b6b-r92p8 1/1 Running 0 7m30s
myapp-deploy-6b56d98b6b-rrb5b 1/1 Running 0 7m30s
node-exporter-788bd 1/1 Running 1 43h
node-exporter-7vfs7 1/1 Running 1 43h
node-exporter-xkj2b 1/1 Running 1 43h
prometheus-848d44c7bc-zwlb8 1/1 Running 0 15h
redis-58c6c94968-qcq6p 2/2 Running 2 44h
traefik-ingress-controller-86d4b5fcbf-6pfm5 1/1 Running 0 25m
traefik-ingress-controller-86d4b5fcbf-bs69c 1/1 Running 0 25m
[root&＃64;kubemaster traefik]# kubectl get svc -n kube-ops
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myapp ClusterIP 10.98.239.156
prometheus NodePort 10.109.108.37
redis ClusterIP 10.100.225.179
traefik-ingress-service NodePort 10.111.9.88
Hello MyApp | Version: v2 |
apiVersion: v1
kind: Service
metadata:name: myappnamespace: kube-ops
spec:selector:app: myapprelease: canaryports:- name: http targetPort: 80port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:name: myapp-deploy namespace: kube-ops
spec:replicas: 3selector:matchLabels:app: myapprelease: canarytemplate:metadata:labels:app: myapprelease: canaryspec:containers:- name: myappimage: ikubernetes/myapp:v2ports&＃xff1a;- name: http
kind: Ingress
metadata:name: ingress-appnamespace: kube-opsannotations:kubernetes.io/ingress.class: traefik
spec:rules:- host: myapp.maimaiti.cnhttp:paths:- backend:serviceName: myappservicePort: 80
kubectl apply -f traefik-ingress.yaml
[root&＃64;kubemaster traefik]# kubectl get ingress -n kube-ops
NAME HOSTS ADDRESS PORTS AGE
ingress-app myapp.maimaiti.cn 80 8s
10.83.32.138 myapp.maimaiti.cn
Hello MyApp | Version: v2 | Pod Name
我们除了通过Ingress Controller访问k8s集群的应用的Pod之外&＃xff0c;traefik Ingress还有一个管理界面可以访问&＃xff0c;现在我们再创建一个deployment,用于部署tomcat应用&＃xff0c;然后也通过traefik Ingress Controller来提供流量访问入口
kind: Service
metadata:name: tomcatnamespace: kube-ops
spec:selector:app: tomcatrelease: canaryports:- name: http targetPort: 8080port: 8080- name: ajptargetPort: 8009port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:name: tomcat-deploy namespace: kube-ops
spec:replicas: 3selector:matchLabels:app: tomcatrelease: canarytemplate:metadata:labels:app: tomcatrelease: canaryspec:containers:- name: tomcatimage: tomcat:8.5.32-jre8-alpineports:- name: http containerPort: 8080- name: ajpcontainerPort: 8009
kubectl apply -f traefik-backend-tomcat.yaml
kind: Ingress
metadata:name: ingress-appnamespace: kube-opsannotations:kubernetes.io/ingress.class: traefik
spec:rules:- host: myapp.maimaiti.cnhttp:paths:- backend:serviceName: myappservicePort: 80- host: tomcat.maimaiti.cnhttp:paths:- backend:serviceName: tomcatservicePort: 8080
kubectl apply -f treafik-ingress.yaml
10.83.32.138 myapp.maimaiti.cn tomcat.maimaiti.cn
2.1. 配置traefik Ingress Controller的配置文件toml:
vim traefik.toml
[entryPoints][entryPoints.http]address &＃61; ":80"[entryPoints.https]address &＃61; ":443"[entryPoints.https.tls][[entryPoints.https.tls.certificates]]CertFile &＃61; "/ssl/tls.crt"KeyFile &＃61; "/ssl/tls.key"
[metrics][metrics.prometheus]entryPoint &＃61; "traefik"buckets &＃61; [0.1, 0.3, 1.2, 5.0]
kubectl create configmap traefik-conf --from-file&＃61;traefik.toml -n kube-ops[root&＃64;kubemaster traefik]# kubectl describe cm -n kube-ops traefik-conf
Name: traefik-conf
Namespace: kube-ops
Labels:
Annotations:
&＃61;&＃61;&＃61;&＃61;
traefik.toml:
----
defaultEntryPoints &＃61; ["http","https"]
[entryPoints][entryPoints.http]address &＃61; ":80"[entryPoints.https]address &＃61; ":443"[entryPoints.https.tls][[entryPoints.https.tls.certificates]]CertFile &＃61; "/ssl/tls.crt"KeyFile &＃61; "/ssl/tls.key"
[metrics][metrics.prometheus]entryPoint &＃61; "traefik"buckets &＃61; [0.1, 0.3, 1.2, 5.0]Events:
[root&＃64;kubemaster traefik]#
Generating a 2048 bit RSA private key
...........&＃43;&＃43;&＃43;
................................................................&＃43;&＃43;&＃43;
writing new private key to &＃39;tls.key&＃39;
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter &＃39;.&＃39;, the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:GD
Locality Name (eg, city) [Default City]:SZ
Organization Name (eg, company) [Default Company Ltd]:MMT
Organizational Unit Name (eg, section) []:IT
Common Name (eg, your name or your server&＃39;s hostname) []:gaoyang
Email Address []:gaoyang&＃64;maimaiti.cn
[root&＃64;kubemaster traefik]# ll
total 32
-rw-r--r-- 1 root root 1367 Mar 7 14:55 tls.crt
-rw-r--r-- 1 root root 1708 Mar 7 14:55 tls.key
-rw-r--r-- 1 root root 601 Mar 7 10:55 traefik-backend-app.yaml
-rw-r--r-- 1 root root 718 Mar 7 13:44 traefik-backend-tomcat.yaml
-rw-r--r-- 1 root root 1028 Mar 7 11:02 traefik-deployment.yaml
-rw-r--r-- 1 root root 418 Mar 7 14:07 traefik-ingress.yaml
-rw-r--r-- 1 root root 800 Mar 7 10:28 traefik-rbac.yaml
-rw-r--r-- 1 root root 364 Mar 7 14:50 traefik.toml
#创建所需要的证书文件和Pod里面调用的secret资源
kubectl create secret generic traefik-cert --from-file&＃61;tls.crt --from-file&＃61;tls.key -n kube-ops
kind: Deployment
apiVersion: extensions/v1beta1
metadata:name: traefik-ingress-controllernamespace: kube-opslabels:k8s-app: traefik-ingress-lb
spec:replicas: 2selector:matchLabels:k8s-app: traefik-ingress-lbtemplate:metadata:labels:k8s-app: traefik-ingress-lbname: traefik-ingress-lbspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 60volumes:- name: sslsecret:secretName: traefik-cert- name: configconfigMap:name: traefik-confcontainers:- image: traefikname: traefik-ingress-lbvolumeMounts:- name: "ssl"mountPath: "/ssl"- name: "config"mountPath: "/config"ports:- name: httpcontainerPort: 80hostPort: 80- name: httpscontainerPort: 443hostPort: 443- name: admincontainerPort: 8080args:- --configfile&＃61;/config/traefik.toml- --api- --kubernetes- --logLevel&＃61;INFO
---
kind: Service
apiVersion: v1
metadata:name: traefik-ingress-servicenamespace: kube-ops
spec:selector:k8s-app: traefik-ingress-lbports:- protocol: TCPport: 80name: web- protocol: TCPport: 8080name: admintype: NodePort
# 注意此处重新修改了deployment文件&＃xff0c;增加了secret和configmap的挂载&＃xff0c;增加了启动读取配置文件的参数
kind: Ingress
metadata:name: ingress-appnamespace: kube-opsannotations:kubernetes.io/ingress.class: traefik
spec:tls:- hosts:- myapp.maimaiti.cnsecretName: traefik-certrules:- host: myapp.maimaiti.cnhttp:paths:- backend:serviceName: myappservicePort: 80- host: tomcat.maimaiti.cnhttp:paths:- backend:serviceName: tomcatservicePort: 8080
kubectl apply -f traefik-ingress.yaml
推荐关注我的个人微信公众号 “云时代IT运维”,周期性更新最新的应用运维类技术文档。关注虚拟化和容器技术、CI/CD、自动化运维等最新前沿运维技术和趋势&＃xff1b;
京公网安备 11010802041100号