skywalking监控nginxingresscontroller

微信公众号&＃xff1a;运维开发故事&＃xff0c;作者: 乔克

前提&＃xff1a;有可用的kubernetes集群和skywalking监控。

软件版本信息&＃xff1a;

下面直接进入正题…

&＃xff08;1&＃xff09;下载skywalking-nginx-lua

git clone https://github.com/apache/skywalking-nginx-lua.git

&＃xff08;2&＃xff09;修改util.lua文件名&＃xff0c;因为其和nginx-ingress默认的一个lua脚本名字冲突&＃xff0c;这里改一下。

$ skywalking-nginx-lua/lib/skywalking
# 修改文件名
$ mv util.lua swutil.lua
# 修改文件调用
## 可以使用grep查看一下哪些文件有调用这个lua
$ grep util &＃96;find ./ -type f&＃96;
./correlation_context.lua:local Util &＃61; require(&＃39;util&＃39;)
./segment_ref.lua:local Util &＃61; require(&＃39;util&＃39;)
./span.lua:local Util &＃61; require(&＃39;util&＃39;)
./tracing_context.lua:local Util &＃61; require(&＃39;util&＃39;)
./swutil_test.lua:local Util &＃61; require(&＃39;util&＃39;)
# 将里面调用的util都改为swuitl

&＃xff08;3&＃xff09;修改Nginx-ingress的nginx.tmpl模板文件&＃xff0c;增加Skywalking的配置。

• 添加Skywalking Lua脚本扫描路径

• 增加环境变量读取&＃xff0c;如&＃xff1a;SW_SERVICE_NAME、SW_SERVICE_INSTANCE_NAME、SW_BACKEND_SERVERS

• 添加Tracing使用的缓存tracing_buffer

• 设置Skywalking Lua Agent的初始化方法&＃xff0c;并将相关配置从环境变量中提取。

• 设置http节点的追踪配置。

# Skywalking ENV
env SW_SERVICE_NAME;
env SW_SERVICE_INSTANCE_NAME;
env SW_BACKEND_SERVERS;events {multi_accept {{ if $cfg.EnableMultiAccept }}on{{ else }}off{{ end }};worker_connections {{ $cfg.MaxWorkerConnections }};use epoll;
}http {# 引入lua脚本lua_package_path "/etc/nginx/lua/?.lua;/etc/nginx/lua/skywalking/?.lua;;";# 使用缓存lua_shared_dict tracing_buffer 100m;{{ buildLuaSharedDictionaries $cfg $servers }}init_by_lua_block {collectgarbage("collect")-- init moduleslocal ok, resok, res &＃61; pcall(require, "lua_ingress")if not ok thenerror("require failed: " .. tostring(res))elselua_ingress &＃61; reslua_ingress.set_config({{ configForLua $all }})endok, res &＃61; pcall(require, "configuration")if not ok thenerror("require failed: " .. tostring(res))elseconfiguration &＃61; resendok, res &＃61; pcall(require, "balancer")if not ok thenerror("require failed: " .. tostring(res))elsebalancer &＃61; resend{{ if $all.EnableMetrics }}ok, res &＃61; pcall(require, "monitor")if not ok thenerror("require failed: " .. tostring(res))elsemonitor &＃61; resend{{ end }}ok, res &＃61; pcall(require, "certificate")if not ok thenerror("require failed: " .. tostring(res))elsecertificate &＃61; rescertificate.is_ocsp_stapling_enabled &＃61; {{ $cfg.EnableOCSP }}endok, res &＃61; pcall(require, "plugins")if not ok thenerror("require failed: " .. tostring(res))elseplugins &＃61; resend-- load all plugins that&＃39;ll be used hereplugins.init({ {{ range $idx, $plugin :&＃61; $cfg.Plugins }}{{ if $idx }},{{ end }}{{ $plugin | quote }}{{ end }} })}init_worker_by_lua_block {
......-- Skywalkinglocal metadata_buffer &＃61; ngx.shared.tracing_buffermetadata_buffer:set(&＃39;serviceName&＃39;, os.getenv("SW_SERVICE_NAME"))metadata_buffer:set(&＃39;serviceInstanceName&＃39;, os.getenv("SW_SERVICE_INSTANCE_NAME"))-- set random seedrequire("swutil").set_randomseed()require("client"):startBackendTimer(os.getenv("SW_BACKEND_SERVERS"))}
...... rewrite_by_lua_block {lua_ingress.rewrite({{ locationConfigForLua $location $all }})balancer.rewrite()plugins.run()-- Skywalkingrequire("tracer"):start({{ buildUpstreamName $location | quote }})}
...... body_filter_by_lua_block {-- Skywalkingif ngx.arg[2] thenrequire("tracer"):finish()end}
...... log_by_lua_block {balancer.log(){{ if $all.EnableMetrics }}monitor.call(){{ end }}plugins.run()-- Skywalkingrequire("tracer"):prepareForReport()}
.......

&＃xff08;4&＃xff09;将这些脚本添加到nginx-ingress镜像中&＃xff0c;Dockerfile如下&＃xff1a;

FROM quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0ADD --chown&＃61;www-data nginx.tmpl /etc/nginx/template
ADD --chown&＃61;www-data skywalking /etc/nginx/lua/skywalking

&＃xff08;5&＃xff09;构建镜像并上传仓库

$ docker build -t registry.cn-hangzhou.aliyuncs.com/rookieops/ingress-nginx-controller-skywalking:0.34.1 .
$ docker push registry.cn-hangzhou.aliyuncs.com/rookieops/ingress-nginx-controller-skywalking:0.34.1

&＃xff08;6&＃xff09;修改nginx-ingress的deployment文件&＃xff0c;主要增加以下环境变量

......containers:- name: controllerimage: registry.cn-hangzhou.aliyuncs.com/rookieops/nginx-ingress-controller:0.32.0imagePullPolicy: IfNotPresent
......- name: SW_SERVICE_NAMEvalue: Kubernetes Ingress- name: SW_BACKEND_SERVERSvalue: http://skywalking-oap.skywalking.svc.cluster.local:12800- name: SW_SERVICE_INSTANCE_NAMEvalueFrom:fieldRef:fieldPath: metadata.uid
......

&＃xff08;7&＃xff09;然后重新部署ingress-controller应用。

然后可以在skywalking的面板上看到了。

image.png

已将所需的代码都放在github了&＃xff0c;仓库地址&＃xff1a;https://github.com/sunsharing-note/skywalking-ingress.git

参考&＃xff1a;

• https://github.com/lipangeng/Skywalking-Ingress-Overlay

• https://github.com/apache/skywalking-nginx-lua

公众号&＃xff1a;运维开发故事

github&＃xff1a;https://github.com/orgs/sunsharing-note/dashboard

爱生活&＃xff0c;爱运维

如果你觉得文章还不错&＃xff0c;就请点击右上角选择发送给朋友或者转发到朋友圈。您的支持和鼓励是我最大的动力。喜欢就请关注我吧~

扫码二维码

关注我&＃xff0c;不定期维护优质内容

温馨提示

如果你喜欢本文&＃xff0c;请分享到朋友圈&＃xff0c;想要获得更多信息&＃xff0c;请关注我。

........................