热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

【Grafana】通过阿里云日志服务监控Nginx访问日志显示统计信息

【Grafana】通过阿里云日志服务监控Nginx访问日志显示统计信息,Go语言社区,Golang程序员人脉社
Grafana通过阿里云日志服务监控Nginx访问日志显示统计信息

前言

通过之前的文章,我们做到了Grafana的安装部署和初期的设置,没有了解过的同学可以参考下面的文章链接

https://blog.csdn.net/diandianxiyu_geek/article/details/97013330

https://blog.csdn.net/diandianxiyu_geek/article/details/96577082

接下来要做的是数据可视化的一个应用,nginx访问文件的监控,输出图表。

实操

修改nginx配置文件

如果你没有进行额外的配置,那么你的nginx日志是这样的

101.254.141.164 - - [19/Jul/2019:11:22:03 +0800] "GET /online/cart/info HTTP/1.1" 200 120 "https://servicewechat.com/wx855bd11a98860d4e/devtools/page-frame.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92 Safari/601.1 wechatdevtools/1.02.1905151 MicroMessenger/6.7.3 Language/zh_CN webview/"
101.254.141.164 - - [19/Jul/2019:11:22:03 +0800] "GET /online/item/new?page=1&count=99 HTTP/1.1" 200 469 "https://servicewechat.com/wx855bd11a98860d4e/devtools/page-frame.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92 Safari/601.1 wechatdevtools/1.02.1905151 MicroMessenger/6.7.3 Language/zh_CN webview/"
101.254.141.164 - - [19/Jul/2019:11:22:03 +0800] "GET /online/item/list?page=1&count=10 HTTP/1.1" 200 618 "https://servicewechat.com/wx855bd11a98860d4e/devtools/page-frame.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92 Safari/601.1 wechatdevtools/1.02.1905151 MicroMessenger/6.7.3 Language/zh_CN webview/“

接下来修改配置文件,让日志记录更多信息

找到你的nginx

[root@VM_0_15_centos conf]# whereis  nginx
nginx: /usr/local/nginx /usr/local/nginx/sbin/nginx

找到配置文件

[root@VM_0_15_centos conf]# cd /usr/local/nginx/conf
[root@VM_0_15_centos conf]# ls
fastcgi.conf          fastcgi_params          koi-utf  mime.types          nginx.conf     nginx.conf.default  scgi_params          uwsgi_params          vhost
fastcgi.conf.default  fastcgi_params.default  koi-win  mime.types.default  nginx.conf_bk  proxy.conf          scgi_params.default  uwsgi_params.default  win-utf
[root@VM_0_15_centos conf]# cat nginx.conf
user www www;
worker_processes auto;
error_log /data/wwwlogs/error_nginx.log crit;
pid /var/run/nginx.pid;
worker_rlimit_nofile 51200;
events {
  use epoll;
  worker_connections 51200;
  multi_accept on;
}
http {
  include mime.types;
  default_type application/octet-stream;
  server_names_hash_bucket_size 128;
  client_header_buffer_size 32k;
  large_client_header_buffers 4 32k;
  client_max_body_size 1024m;
  client_body_buffer_size 10m;
  sendfile on;
  tcp_nopush on;
  keepalive_timeout 120;
  server_tokens off;
  tcp_nodelay on;
  fastcgi_connect_timeout 300;
  fastcgi_send_timeout 300;
  fastcgi_read_timeout 300;
  fastcgi_buffer_size 64k;
  fastcgi_buffers 4 64k;
  fastcgi_busy_buffers_size 128k;
  fastcgi_temp_file_write_size 128k;
  fastcgi_intercept_errors on;
 #注意这部分是新增的,日志配置
  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" $http_host '
                        '$status $request_length $body_bytes_sent "$http_referer" '
                        '"$http_user_agent"  $request_time $upstream_response_time';
  #Gzip Compression
  gzip on;
  gzip_buffers 16 8k;
  gzip_comp_level 6;
  gzip_http_version 1.1;
  gzip_min_length 256;
  gzip_proxied any;
  gzip_vary on;
  gzip_types
    text/xml application/xml application/atom+xml application/rss+xml application/xhtml+xml image/svg+xml
    text/Javascript application/Javascript application/x-Javascript
    text/x-json application/json application/x-web-app-manifest+json
    text/css text/plain text/x-component
    font/opentype application/x-font-ttf application/vnd.ms-fontobject
    image/x-icon;
  gzip_disable "MSIE [1-6].(?!.*SV1)";
  ##If you have a lot of static files to serve through Nginx then caching of the files' metadata (not the actual files' contents) can save some latency.
  #open_file_cache max=1000 inactive=20s;
  #open_file_cache_valid 30s;
  #open_file_cache_min_uses 2;
  #open_file_cache_errors on;
######################## default ############################
  server {
    listen 80;
    server_name _;
    access_log /data/wwwlogs/access_nginx.log combined;
    root /data/wwwroot/default;
    index index.html index.htm index.php;
    #error_page 404 /404.html;
    #error_page 502 /502.html;
    location /nginx_status {
      stub_status on;
      access_log off;
      allow 127.0.0.1;
      deny all;
    }
    location ~ [^/].php(/|$) {
      #fastcgi_pass remote_php_ip:9000;
      fastcgi_pass unix:/dev/shm/php-cgi.sock;
      fastcgi_index index.php;
      include fastcgi.conf;
    }
    location ~ .*.(gif|jpg|jpeg|png|bmp|swf|flv|mp4|ico)$ {
      expires 30d;
      access_log off;
    }
    location ~ .*.(js|css)?$ {
      expires 7d;
      access_log off;
    }
    location ~ ^/(.user.ini|.ht|.git|.svn|.project|LICENSE|README.md) {
      deny all;
    }
  }
########################## vhost #############################
  include vhost/*.conf;
}

在上面的配置文件中我已经把日志配置加了进去,为了方便辨认再贴一遍

在http中添加对应日志配置->

 log_format  main  '$remote_addr - $remote_user [$time_local] "$request" $http_host '
                        '$status $request_length $body_bytes_sent "$http_referer" '
                        '"$http_user_agent"  $request_time $upstream_response_time’;

上面的配置的意思是增加一个main的日志配置

接下来将nginx.conf的server中的access_log /data/wwwlogs/access_nginx.log combined;
combined 替换为main,combined为nginx内置的不需要配置的默认文件格式。

vhost里的所有文件也要进行替换。

重启nginx

再查看日志,就变成了

101.254.141.164 - - [19/Jul/2019:11:38:36 +0800] "GET /online/item/list?page=1&count=10 HTTP/1.1" api4621d373.henibox.com 200 587 619 "https://servicewechat.com/wx855bd11a98860d4e/0/page-frame.html" "Mozilla/5.0 (Linux; Android 8.0.0; MIX Build/OPR1.170623.032; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/68.0.3440.91 Mobile Safari/537.36 MicroMessenger/7.0.5.1440(0x27000537) Process/appbrand2 NetType/WIFI Language/zh_CN"  0.351 0.351
101.254.141.164 - - [19/Jul/2019:11:38:36 +0800] "GET /online/pick/list?page=1&count=4 HTTP/1.1" api4621d373.henibox.com 200 586 634 "https://servicewechat.com/wx855bd11a98860d4e/0/page-frame.html" "Mozilla/5.0 (Linux; Android 8.0.0; MIX Build/OPR1.170623.032; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/68.0.3440.91 Mobile Safari/537.36 MicroMessenger/7.0.5.1440(0x27000537) Process/appbrand2 NetType/WIFI Language/zh_CN"  0.389 0.389

为了防止日志系统识别格式问题,可以将之前的日志文件转移到其他文件夹作为备份

nginx日志接入阿里云日志系统

在阿里云日志系统中添加一个项目,在项目中添加一个Logstore,然后接入配置向导

选择nginx访问日志
在这里插入图片描述
填写的日志格式就是我们之前修改的日志格式

在这里插入图片描述

选择需要应用的机器,这里作为测试选择测试机器

下一步就会自动识别文本,匹配对应的字段,并增加索引进行统计

在这里插入图片描述

到这一步,在阿里云日志服务的部分就完成了。

阿里云日志系统接入Grafana

首先接入数据源,安装插件

cd /var/lib/grafana/plugins
git clone https://github.com/aliyun/aliyun-log-grafana-datasource-plugin

在这里插入图片描述

查看新增一个数据源

在这里插入图片描述

下一步开始配置

在这里插入图片描述

然后配置变量,配置变量用于在仪表盘页面公共选择时间段

我们用简单粗暴的引入方式,先直接引入阿里云官方给出的demo

{
  "annotations": {
    "list": []
  },
  "description": "日志服务demo",
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "hideControls": false,
  "id": 2,
  "links": [],
  "rows": [
    {
      "collapse": false,
      "height": "250px",
      "panels": [
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "logservice",
          "fill": 1,
          "id": 3,
          "legend": {
            "avg": false,
            "current": false,
            "max": false,
            "min": false,
            "rightSide": false,
            "show": true,
            "total": false,
            "values": false
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [
            {
              "alias": "pv",
              "yaxis": 2
            }
          ],
          "spaceLength": 10,
          "span": 3,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname| select approx_distinct(remote_addr) as uv ,count(1) as pv , __time__ - __time__%$$myinterval  as time group by __time__ - __time__%$$myinterval  order by time limit 1000",
              "refId": "A",
              "target": "uv,pv",
              "type": "timeserie",
              "xcol": "time",
              "ycol": "uv,pv"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "UV & PV",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "decimals": null,
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "decimals": null,
              "format": "short",
              "label": "",
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "logservice",
          "fill": 1,
          "id": 6,
          "legend": {
            "avg": false,
            "current": false,
            "max": false,
            "min": false,
            "show": true,
            "total": false,
            "values": false
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 3,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select sum(body_byte_sent) as net_out, sum(request_length) as net_in  ,__time__ - __time__ % $$myinterval  as time group by __time__ - __time__ % $$myinterval limit 10000",
              "refId": "A",
              "target": "net_in,net_out",
              "type": "timeserie",
              "xcol": "time",
              "ycol": "net_in,net_out"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "入网、出网带宽",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "columns": [],
          "datasource": "logservice",
          "fontSize": "100%",
          "id": 15,
          "links": [],
          "pageSize": null,
          "scroll": true,
          "showHeader": true,
          "sort": {
            "col": 2,
            "desc": true
          },
          "span": 3,
          "styles": [
            {
              "alias": "Time",
              "dateFormat": "YYYY-MM-DD HH:mm:ss",
              "pattern": "Time",
              "type": "hidden"
            },
            {
              "alias": "",
              "colorMode": null,
              "colors": [
                "rgba(245, 54, 54, 0.9)",
                "rgba(237, 129, 40, 0.89)",
                "rgba(50, 172, 45, 0.97)"
              ],
              "decimals": 2,
              "pattern": "/.*/",
              "thresholds": [],
              "type": "number",
              "unit": "short"
            }
          ],
          "targets": [
            {
              "bucketAggs": [],
              "query": "* | select diff[1] as today, diff[2] as yestoday, concat( cast( round((diff[3] - 1)*100,2) as varchar),'%')  as growth from(select compare(pv, 86400) as diff  from(select count(1) as pv from log))",
              "refId": "A",
              "target": "today,yestoday,growth",
              "type": "timeserie",
              "xcol": "table",
              "ycol": "today,yestoday,growth"
            }
          ],
          "title": "环比比较",
          "transform": "timeseries_to_columns",
          "type": "table"
        },
        {
          "cacheTimeout": null,
          "colorBackground": false,
          "colorValue": false,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "datasource": "logservice",
          "format": "none",
          "gauge": {
            "maxValue": 100,
            "minValue": 0,
            "show": false,
            "thresholdLabels": false,
            "thresholdMarkers": true
          },
          "id": 17,
          "interval": null,
          "links": [],
          "mappingType": 1,
          "mappingTypes": [
            {
              "name": "value to text",
              "value": 1
            },
            {
              "name": "range to text",
              "value": 2
            }
          ],
          "maxDataPoints": 100,
          "nullPointMode": "connected",
          "nullText": null,
          "postfix": "",
          "postfixFontSize": "50%",
          "prefix": "",
          "prefixFontSize": "50%",
          "rangeMaps": [
            {
              "from": "null",
              "text": "N/A",
              "to": "null"
            }
          ],
          "span": 3,
          "sparkline": {
            "fillColor": "rgba(31, 118, 189, 0.18)",
            "full": false,
            "lineColor": "rgb(31, 120, 193)",
            "show": false
          },
          "tableColumn": "",
          "targets": [
            {
              "bucketAggs": [],
              "query": "* | select count(1) as total_pv",
              "refId": "A",
              "target": "total_pv",
              "type": "timeserie",
              "ycol": "total_pv"
            }
          ],
          "thresholds": "",
          "title": "总pv",
          "type": "singlestat",
          "valueFontSize": "80%",
          "valueMaps": [
            {
              "op": "=",
              "text": "N/A",
              "value": "null"
            }
          ],
          "valueName": "avg"
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 207,
      "panels": [
        {
          "aliasColors": {},
          "cacheTimeout": null,
          "combine": {
            "label": "Others",
            "threshold": 0
          },
          "datasource": "logservice",
          "fontSize": "80%",
          "format": "short",
          "id": 2,
          "interval": null,
          "legend": {
            "show": true,
            "values": true
          },
          "legendType": "Under graph",
          "links": [],
          "maxDataPoints": 3,
          "nullPointMode": "connected",
          "pieType": "pie",
          "span": 2,
          "strokeWidth": 1,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select count(1) as pv ,method group by method",
              "refId": "A",
              "target": "method,pv",
              "type": "timeserie",
              "xcol": "pie",
              "ycol": "method,pv"
            }
          ],
          "title": "HTTP方法汇总",
          "type": "grafana-piechart-panel",
          "valueName": "current"
        },
        {
          "aliasColors": {},
          "cacheTimeout": null,
          "combine": {
            "label": "Others",
            "threshold": 0
          },
          "datasource": "logservice",
          "fontSize": "80%",
          "format": "short",
          "id": 5,
          "interval": null,
          "legend": {
            "show": true,
            "values": true
          },
          "legendType": "Under graph",
          "links": [],
          "maxDataPoints": 3,
          "nullPointMode": "connected",
          "pieType": "pie",
          "span": 2,
          "strokeWidth": 1,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select count(1) as pv ,status group by status",
              "refId": "A",
              "target": "status,pv",
              "type": "timeserie",
              "xcol": "pie",
              "ycol": "status,pv"
            }
          ],
          "title": "HTTP状态码",
          "type": "grafana-piechart-panel",
          "valueName": "current"
        },
        {
          "aliasColors": {},
          "cacheTimeout": null,
          "combine": {
            "label": "Others",
            "threshold": 0
          },
          "datasource": "logservice",
          "fontSize": "80%",
          "format": "short",
          "id": 8,
          "interval": null,
          "legend": {
            "show": true,
            "values": true
          },
          "legendType": "Under graph",
          "links": [],
          "maxDataPoints": 3,
          "nullPointMode": "connected",
          "pieType": "pie",
          "span": 4,
          "strokeWidth": 1,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select count(1) as pv , http_referer  group by http_referer order by pv desc",
              "refId": "A",
              "target": "referer,pv",
              "type": "timeserie",
              "xcol": "pie",
              "ycol": "http_referer,pv"
            }
          ],
          "title": "Top来源域名",
          "type": "grafana-piechart-panel",
          "valueName": "current"
        },
        {
          "aliasColors": {},
          "cacheTimeout": null,
          "combine": {
            "label": "Others",
            "threshold": 0
          },
          "datasource": "logservice",
          "fontSize": "80%",
          "format": "short",
          "id": 11,
          "interval": null,
          "legend": {
            "show": true,
            "values": true
          },
          "legendType": "Under graph",
          "links": [],
          "maxDataPoints": 3,
          "nullPointMode": "connected",
          "pieType": "pie",
          "span": 4,
          "strokeWidth": 1,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select count(1) as pv, case when http_user_agent like '%Android%' then 'Android' when http_user_agent like '%iPhone%' then 'iOS' else 'unKnown' end as http_user_agent  group by case when http_user_agent like '%Android%' then 'Android' when http_user_agent like '%iPhone%' then 'iOS' else 'unKnown' end   order by pv desc limit 10",
              "refId": "A",
              "target": "http_user_agent,pv",
              "type": "timeserie",
              "xcol": "pie",
              "ycol": "http_user_agent,pv"
            }
          ],
          "title": "客户端分布",
          "type": "grafana-piechart-panel",
          "valueName": "current"
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 362,
      "panels": [
        {
          "columns": [],
          "datasource": "logservice",
          "fontSize": "100%",
          "id": 9,
          "links": [],
          "pageSize": null,
          "scroll": true,
          "showHeader": true,
          "sort": {
            "col": 0,
            "desc": true
          },
          "span": 3,
          "styles": [
            {
              "alias": "Time",
              "dateFormat": "YYYY-MM-DD HH:mm:ss",
              "pattern": "Time",
              "type": "date"
            },
            {
              "alias": "",
              "colorMode": null,
              "colors": [
                "rgba(245, 54, 54, 0.9)",
                "rgba(237, 129, 40, 0.89)",
                "rgba(50, 172, 45, 0.97)"
              ],
              "decimals": 2,
              "pattern": "/.*/",
              "thresholds": [],
              "type": "number",
              "unit": "short"
            }
          ],
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select uri  as top_latency_url ,request_time order by request_time  desc limit 10",
              "refId": "A",
              "target": "top_latency_url,request_time",
              "type": "timeserie",
              "ycol": "top_latency_url,request_time"
            }
          ],
          "title": "延时最高的页面",
          "transform": "timeseries_to_columns",
          "type": "table"
        },
        {
          "columns": [],
          "datasource": "logservice",
          "fontSize": "100%",
          "id": 4,
          "links": [],
          "pageSize": null,
          "scroll": true,
          "showHeader": true,
          "sort": {
            "col": 0,
            "desc": false
          },
          "span": 4,
          "styles": [
            {
              "alias": "Time",
              "dateFormat": "YYYY-MM-DD HH:mm:ss",
              "pattern": "Time",
              "type": "date"
            },
            {
              "alias": "",
              "colorMode": null,
              "colors": [
                "rgba(245, 54, 54, 0.9)",
                "rgba(237, 129, 40, 0.89)",
                "rgba(50, 172, 45, 0.97)"
              ],
              "decimals": 2,
              "pattern": "/.*/",
              "thresholds": [],
              "type": "number",
              "unit": "short"
            }
          ],
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select count(1) as pv, split_part(uri,'?',1) as path  group by split_part(uri,'?',1) order by pv desc limit 20",
              "refId": "A",
              "target": "path,pv",
              "type": "timeserie",
              "ycol": "path,pv"
            }
          ],
          "title": "热门页面",
          "transform": "timeseries_to_columns",
          "type": "table"
        },
        {
          "columns": [],
          "datasource": "logservice",
          "fontSize": "100%",
          "id": 10,
          "links": [],
          "pageSize": null,
          "scroll": true,
          "showHeader": true,
          "sort": {
            "col": 0,
            "desc": false
          },
          "span": 5,
          "styles": [
            {
              "alias": "Time",
              "dateFormat": "YYYY-MM-DD HH:mm:ss",
              "pattern": "Time",
              "type": "date"
            },
            {
              "alias": "",
              "colorMode": null,
              "colors": [
                "rgba(245, 54, 54, 0.9)",
                "rgba(237, 129, 40, 0.89)",
                "rgba(50, 172, 45, 0.97)"
              ],
              "decimals": 2,
              "pattern": "/.*/",
              "thresholds": [],
              "type": "number",
              "unit": "short"
            }
          ],
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname not status:200| select count(1) as pv , uri  group by uri order by pv desc",
              "refId": "A",
              "target": "url,pv",
              "type": "timeserie",
              "ycol": "url,pv"
            }
          ],
          "title": "非200请求TOP页面",
          "transform": "timeseries_to_columns",
          "type": "table"
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 282,
      "panels": [
        {
          "aliasColors": {},
          "bars": false,
          "dashLength": 10,
          "dashes": false,
          "datasource": "logservice",
          "fill": 1,
          "id": 7,
          "legend": {
            "avg": false,
            "current": false,
            "max": false,
            "min": false,
            "show": true,
            "total": false,
            "values": false
          },
          "lines": true,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 3,
          "stack": false,
          "steppedLine": false,
          "targets": [
            {
              "bucketAggs": [],
              "query": "$hostname | select avg(request_time) as response_time, avg(upstream_response_time) as upstream_response_time  ,__time__ - __time__ % $$myinterval  as time group by __time__ - __time__ % $$myinterval limit 10000",
              "refId": "A",
              "target": "upstream_response_time,response_time",
              "type": "timeserie",
              "xcol": "time",
              "ycol": "upstream_response_time,response_time"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "前端平均延时和后端平均延时",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        },
        {
          "aliasColors": {},
          "cacheTimeout": null,
          "combine": {
            "label": "Others",
            "threshold": 0
          },
          "datasource": "logservice",
          "fontSize": "80%",
          "format": "short",
          "id": 12,
          "interval": null,
          "legend": {
            "show": true,
            "values": true
          },
          "legendType": "Under graph",
          "links": [],
          "maxDataPoints": 3,
          "nullPointMode": "connected",
          "pieType": "pie",
          "span": 5,
          "strokeWidth": 1,
          "targets": [
            {
              "bucketAggs": [],
              "query": "*| select ip_to_province(remote_addr) as province, count(1) as pv group by province order by pv desc  limit 10",
              "refId": "A",
              "target": "province,pv",
              "type": "timeserie",
              "xcol": "pie",
              "ycol": "province,pv"
            }
          ],
          "title": "不同省份PV",
          "type": "grafana-piechart-panel",
          "valueName": "current"
        },
        {
          "aliasColors": {},
          "cacheTimeout": null,
          "combine": {
            "label": "Others",
            "threshold": 0
          },
          "datasource": "logservice",
          "fontSize": "80%",
          "format": "short",
          "id": 13,
          "interval": null,
          "legend": {
            "show": true,
            "values": true
          },
          "legendType": "Under graph",
          "links": [],
          "maxDataPoints": 3,
          "nullPointMode": "connected",
          "pieType": "pie",
          "span": 4,
          "strokeWidth": 1,
          "targets": [
            {
              "bucketAggs": [],
              "query": "*| select ip_to_provider(remote_addr) as vendor, count(1) as pv group by vendor order by pv desc  limit 10",
              "refId": "A",
              "type": "timeserie",
              "xcol": "pie",
              "ycol": "vendor,pv"
            }
          ],
          "title": "网络供应商",
          "type": "grafana-piechart-panel",
          "valueName": "current"
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 250,
      "panels": [
        {
          "circleMaxSize": 30,
          "circleMinSize": 2,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "datasource": "logservice",
          "decimals": 0,
          "esMetric": "Count",
          "hideEmpty": false,
          "hideZero": false,
          "id": 14,
          "initialZoom": 1,
          "locationData": "countries",
          "mapCenter": "(0°, 0°)",
          "mapCenterLatitude": 0,
          "mapCenterLongitude": 0,
          "maxDataPoints": 1,
          "showLegend": true,
          "span": 6,
          "stickyLabels": false,
          "targets": [
            {
              "bucketAggs": [],
              "query": "* | select count(1) as pv , ip_to_country_code(remote_addr) as country group by country",
              "refId": "A",
              "type": "timeserie",
              "xcol": "bar",
              "ycol": "country,pv"
            }
          ],
          "thresholds": "0,10",
          "title": "Panel Title",
          "type": "grafana-worldmap-panel",
          "unitPlural": "",
          "unitSingle": "",
          "valueName": "total"
        },
        {
          "cards": {
            "cardPadding": null,
            "cardRound": null
          },
          "color": {
            "cardColor": "#b4ff00",
            "colorScale": "sqrt",
            "colorScheme": "interpolateOranges",
            "exponent": 0.5,
            "mode": "spectrum"
          },
          "dataFormat": "timeseries",
          "datasource": "logservice",
          "heatmap": {},
          "highlightCards": true,
          "id": 18,
          "legend": {
            "show": false
          },
          "links": [],
          "span": 6,
          "targets": [
            {
              "bucketAggs": [],
              "query": "* | select __time__ - __time__ % $$myinterval as t, method,count(1) as pv  group by  t, method  order by t limit 10000",
              "refId": "A",
              "target": "method#:#pv",
              "type": "timeserie",
              "xcol": "t",
              "ycol": "method#:#pv"
            }
          ],
          "title": "热力图演示,method热力图",
          "tooltip": {
            "show": true,
            "showHistogram": false
          },
          "type": "heatmap",
          "xAxis": {
            "show": true
          },
          "xBucketNumber": null,
          "xBucketSize": null,
          "yAxis": {
            "decimals": null,
            "format": "short",
            "logBase": 1,
            "max": null,
            "min": null,
            "show": true,
            "splitFactor": null
          },
          "yBucketNumber": null,
          "yBucketSize": null
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    },
    {
      "collapse": false,
      "height": 250,
      "panels": [
        {
          "content": "# 查看配置nn在本页面顶端,点击“share dashboard” 按钮  ->  Export    。 导出为json文件,该配置可以导入您的dashboard。",
          "id": 16,
          "links": [],
          "mode": "markdown",
          "span": 6,
          "title": "",
          "type": "text"
        },
        {
          "aliasColors": {},
          "bars": true,
          "dashLength": 10,
          "dashes": false,
          "datasource": "logservice",
          "fill": 1,
          "id": 19,
          "legend": {
            "avg": false,
            "current": false,
            "max": false,
            "min": false,
            "show": true,
            "total": false,
            "values": false
          },
          "lines": false,
          "linewidth": 1,
          "links": [],
          "nullPointMode": "null",
          "percentage": false,
          "pointradius": 5,
          "points": false,
          "renderer": "flot",
          "seriesOverrides": [],
          "spaceLength": 10,
          "span": 6,
          "stack": true,
          "steppedLine": false,
          "targets": [
            {
              "bucketAggs": [],
              "query": "* | select __time__ - __time__ % $$myinterval  as t, count(1) as pv , method group by t , method limit 10000",
              "refId": "A",
              "target": "method#:#pv",
              "type": "timeserie",
              "xcol": "t",
              "ycol": "method#:#pv"
            }
          ],
          "thresholds": [],
          "timeFrom": null,
          "timeShift": null,
          "title": "各个Method堆叠图",
          "tooltip": {
            "shared": true,
            "sort": 0,
            "value_type": "individual"
          },
          "type": "graph",
          "xaxis": {
            "buckets": null,
            "mode": "time",
            "name": null,
            "show": true,
            "values": []
          },
          "yaxes": [
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            },
            {
              "format": "short",
              "label": null,
              "logBase": 1,
              "max": null,
              "min": null,
              "show": true
            }
          ]
        }
      ],
      "repeat": null,
      "repeatIteration": null,
      "repeatRowId": null,
      "showTitle": false,
      "title": "Dashboard Row",
      "titleSize": "h6"
    }
  ],
  "schemaVersion": 14,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "auto": true,
        "auto_count": 30,
        "auto_min": "10s",
        "current": {
          "text": "30m",
          "value": "30m"
        },
        "hide": 0,
        "label": "time interval",
        "name": "myinterval",
        "options": [
          {
            "selected": false,
            "text": "auto",
            "value": "$__auto_interval"
          },
          {
            "selected": false,
            "text": "1m",
            "value": "1m"
          },
          {
            "selected": false,
            "text": "10m",
            "value": "10m"
          },
          {
            "selected": true,
            "text": "30m",
            "value": "30m"
          },
          {
            "selected": false,
            "text": "1h",
            "value": "1h"
          },
          {
            "selected": false,
            "text": "6h",
            "value": "6h"
          },
          {
            "selected": false,
            "text": "12h",
            "value": "12h"
          },
          {
            "selected": false,
            "text": "1d",
            "value": "1d"
          },
          {
            "selected": false,
            "text": "7d",
            "value": "7d"
          },
          {
            "selected": false,
            "text": "14d",
            "value": "14d"
          },
          {
            "selected": false,
            "text": "30d",
            "value": "30d"
          }
        ],
        "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d",
        "refresh": 2,
        "type": "interval"
      },
      {
        "allValue": null,
        "current": {
          "text": "*",
          "value": "*"
        },
        "hide": 0,
        "includeAll": false,
        "label": "域名",
        "multi": false,
        "name": "hostname",
        "options": [
          {
            "selected": true,
            "text": "*",
            "value": "*"
          },
          {
            "selected": false,
            "text": "www.host.com",
            "value": "www.host.com"
          },
          {
            "selected": false,
            "text": "www.host0.com",
            "value": "www.host0.com"
          },
          {
            "selected": false,
            "text": "www.host1.com",
            "value": "www.host1.com"
          }
        ],
        "query": "*,www.host.com,www.host0.com,www.host1.com",
        "type": "custom"
      }
    ]
  },
  "time": {
    "from": "now-24h",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ],
    "time_options": [
      "5m",
      "15m",
      "1h",
      "6h",
      "12h",
      "24h",
      "2d",
      "7d",
      "30d"
    ]
  },
  "timezone": "",
  "title": "Nginx访问统计",
  "version": 79
}

注意上面的json文件,千万不要直接复制,这里需要修改的地方有两个,
一个是数据源,一定要改成自己前面配置的数据源的名称,不能用这个默认的
还有就是变量的域名部分,在这里直接改成自己的域名,就不用再去配置里的变量去改了

注意,由于这个官方的版本比较老,配置之后还存在字段兼容性的问题,需要按下面的进行修改

  • method —> request_method
  • url -> request_uri
  • body_byte_sent

我们可以在仪表盘的配置界面直接修改json文件

在这里插入图片描述

经过修改json并保存之后,我们就得到了nginx访问日志的监控数据

总结

我认为理解Grafana的点在于他是一个时序概念的数据可视化工具。
以后我会分享如何用它做业务相关的数据展示。


推荐阅读
  • 字符串学习时间:1.5W(“W”周,下同)知识点checkliststrlen()函数的返回值是什么类型的?字 ... [详细]
  • 网站访问全流程解析
    本文详细介绍了从用户在浏览器中输入一个域名(如www.yy.com)到页面完全展示的整个过程,包括DNS解析、TCP连接、请求响应等多个步骤。 ... [详细]
  • 为了确保iOS应用能够安全地访问网站数据,本文介绍了如何在Nginx服务器上轻松配置CertBot以实现SSL证书的自动化管理。通过这一过程,可以确保应用始终使用HTTPS协议,从而提升数据传输的安全性和可靠性。文章详细阐述了配置步骤和常见问题的解决方法,帮助读者快速上手并成功部署SSL证书。 ... [详细]
  • 在配置Nginx的SSL证书后,虽然HTTPS访问能够正常工作,但HTTP请求却会遇到400错误。本文详细解析了这一问题,并提供了Nginx配置的具体示例。此外,还深入探讨了DNS服务器证书、SSL证书的申请与安装流程,以及域名注册、查询方法和CDN加速技术的应用,帮助读者全面了解相关技术细节。 ... [详细]
  • 在腾讯云服务器上部署Nginx的详细指南中,首先需要确保安装必要的依赖包。如果这些依赖包已安装,可直接跳过此步骤。具体命令包括 `yum -y install gcc gcc-c++ wget net-tools pcre-devel zlib-devel`。接下来,本文将详细介绍如何下载、编译和配置Nginx,以确保其在腾讯云服务器上顺利运行。此外,还将提供一些优化建议,帮助用户提升Nginx的性能和安全性。 ... [详细]
  • 来自微信官方:微信支付跨平台软件架构首次曝光
    大纲背景线上效果指标什么是软件架构为什么需要软件架构从零到一构建支付跨平台软件架构1.抽象业务流程2.加入路由机制3.管理网络请求4.规范数据传递总结背景作为一个重要业务ÿ ... [详细]
  • 安卓开发入门!BAT大厂面试基础题集合,顺利通过阿里Android岗面试
    其实不是Android不行了,而是你跟不上了我的很多读者都在反馈说,现在一个岗位可以收到的简历数,是前几年的几倍。我们必须承认ÿ ... [详细]
  • 大家好,我是李白。本文将分享一个从零开始的全栈项目,涵盖了设计、前端、后端和服务端的全面学习过程。通过这个项目,我希望能够帮助初学者更好地理解和掌握全栈开发的技术栈。 ... [详细]
  • 在优化Nginx与PHP的高效配置过程中,许多教程提供的配置方法存在诸多问题或不良实践。本文将深入探讨这些常见错误,并详细介绍如何正确配置Nginx和PHP,以实现更高的性能和稳定性。我们将从Nginx配置文件的基本指令入手,逐步解析每个关键参数的最优设置,帮助读者理解其背后的原理和实际应用效果。 ... [详细]
  • 深入解析 Vue 中的 Axios 请求库
    本文深入探讨了 Vue 中的 Axios 请求库,详细解析了其核心功能与使用方法。Axios 是一个基于 Promise 的 HTTP 客户端,支持浏览器和 Node.js 环境。文章首先介绍了 Axios 的基本概念,随后通过具体示例展示了如何在 Vue 项目中集成和使用 Axios 进行数据请求。无论你是初学者还是有经验的开发者,本文都能为你解决 Vue.js 相关问题提供有价值的参考。 ... [详细]
  • Linux下安装grafana并且添加influxdb监控的方法
    这篇文章主要介绍了Linux下安装grafana并且添加influxdb监控的方法,本文给大家介绍的非常详细,具有一定的参考借鉴价值, ... [详细]
  • 安卓端开源移动浏览器开源项目
    进入android开发以来,webview用的比较多,后来又看了一些基于webview的浏览器代码,慢慢的将积累的东西做成了一个安卓移动浏 ... [详细]
  • 资源:吊炸天!74款APP完整源码!android界面中点击输入框时弹出输入法如果输入框在底部会出现输入法遮挡输入内容的问题解决办法设置activity的windowsoftinpu ... [详细]
  • 01mui框架使用概述
    1MUI概述1.1MUI诞生背景?性能和体验的差距,一直是手机APP开发者放弃HTML5的首要原因。浏览器默认控件样式又少又丑,制作一 ... [详细]
  • redis 端口_使用Prometheus和Grafana构建Redis实时监控平台
    Redis作为缓存系统,在整个后端体系中是较为重要的一环,需要实时监控运行状态。现在有各种各样的工具都可以对Redis进行监控,例如&#x ... [详细]
author-avatar
兔子是不常吃萝卜的
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有