packetbeat扩展oracle,简单使用packetbeat

在前面两篇文章中记录了使用logstash来收集mysql的慢查询日志&＃xff0c;而后经过kibana以web的方式展现出来&＃xff0c;但在生产环境中&＃xff0c;需求会更复杂一些&＃xff0c;并且经过logstash写正则&＃xff0c;实在是个费时费劲的事。例如在生产环境中会有要求分析某个时间段mysql或者mongodb的慢查询日志状况&＃xff1b;还有I/O吞吐量&＃xff1b;这个时间段内常常执行的查询语句&＃xff0c;http访问状况等信息&＃xff1b;而后将分析出来的结果以图表的形式展示出来。听起来是否是有点头晕&＃xff0c;有点高大上的感受&＃xff0c;其实经过packetbeat&＃xff0c;一切将变得简单高效。本文介绍使用packetbeat&＃xff0c;elasticsearch,kibana实现这个需求。mysql

操做系统版本&＃xff1a;centos6.6 64bitgit

Elasticsearch版本&＃xff1a;elasticsearch-2.1.0.tar.gzgithub

Kibana版本&＃xff1a;Kibana 4.2.1web

Packetbeat版本&＃xff1a;packetbeat-1.0.0-1.x86_64redis

Topbeat版本&＃xff1a;topbeat-1.0.0-x86_64(topbeat实际上是用来收集操做系统信息的)sql

在前两篇文章中未介绍若是安装elasticsearch和kibana&＃xff0c;这个其实很简单&＃xff0c;基本下载下来解压一下&＃xff0c;稍微修改一下配置文件便可运行起来&＃xff0c;全部就忽略了&＃xff0c;若是有问题&＃xff0c;能够自行百度或者bing一下。mongodb

目前packetbeat支持的网络协议有http,mysql,postgresql,redis,mongodb和thrift。Packetet支持pcap&＃xff0c;pf_ring等抓包方式&＃xff0c;采用哪一种方式进行抓包&＃xff0c;则须要安装相应的依赖包。json

一&＃xff1a;下载并安装packetbeatcentos

# yum -y install libpcap

# rpm -ivh https://download.elastic.co/beats/packetbeat/packetbeat-1.0.0-x86_64.rpm

# rpm -ivh https://download.elastic.co/beats/topbeat/topbeat-1.0.0-x86_64.rpm

二&＃xff1a;向elasticsearch导入packetbeat模板网络

# curl -XPUT

&＃39;http://192.168.1.226:9200/_template/packetbeat&＃39; -d&＃64;/etc/packetbeat/packetbeat.template.json

三&＃xff1a;修改packetbeat配置文件

# cat /etc/packetbeat/packetbeat.yml  --server15

shipper:

name: server15

tags: ["server15"]

interfaces:

device: any

type: pcap

buffer_size_mb: 100

protocols:

mysql:

ports: [3306]

output:

elasticsearch:

host: 192.168.1.207

port: 9200

enabled: true

# cat /etc/packetbeat/packetbeat.yml  --server226

shipper:

name: server226

tags: ["server226"]

interfaces:

device: eth0

type: pcap

buffer_size_mb: 100

protocols:

mongodb:

ports: [37017, 38017]

send_request: true     # index the request payload

send_response: true    # index the response payload

max_docs: 10           # maximum number of documents to index per request/response

max_doc_length: 1024   # maximum document size to index

protocols:

mysql:

ports: [3306]

protocols:

redis:

ports: [6379]

output:

elasticsearch:

enabled: true

host: 192.168.1.207

port: 9200

四&＃xff1a;启动packetbeat服务

# /etc/init.d/packetbeat start

五&＃xff1a;导入packetbeat-dashboards

# git clone https://github.com/elastic/packetbeat-dashboards

# cd packetbeat-dashboards

# sh load.sh -url http://192.168.1.207:9200

六&＃xff1a;web展现

1: 配置索引&＃xff0c;这个在执行完load.sh脚本以后&＃xff0c;索引会自动建立

2: 查看客户端的数据推送状况

4: 图形展现,点击dashboard-load save dashboards

Mysql状况&＃xff1a;

Mongodb状况

汇总状况&＃xff1a;

七&＃xff1a;故障排错

1: 在测试过程当中曾经发现mysql里面的most frequent Mysql queries和slowest mysql queries数据显示不全&＃xff0c;像是被截断的样子&＃xff0c;排查后发现实际上是模板的问题&＃xff0c;删除模板后从新导入便可.

# curl -XDELETE &＃39;http://192.168.1.207:9200/*&＃39;

# curl -XPUT

&＃39;http://192.168.1.207:9200/_template/packetbeat&＃39; -d&＃64;/etc/packetbeat/packetbeat.template.json

# cd packetbeat-dashboards

# sh load.sh -url http://192.168.1.207:9200

2: elasticsearch数据维护

搜索数据&＃xff1a;(若是你有多个索引&＃xff0c;能够把packetbeat-*换成对应的索引名)&＃xff1a;

# curl -XGET &＃39;http://192.168.1.226:9200/packetbeat-*/_search?pretty&＃39;

删除数据(若是你有多个索引&＃xff0c;能够把packetbeat-*换成对应的索引名)&＃xff1a;

# curl -XDELETE &＃39;http://192.168.1.207:9200/packetbeat-*&＃39;