作者：u44093631 | 来源：互联网 | 2023-07-16 10:00
一、ELK概述1、需求背景业务发展越来越庞大、服务器越来越多各种访问日志、应用日志、错误日志量越来越多开发人员排查问题，需要服务器上查日志，不方便运营人员需要一些数据，
一、ELK概述
1、需求背景
- 业务发展越来越庞大、服务器越来越多
- 各种访问日志、应用日志、错误日志量越来越多
- 开发人员排查问题，需要服务器上查日志，不方便
- 运营人员需要一些数据，需要运维到服务器上分析日志
2、ELK介绍
ELK Stack包含：ElasticSearch、Logstash、Kibana。（ELK Stack 5.0版本以后-->Elastic Stack == ELK Stack+Beats）
ElasticSearch是一个搜索引擎，用来搜索、分析、存储日志。它是分布式的，也就是说可以横向扩容，可以自动发现，索引自动分片，总之很强大。
Logstash用来采集日志，把日志解析为Json格式交给ElasticSearch。
Kibana是一个数据可视化组件，把处理后的结果通过WEB界面展示。
Beats是一个轻量级日志采集器，其实Beats家族有5个成员。（早起的Logstash对性能资源消耗比较高，Beats性能和消耗可以忽略不计）
X-pach对Elastic Stack提供了安全、警报、监控、报表、图标于一身的扩展包，收费。
官网：https://www.elastic.co/cn/
中文文档：https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html
3、ELK架构
二、ELK安装
1、环境
2、安装配置
（1）、安装ElasticSearch
#安装
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm
yum localinstall elasticsearch-6.2.3.rpm
#配置
vim /etc/elasticsearch/elasticsearch.yml
network.host: 0.0.0.0
http.port: 9200
vim /etc/sysconfig/elasticsearch
JAVA_HOME=/usr/local/jdk1.8.0_131
#启动
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
#检查
curl '192.168.1.202:9200/_cluster/state?pretty'
{
"cluster_name" : "elasticsearch",
"compressed_size_in_bytes" : 229,
"version" : 2,
"state_uuid" : "GNdae66nQUOQ773XdbbdSQ",
"master_node" : "5jQNETNwQp6d6eXJYIwiCg",
"blocks" : { },
"nodes" : {
"5jQNETNwQp6d6eXJYIwiCg" : {
"name" : "5jQNETN",
"ephemeral_id" : "zAXkd9hnRxKcLJ225NoRyA",
"transport_address" : "192.168.1.202:9300",
"attributes" : { }
}
},
"metadata" : {
"cluster_uuid" : "oyhPKJf-QtiRzxha5jy14Q",
"templates" : { },
"indices" : { },
"index-graveyard" : {
"tombstones" : [ ]
}
},
"routing_table" : {
"indices" : { }
},
"routing_nodes" : {
"unassigned" : [ ],
"nodes" : {
"5jQNETNwQp6d6eXJYIwiCg" : [ ]
}
},
"snapshot_deletions" : {
"snapshot_deletions" : [ ]
},
"restore" : {
"snapshots" : [ ]
},
"snapshots" : {
"snapshots" : [ ]
}
}
（2）、安装Kibana
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-x86_64.rpm
vim /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://localhost:9200"
logging.dest: /var/log/kibana.log
touch /var/log/kibana.log ；chmod 777 /var/log/kibana.log
systemctl enable kibana
systemctl start kibana
（3）、安装Logstash
yum localinstall logstash-6.2.3.rpm #Logstash不支持JAVA9
chown -R logstash:root /var/log/logstash /var/lib/logstash
systemctl enable logstash
systemctl start logstash
3、坑
（1）：JAVA环境丢失
elasticsearch: could not find java
解决：
vim /etc/sysconfig/elasticsearch
JAVA_HOME=/usr/local/jdk1.8.0_131
（2）：缺少jar包，可能安装包有问题
error: unpacking of archive failed on file /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.0.4/vendor/jar-dependencies/runtime-jars/log4j-api-2.8.2.jar;5ab9a80b: cpio: read
解决：
yum install logstash
https://www.elastic.co/guide/en/logstash/6.2/installing-logstash.html #官网yum安装文档
（3）：JAVA环境丢失
/usr/share/logstash/vendor/jruby/bin/jruby: line 401: /usr/bin/java: No such file or directory
解决：
ln -s /usr/local/jdk1.8.0_131/bin/java /usr/bin/java
（4）：Logstash无法启动，或没有日志输出
[2018-03-27T13:27:33,839][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:448:in `validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:230:in `validate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:141:in `block in validate_all'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in `validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:264:in `execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:219:in `run'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bootstrap/environment.rb:67:in `'"]}
[2018-03-27T13:27:33,843][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit
解决：
chown -R logstash /var/log/logstash /var/lib/logstash
（5）：Logstash没有centos6启动脚本
解决：
/usr/share/logstash/bin/system-install /etc/logstash/startup.options sysv
京公网安备 11010802041100号