当前位置: 开发笔记 > 编程语言 > 正文

dockercompose安装ELK7.12.0并启用xpack及kibana设置日志报警

作者：muc4093631 | 来源：互联网 | 2023-09-10 11:30

ELKStack简介ELK是三个开源软件的缩写，分别为：Elasticsearch、Logstash以及Kibana，它们都是开源软件。不

ELK Stack 简介
ELK 是三个开源软件的缩写&＃xff0c;分别为&＃xff1a;Elasticsearch、Logstash 以及 Kibana&＃xff0c;它们都是开源软件。不过现在还新增了一个 Beats&＃xff0c;它是一个轻量级的日志收集处理工具&＃xff08;Agent&＃xff09;&＃xff0c;Beats 占用资源少&＃xff0c;适合于在各个服务器上搜集日志后传输给 Logstash&＃xff0c;官方也推荐此工具&＃xff0c;目前由于原本的 ELK Stack 成员中加入了 Beats 工具所以已改名为 Elastic Stack。

Elastic Stack 包含&＃xff1a;

Elasticsearch是个开源分布式搜索引擎&＃xff0c;提供搜集、分析、存储数据三大功能。它的特点有&＃xff1a;分布式&＃xff0c;零配置&＃xff0c;自动发现&＃xff0c;索引自动分片&＃xff0c;索引副本机制&＃xff0c;restful
风格接口&＃xff0c;多数据源&＃xff0c;自动搜索负载等。详细可参考 Elasticsearch 权威指南
Logstash 主要是用来日志的搜集、分析、过滤日志的工具&＃xff0c;支持大量的数据获取方式。一般工作方式为 c/s 架构&＃xff0c;client
端安装在需要收集日志的主机上&＃xff0c;server 端负责将收到的各节点日志进行过滤、修改等操作在一并发往 Elasticsearch 上去。
Kibana 也是一个开源和免费的工具&＃xff0c;Kibana 可以为 Logstash 和 ElasticSearch 提供的日志分析友好的 Web 界面&＃xff0c;可以帮助汇总、分析和搜索重要数据日志。
Beats 在这里是一个轻量级日志采集器&＃xff0c;其实 Beats 家族有 6 个成员&＃xff0c;早期的 ELK 架构中使用 Logstash收集、解析日志&＃xff0c;但是 Logstash 对内存、cpu、io 等资源消耗比较高。相比 Logstash&＃xff0c;Beats 所占系统的 CPU和内存几乎可以忽略不计。

ELK Stack &＃xff08;5.0版本之后&＃xff09;–> Elastic Stack &＃61;&＃61; &＃xff08;ELK Stack &＃43; Beats&＃xff09;。

目前 Beats 包含六种工具&＃xff1a;

Packetbeat&＃xff1a; 网络数据&＃xff08;收集网络流量数据&＃xff09;
Metricbeat&＃xff1a; 指标&＃xff08;收集系统、进程和文件系统级别的 CPU 和内存使用情况等数据&＃xff09;
Filebeat&＃xff1a; 日志文件&＃xff08;收集文件数据&＃xff09;
Winlogbeat&＃xff1a; windows 事件日志&＃xff08;收集 Windows 事件日志数据&＃xff09;
Auditbeat&＃xff1a;审计数据&＃xff08;收集审计日志&＃xff09;
Heartbeat&＃xff1a;运行时间监控&＃xff08;收集系统运行时的数据&＃xff09;

k8s的filebeat配置文件地址/home/k8s_filebeat
ip&＃xff08;uat-192.168.180.37,prd-192.168.192.10&＃xff09;
现部署地址192.168.181.52:/home/stack_elk
前提&＃xff1a;

1、数据来源于redis中的数据&＃xff08;filebeat推送到redis中&＃xff09;
2、centos7、Docker version 18.03.1-ce

一、创建以下compose和配置文件&＃xff08;es&＃43;kibana&＃xff09;&＃xff0c;传输层配置传输层安全性&＃xff08;TLS&＃xff09;加密

1、&＃xff08;基础文件准备开始&＃xff09;创建`instances.yml` 标识您需要为其创建证书的实例。

instances:- name: es01dns:- es01- localhostip:- 192.168.181.52- name: &＃39;kib01&＃39;dns:- kib01- localhost

2、创建`create-certs.yml` 用来生成Elasticsearch和Kibana的证书。

version: &＃39;2&＃39;services:create_certs:image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0#image: elk_es:v1container_name: create_certscommand: >bash -c &＃39;yum install -y -q -e 0 unzip;if [[ ! -f /certs/bundle.zip ]]; thenbin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;unzip /certs/bundle.zip -d /certs;fi;chown -R 1000:0 /certs&＃39;working_dir: /usr/share/elasticsearchvolumes:#- certs:/certs- ./certs:/certs- .:/usr/share/elasticsearch/config/certificatesnetworks:- elastic#volumes: # certs: # driver: localnetworks:elastic:driver: bridge

3、创建单节点es配置elastic-docker-tls.yml

其中包含单节点的Elasticsearch和一个启用了TLS的Kibana实例和一个logstash实例。

version: &＃39;2.1&＃39; services:es01:image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0container_name: es01restart: alwaysenvironment:- "discovery.type&＃61;single-node"- "ES_JAVA_OPTS&＃61;-Xms2048m -Xmx2048m"# 生成并应用支持传输层安全性的试用许可证。- xpack.license.self_generated.type&＃61;trial- xpack.security.enabled&＃61;true# 启用传输层安全性以加密客户端通信。- xpack.security.http.ssl.enabled&＃61;true- xpack.security.http.ssl.key&＃61;/usr/share/elasticsearch/config/certificates/es01/es01.key- xpack.security.http.ssl.certificate_authorities&＃61;/usr/share/elasticsearch/config/certificates/ca/ca.crt- xpack.security.http.ssl.certificate&＃61;/usr/share/elasticsearch/config/certificates/es01/es01.crt# 启用传输层安全性以加密节点间通信。- xpack.security.transport.ssl.enabled&＃61;true# 通过不需要主机名验证来允许使用自签名证书。- xpack.security.transport.ssl.verification_mode&＃61;certificate- xpack.security.transport.ssl.certificate_authorities&＃61;/usr/share/elasticsearch/config/certificates/ca/ca.crt- xpack.security.transport.ssl.certificate&＃61;/usr/share/elasticsearch/config/certificates/es01/es01.crt- xpack.security.transport.ssl.key&＃61;/usr/share/elasticsearch/config/certificates/es01/es01.key- "http.cors.enabled&＃61;true"- "http.cors.allow-origin&＃61;*"- "http.cors.allow-headers&＃61;Authorization,X-Requested-With,Content-Length,Content-Type"ulimits:memlock:soft: -1hard: -1volumes:- ./data01:/usr/share/elasticsearch/data- ./certs:/usr/share/elasticsearch/config/certificatesports:- 9200:9200networks:- elastichealthcheck:test: curl --cacert /usr/share/elasticsearch/config/certificates/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? &＃61;&＃61; 52 ]]; then echo 0; else echo 1; fi# 每次检查之间的间隔时间interval: 30s# 运行命令的超时时间timeout: 10s# 重试次数retries: 5es-head:image: mobz/elasticsearch-head:5container_name: elasticsearch-headrestart: alwaysports:- 9100:9100kib01:image: docker.elastic.co/kibana/kibana:7.12.0container_name: kib01restart: alwaysdepends_on: {"es01": {"condition": "service_healthy"}}ports:- 5601:5601environment:SERVERNAME: localhostELASTICSEARCH_URL: https://es01:9200ELASTICSEARCH_HOSTS: https://es01:9200ELASTICSEARCH_USERNAME: kibana_systemELASTICSEARCH_PASSWORD: n3nhaKiYczkls3AoXup6ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES: /usr/share/elasticsearch/config/certificates/ca/ca.crtSERVER_SSL_ENABLED: "true"SERVER_SSL_KEY: /usr/share/elasticsearch/config/certificates/kib01/kib01.keySERVER_SSL_CERTIFICATE: /usr/share/elasticsearch/config/certificates/kib01/kib01.crtvolumes:- ./certs:/usr/share/elasticsearch/config/certificates- ./kibana.yml:/usr/share/kibana/config/kibana.ymlnetworks:- elasticnetworks:elastic:driver: bridge

①kibana配置插件logtrail&＃xff08;如不需要此插件可以跳过①②③&＃xff09;

cd images/kibana/
下载对应的logtrail插件包&＃xff1a;

wget https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-7.9.1-0.1.31.zip

②創建kibana的Dockerfile

FROM docker.elastic.co/kibana/kibana:7.9.1 ADD ./logtrail-7.9.1-0.1.31.zip /opt/kibana/plugins/logtrail-7.9.1-0.1.3 RUN ./bin/kibana-plugin install file:///opt/kibana/plugins//logtrail-7.9.1-0.1.31.zip

③&＃xff08;基础文件准备结束&＃xff09;配置logtrail显示规则&＃xff08;logtrail.json&＃xff09;

cat logtrail.json

{"version": 2,"index_patterns": [{"es": {"default_index": "prod-*","allow_url_parameter": false,"timezone": "UTC"},"tail_interval_in_seconds": 10,"es_index_time_offset_in_seconds": 0,"display_timezone": "local","display_timestamp_format": "YYYY年MM月DD日 HH:mm:ss","max_buckets": 500,"nested_objects": false,"default_time_range_in_days": 5,"max_hosts": 100,"max_events_to_keep_in_viewer": 5000,"default_search": "","fields": {"mapping": {"timestamp": "&＃64;timestamp","program": "tags","hostname": "kubernetes.labels.name","message": "message"},"hostname_format": "{{{kubernetes.namespace}}} | {{{hostname}}}","message_format": "{{{kubernetes.namespace}}} | {{{message}}}","keyword_suffix": "keyword"}},{"es": {"default_index": "uat-*","allow_url_parameter": false,"timezone": "UTC"},"tail_interval_in_seconds": 10,"es_index_time_offset_in_seconds": 0,"display_timezone": "local","display_timestamp_format": "YYYY年MM月DD日 HH:mm:ss","max_buckets": 500,"nested_objects": false,"default_time_range_in_days": 5,"max_hosts": 100,"max_events_to_keep_in_viewer": 5000,"default_search": "","fields": {"mapping": {"timestamp": "&＃64;timestamp","program": "tags","hostname": "kubernetes.labels.name","message": "message"},"hostname_format": "{{{kubernetes.namespace}}} | {{{hostname}}}","message_format": "{{{kubernetes.namespace}}} | {{{message}}}","keyword_suffix": "keyword"}},{"es": {"default_index": "st-*","allow_url_parameter": false,"timezone": "UTC"},"tail_interval_in_seconds": 10,"es_index_time_offset_in_seconds": 0,"display_timezone": "local","display_timestamp_format": "YYYY年MM月DD日 HH:mm:ss","max_buckets": 500,"nested_objects": false,"default_time_range_in_days": 5,"max_hosts": 100,"max_events_to_keep_in_viewer": 5000,"default_search": "","fields": {"mapping": {"timestamp": "&＃64;timestamp","program": "tags","hostname": "attrs.service","message": "log"},"message_format": "{{{log}}} | {{{marker}}}","keyword_suffix": "keyword"}}] }

1.1、&＃xff08;开始&＃xff09;通过启动create-certs容器为Elasticsearch生成证书&＃xff1a;

[root&＃64;dev23 stack_elk]# docker-compose -f create-certs.yml run --rm create_certs

1.2、建立单节点的Elasticsearch集群&＃xff1a;

[root&＃64;dev23 stack_elk]# mkdir data01 [root&＃64;dev23 stack_elk]# chmod 777 data01 [root&＃64;dev23 stack_elk]# docker-compose -f elastic-docker-tls.yml up -d es01

1.3、运行该elasticsearch-setup-passwords工具为所有内置用户&＃xff08;包括该kibana_system用户&＃xff09;生成密码&＃xff0c;自动生成密码用auto&＃xff0c; 自己设置用 interactive。

[root&＃64;dev23 stack_elk]# docker exec es01 /bin/bash -c "bin/elasticsearch-setup-passwords auto --batch --url https://es01:9200" #为elastic用户设置密码后&＃xff0c;引导密码将不再有效。并且再次执行elasticsearch-setup-passwords命令会抛出异常 Failed to authenticate user &＃39;elastic&＃39; against https://es01:9200/_security/_authenticate?pretty Possible causes include:* The password for the &＃39;elastic&＃39; user has already been changed on this cluster* Your elasticsearch node is running against a different keystoreThis tool used the keystore at /usr/share/elasticsearch/config/elasticsearch.keystoreERROR: Failed to verify bootstrap password

1.4、`ELASTICSEARCH_PASSWORD`在`kibana-docker.yml`撰写文件中设置为`kibana_system`用户生成的密码。

kib01:image: docker.elastic.co/kibana/kibana:${VERSION}container_name: kib01depends_on: {"es01": {"condition": "service_healthy"}}ports:- 5601:5601environment:SERVERNAME: localhostELASTICSEARCH_URL: https://es01:9200ELASTICSEARCH_HOSTS: https://es01:9200ELASTICSEARCH_USERNAME: kibana_system# 修改此处的密码ELASTICSEARCH_PASSWORD: CHANGEME...

1.5、使用docker-compose重启es和Kibana&＃xff1a;

[root&＃64;dev23 stack_elk]# docker-compose -f elastic-docker-tls.yml up -d

1.6、访问kibana《https://192.168.181.52:5601》

使用elastic用户登录kibana

二、配置 Logstash 启用 TLS连接es

cd logstash/
以下配置文件中涉及到用户名和密码需要引用1.3步骤中所生成的密码

2.1、配置Logstash的docker-compose.yml

version: &＃39;2&＃39; services:logstash:container_name: logstash#image: logstash_elk:v1image: logstash:7.12.0restart: alwaysports:- 5044:5044volumes:- ./config/logstash.conf:/usr/share/logstash/pipeline/logstash.conf- ./config/logstash.yml:/usr/share/logstash/config/logstash.yml- ../certs/ca/:/etc/logstash/config/certs/

2.2、创建配置文件logstash.conf

cat config/logstash.conf

input {redis {host &＃61;> "192.168.181.18"port &＃61;> "6379"password &＃61;> "sinoeyes"key &＃61;> "sinoeyes-io"data_type &＃61;> "list"db &＃61;> "5"} }filter {grok {match &＃61;> { "message" &＃61;> "%{TIMESTAMP_ISO8601:log_date}\s*(?([\S&＃43;]*))\s*(?([\S&＃43;]*))\s*%{LOGLEVEL:log_level}" }}}output {elasticsearch {hosts &＃61;> "192.168.181.52:9200"index &＃61;> "%{tags}-%{&＃43;YYYY.MM.dd}"ssl &＃61;> trueuser &＃61;> "elastic"password &＃61;> "W7ZOoGqxdncD5IyLSeKX"ssl &＃61;> truecacert &＃61;> &＃39;/etc/logstash/config/certs/ca.crt&＃39;} }

2.3、创建Logstash的配置文件

cat config/logstash.yml

# Settings file in YAML # # Settings can be specified either in hierarchical form, e.g.: # # pipeline: # batch: # size: 125 # delay: 5 # # Or as flat keys: # # pipeline.batch.size: 125 # pipeline.batch.delay: 5 # # ------------ Node identity ------------ # # Use a descriptive name for the node: # # node.name: test # # If omitted the node name will default to the machine&＃39;s host name # # ------------ Data path ------------------ # # Which directory should be used by logstash and its plugins # for any persistent needs. Defaults to LOGSTASH_HOME/data # # path.data: # # ------------ Pipeline Settings -------------- # # The ID of the pipeline. # # pipeline.id: main # # Set the number of workers that will, in parallel, execute the filters&＃43;outputs # stage of the pipeline. # # This defaults to the number of the host&＃39;s CPU cores. # # pipeline.workers: 2 # # How many events to retrieve from inputs before sending to filters&＃43;workers # # pipeline.batch.size: 125 # # How long to wait in milliseconds while polling for the next event # before dispatching an undersized batch to filters&＃43;outputs # # pipeline.batch.delay: 50 # # Force Logstash to exit during shutdown even if there are still inflight # events in memory. By default, logstash will refuse to quit until all # received events have been pushed to the outputs. # # WARNING: enabling this can lead to data loss during shutdown # # pipeline.unsafe_shutdown: false # # Set the pipeline event ordering. Options are "auto" (the default), "true" or "false". # "auto" will automatically enable ordering if the &＃39;pipeline.workers&＃39; setting # is also set to &＃39;1&＃39;. # "true" will enforce ordering on the pipeline and prevent logstash from starting # if there are multiple workers. # "false" will disable any extra processing necessary for preserving ordering. # pipeline.ordered: auto # # ------------ Pipeline Configuration Settings -------------- # # Where to fetch the pipeline configuration for the main pipeline # # path.config: # # Pipeline configuration string for the main pipeline # # config.string: # # At startup, test if the configuration is valid and exit (dry run) # # config.test_and_exit: false # # Periodically check if the configuration has changed and reload the pipeline # This can also be triggered manually through the SIGHUP signal # # config.reload.automatic: false # # How often to check if the pipeline configuration has changed (in seconds) # Note that the unit value (s) is required. Values without a qualifier (e.g. 60) # are treated as nanoseconds. # Setting the interval this way is not recommended and might change in later versions. # # config.reload.interval: 3s # # Show fully compiled configuration as debug log message # NOTE: --log.level must be &＃39;debug&＃39; # # config.debug: false # # When enabled, process escaped characters such as \n and \" in strings in the # pipeline configuration files. # # config.support_escapes: false # # ------------ HTTP API Settings ------------- # Define settings related to the HTTP API here. # # The HTTP API is enabled by default. It can be disabled, but features that rely # on it will not work as intended. # http.enabled: true # # By default, the HTTP API is bound to only the host&＃39;s local loopback interface, # ensuring that it is not accessible to the rest of the network. Because the API # includes neither authentication nor authorization and has not been hardened or # tested for use as a publicly-reachable API, binding to publicly accessible IPs # should be avoided where possible. # # http.host: 127.0.0.1 # # The HTTP API web server will listen on an available port from the given range. # Values can be specified as a single port (e.g., &＃96;9600&＃96;), or an inclusive range # of ports (e.g., &＃96;9600-9700&＃96;). # # http.port: 9600-9700 # # ------------ Module Settings --------------- # Define modules here. Modules definitions must be defined as an array. # The simple way to see this is to prepend each &＃96;name&＃96; with a &＃96;-&＃96;, and keep # all associated variables under the &＃96;name&＃96; they are associated with, and # above the next, like this: # # modules: # - name: MODULE_NAME # var.PLUGINTYPE1.PLUGINNAME1.KEY1: VALUE # var.PLUGINTYPE1.PLUGINNAME1.KEY2: VALUE # var.PLUGINTYPE2.PLUGINNAME1.KEY1: VALUE # var.PLUGINTYPE3.PLUGINNAME3.KEY1: VALUE # # Module variable names must be in the format of # # var.PLUGIN_TYPE.PLUGIN_NAME.KEY # # modules: # # ------------ Cloud Settings --------------- # Define Elastic Cloud settings here. # Format of cloud.id is a base64 value e.g. dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRub3RhcmVhbCRpZGVudGlmaWVy # and it may have an label prefix e.g. staging:dXMtZ... # This will overwrite &＃39;var.elasticsearch.hosts&＃39; and &＃39;var.kibana.host&＃39; # cloud.id: # # Format of cloud.auth is: : # This is optional # If supplied this will overwrite &＃39;var.elasticsearch.username&＃39; and &＃39;var.elasticsearch.password&＃39; # If supplied this will overwrite &＃39;var.kibana.username&＃39; and &＃39;var.kibana.password&＃39; # cloud.auth: elastic: # # ------------ Queuing Settings -------------- # # Internal queuing model, "memory" for legacy in-memory based queuing and # "persisted" for disk-based acked queueing. Defaults is memory # # queue.type: memory # # If using queue.type: persisted, the directory path where the data files will be stored. # Default is path.data/queue # # path.queue: # # If using queue.type: persisted, the page data files size. The queue data consists of # append-only data files separated into pages. Default is 64mb # # queue.page_capacity: 64mb # # If using queue.type: persisted, the maximum number of unread events in the queue. # Default is 0 (unlimited) # # queue.max_events: 0 # # If using queue.type: persisted, the total capacity of the queue in number of bytes. # If you would like more unacked events to be buffered in Logstash, you can increase the # capacity using this setting. Please make sure your disk drive has capacity greater than # the size specified here. If both max_bytes and max_events are specified, Logstash will pick # whichever criteria is reached first # Default is 1024mb or 1gb # # queue.max_bytes: 1024mb # # If using queue.type: persisted, the maximum number of acked events before forcing a checkpoint # Default is 1024, 0 for unlimited # # queue.checkpoint.acks: 1024 # # If using queue.type: persisted, the maximum number of written events before forcing a checkpoint # Default is 1024, 0 for unlimited # # queue.checkpoint.writes: 1024 # # If using queue.type: persisted, the interval in milliseconds when a checkpoint is forced on the head page # Default is 1000, 0 for no periodic checkpoint. # # queue.checkpoint.interval: 1000 # # ------------ Dead-Letter Queue Settings -------------- # Flag to turn on dead-letter queue. # # dead_letter_queue.enable: false# If using dead_letter_queue.enable: true, the maximum size of each dead letter queue. Entries # will be dropped if they would increase the size of the dead letter queue beyond this setting. # Default is 1024mb # dead_letter_queue.max_bytes: 1024mb# If using dead_letter_queue.enable: true, the directory path where the data files will be stored. # Default is path.data/dead_letter_queue # # path.dead_letter_queue: # # ------------ Metrics Settings -------------- # # Bind address for the metrics REST endpoint # # http.host: "127.0.0.1" # # Bind port for the metrics REST endpoint, this option also accept a range # (9600-9700) and logstash will pick up the first available ports. # # http.port: 9600-9700 # # ------------ Debugging Settings -------------- # # Options for log.level: # * fatal # * error # * warn # * info (default) # * debug # * trace # # log.level: info # path.logs: # # ------------ Other Settings -------------- # # Where to find custom plugins # path.plugins: [] # # Flag to output log lines of each pipeline in its separate log file. Each log filename contains the pipeline.name # Default is false # pipeline.separate_logs: false # # ------------ X-Pack Settings (not applicable for OSS build)-------------- # # X-Pack Monitoring # https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.username: logstash_system xpack.monitoring.elasticsearch.password: "u0bMA5cORrtEkBaOfW2F" #xpack.monitoring.elasticsearch.proxy: ["http://proxy:port"] xpack.monitoring.elasticsearch.hosts: ["https://192.168.181.52:9200"] # an alternative to hosts &＃43; username/password settings is to use cloud_id/cloud_auth #xpack.monitoring.elasticsearch.cloud_id: monitoring_cluster_id:xxxxxxxxxx #xpack.monitoring.elasticsearch.cloud_auth: logstash_system:password # another authentication alternative is to use an Elasticsearch API key #xpack.monitoring.elasticsearch.api_key: "id:api_key" #xpack.monitoring.elasticsearch.ssl.certificate_authority: [ "/var/lib/docker/volumes/es_certs/_data/ca/ca.crt" ] xpack.monitoring.elasticsearch.ssl.certificate_authority: /etc/logstash/config/certs/ca.crt #xpack.monitoring.elasticsearch.ssl.truststore.path: path/to/file #xpack.monitoring.elasticsearch.ssl.truststore.password: password #xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file #xpack.monitoring.elasticsearch.ssl.keystore.password: password #xpack.monitoring.elasticsearch.ssl.verification_mode: certificate #xpack.monitoring.elasticsearch.sniffing: false #xpack.monitoring.collection.interval: 10s #xpack.monitoring.collection.pipeline.details.enabled: true # # X-Pack Management # https://www.elastic.co/guide/en/logstash/current/logstash-centralized-pipeline-management.html #xpack.management.enabled: false #xpack.management.pipeline.id: ["main", "apache_logs"] #xpack.management.elasticsearch.username: logstash_admin_user #xpack.management.elasticsearch.password: password #xpack.management.elasticsearch.proxy: ["http://proxy:port"] #xpack.management.elasticsearch.hosts: ["https://es1:9200", "https://es2:9200"] # an alternative to hosts &＃43; username/password settings is to use cloud_id/cloud_auth #xpack.management.elasticsearch.cloud_id: management_cluster_id:xxxxxxxxxx #xpack.management.elasticsearch.cloud_auth: logstash_admin_user:password # another authentication alternative is to use an Elasticsearch API key #xpack.management.elasticsearch.api_key: "id:api_key" #xpack.management.elasticsearch.ssl.certificate_authority: [ "/path/to/ca.crt" ] #xpack.management.elasticsearch.ssl.truststore.path: /path/to/file #xpack.management.elasticsearch.ssl.truststore.password: password #xpack.management.elasticsearch.ssl.keystore.path: /path/to/file #xpack.management.elasticsearch.ssl.keystore.password: password #xpack.management.elasticsearch.ssl.verification_mode: certificate #xpack.management.elasticsearch.sniffing: false #xpack.management.logstash.poll_interval: 5s

2.4、启动

docker-compose up -d

Logstash 启用 TLS参考

三、日志告警

1、创建连接器
在这里插入图片描述
2、创建报警
每十分钟检查日志&＃xff0c;如果检测到日志ERROR超过1个就发送邮件

定义警报参考

四、页面调整

5.1、Discover页面日志显示不全

解决办法&＃xff1a;
高级设置页面truncate:maxHeight 这个属性指定了表格中单元格显示时占用的最大高度&＃xff0c;设置为0则不限制。
在这里插入图片描述

高级设置所有字段参考

五、定时删除日志

#/bin/bash #es-index-clear #只保留某几天内的日志索引-5 days || 5 days ago ST_LAST_DATA&＃61;&＃96;date -d "-7 days" "&＃43;%Y.%m.%d"&＃96; UAT_LAST_DATA&＃61;&＃96;date -d "-7 days" "&＃43;%Y.%m.%d"&＃96; PROD_LAST_DATA&＃61;&＃96;date -d "-30 days" "&＃43;%Y.%m.%d"&＃96;#删除上个月份所有的索引 curl --user elastic:passowrd -XDELETE "https://172.188.180.52:9200/st-${ST_LAST_DATA}" -k #curl -XGET "https://172.188.180.52:9200/st-${ST_LAST_DATA}" curl --user elastic:passowrd -XDELETE "https://172.188.180.52:9200/uat-paas-${UAT_LAST_DATA}" -k #curl -XGET "https://172.188.180.52:9200/uat-dev15-${UAT_LAST_DATA}" curl --user elastic:passowrd -XDELETE "https://172.188.180.52:9200/prod-admin-paas-${PROD_LAST_DATA}" -k #curl -XGET "https://172.188.180.52:9200/prod-admin-paas-${PROD_LAST_DATA}"#crontab -e添加定时任务&＃xff1a; #0 1 * * * /home/stack_elk/clear_index/es-index-clear.sh

-k 允许在没有证书的情况下连接到SSL站点

(官方参考)[https://www.elastic.co/guide/en/elastic-stack-get-started/7.9/get-started-docker.html#get-started-docker-tls]

dockercompose安装ELK7.12.0并启用xpack及kibana设置日志报警