作者:周树草 | 来源:互联网 | 2023-07-13 17:29
logstash配置:
1 2 3 4 5 6 7 8 9 10 11
| output {
elasticsearch {
hosts => ["192.168.8.8:9200"]
index => "jdbc_index"
document_id => "%{id}"
}
stdout {
codec => json_lines
}
} |
上面配置两种方式输出的,为什么使用kibana显示ES的记录只有一条呢
![clipboard.png](https://img.php1.cn/3cd4a/1eebe/cd5/7cccb7e4b6cb5cb8.webp)
stdout输出的就是正常的:
![clipboard.png](https://img.php1.cn/3cd4a/1eebe/cd5/5287a7b3296ea13e.webp)
搞了好久了,有前辈知道原因的吗?
配置文件代码:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
| input {
stdin {
}
jdbc {
jdbc_connection_string => "jdbc:mysql://192.168.12.222:3306/main"
jdbc_user => "admin"
jdbc_password => "123456"
jdbc_driver_library => "/usr/local/logstash/mysql-connector-java-5.1.3.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement_filepath => "/usr/local/logstash/config/jdbc.sql"
#statement => "select inquiryId, inquiryNo, carTypeId, orgId, `status`, createTime, publishTime, failTime from qp_inquiry limit 500"
schedule => "* * * * *"
type => "jdbc_type"
lowercase_column_names => "false"
}
}
filter {
json {
source => "message"
remove_field => ["message"]
}
}
output {
elasticsearch {
hosts => ["192.168.8.8:9200"]
index => "jdbc_index"
document_id => "%{id}"
}
stdout {
codec => json_lines
}
} |
sql语句:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
| SELECT
*
FROM
(
SELECT
inquiryId,
inquiryNo,
carTypeId,
orgId,
`status`,
createTime,
publishTime,
failTime
FROM
qp_inquiry
LIMIT 50
) as `t1`
LIMIT 50000 OFFSET 0 |