作者:MR张尉诚 | 来源:互联网 | 2023-02-02 14:04
我正在尝试使用Logstash在MySQL和Elasticsearch之间同步数据.
我将多个jdbc输入和多个输出设置为不同的弹性搜索索引...以及我做错了什么,因为一切都转到了else块.
这是我的配置:
input {
jdbc {
jdbc_connection_string => "jdbc:mysql:127.0.0.1:3306/whatever"
jdbc_user => "xxx"
jdbc_password => "yyy"
jdbc_driver_library => "mysql-connector-java-5.1.41.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
schedule => "* * * * *"
statement => "SELECT * from table1 WHERE updated_at > :sql_last_value order by updated_at"
use_column_value => true
tracking_column => updated_at
type => "table1"
last_run_metadata_path => "/opt/logstash-5.4.0/sql-last-values/table1"
}
jdbc {
jdbc_connection_string => "jdbc:mysql:127.0.0.1:3306/whatever"
jdbc_user => "xxx"
jdbc_password => "yyy"
jdbc_driver_library => "mysql-connector-java-5.1.41.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
schedule => "* * * * *"
statement => "SELECT * from table2 WHERE updated_at > :sql_last_value order by updated_at"
use_column_value => true
tracking_column => updated_at
type => "table2"
last_run_metadata_path => "/opt/logstash-5.4.0/sql-last-values/table2"
}
}
output {
if [type] == "table1" {
elasticsearch {
hosts => ["localhost:9200"]
index => "table1"
document_type => "table1"
document_id => "%{id}"
}
file {
codec => json_lines
path => "/opt/logstash-5.4.0/logs/table1.log"
}
} else if [type] == "table2" {
elasticsearch {
hosts => ["localhost:9200"]
index => "table2"
document_type => "table2"
document_id => "%{id}"
}
} else {
file {
codec => json_lines
path => "/opt/logstash-5.4.0/logs/unknown.log"
}
}
}
我究竟做错了什么 ?一切都将转到else块,转到/opt/logstash-5.4.0/logs/unknown.log
我的方法有误吗?我应该有多个文件吗?
先感谢您
1> Juan Pablo G..:
找到解决方案
我用tags
而不是type
input {
jdbc {
...
tags => "table1"
...
}
jdbc {
...
tags => "table2"
...
}
}
output {
if "table1" in [tags] {
}
https://discuss.elastic.co/t/solved-multiple-logstash-config-file/51692/10