如何解决logstash 7.9.1 docker conatiner:文件输入无效
我正在尝试读取日志文件,但是不起作用,当将logstash.conf配置为侦听端口5000却无法从文件中侦听时,它可以工作。我正在从Docker容器中使用Logstash版本7.9.1,并尝试将日志发送到Elastic search 7.9.1。 这是我的logstash.conf文件
input {
file {
path => ["/home/douglas/projects/incollect/*.log"]
start_position => "beginning"
ignore_older => 0
sincedb_path => "/dev/null"
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
index => "test-elk-%{+YYYY.MM.dd}"
user => "elastic"
password => "changeme"
}
stdout {
codec => rubydebug
}
}
这是来自控制台的日志,我看不到任何错误并说已成功启动
logstash_1 | [2020-10-16T00:38:27,748][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch",:hosts=>["//elasticsearch:9200"]}
logstash_1 | [2020-10-16T00:38:27,795][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7,:ecs_compatibility=>:disabled}
logstash_1 | [2020-10-16T00:38:27,798][INFO ][logstash.javapipeline ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash","pipeline.workers"=>1,"pipeline.batch.size"=>2,"pipeline.batch.delay"=>50,"pipeline.max_inflight"=>2,"pipeline.sources"=>["monitoring pipeline"],:thread=>"#<Thread:0x44d5fe run>"}
logstash_1 | [2020-10-16T00:38:27,800][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main","pipeline.workers"=>4,"pipeline.batch.size"=>125,"pipeline.max_inflight"=>500,"pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"],:thread=>"#<Thread:0x4c6dee32 run>"}
logstash_1 | [2020-10-16T00:38:27,840][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*","version"=>60001,"settings"=>{"index.refresh_interval"=>"5s","number_of_shards"=>1},"mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message","match_mapping_type"=>"string","mapping"=>{"type"=>"text","norms"=>false}}},{"string_fields"=>{"match"=>"*","norms"=>false,"fields"=>{"keyword"=>{"type"=>"keyword","ignore_above"=>256}}}}}],"properties"=>{"@timestamp"=>{"type"=>"date"},"@version"=>{"type"=>"keyword"},"geoip"=>{"dynamic"=>true,"properties"=>{"ip"=>{"type"=>"ip"},"location"=>{"type"=>"geo_point"},"latitude"=>{"type"=>"half_float"},"longitude"=>{"type"=>"half_float"}}}}}}}
logstash_1 | [2020-10-16T00:38:28,535][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>0.73}
logstash_1 | [2020-10-16T00:38:28,599][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
logstash_1 | [2020-10-16T00:38:28,600][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.8}
logstash_1 | [2020-10-16T00:38:28,840][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
logstash_1 | [2020-10-16T00:38:28,909][INFO ][logstash.agent ] Pipelines running {:count=>2,:running_pipelines=>[:".monitoring-logstash",:main],:non_running_pipelines=>[]}
logstash_1 | [2020-10-16T00:38:28,920][INFO ][filewatch.observingtail ][main][4a3eb924128694e00dae8e6fab084bfc5e3c3692e66663362019b182fcb31a48] START,creating discoverer,Watch with file and sincedb collections
logstash_1 | [2020-10-16T00:38:29,386][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
这是我的日志文件:
Oct 9 15:34:19 incollect drupal: http://dev.incollect.com|1602257659|DEV|52.202.31.67|http://dev.incollect.com/icadmin/inquires_report?q=icadmin/ajax_validate_and_fix_inquire_by_id|http://dev.incollect.com/icadmin/inquires_report|3||Validate inquireStep 0
Oct 9 15:34:19 incollect drupal: http://dev.incollect.com|1602257659|DEV|52.202.31.67|http://dev.incollect.com/icadmin/inquires_report?q=icadmin/ajax_validate_and_fix_inquire_by_id|http://dev.incollect.com/icadmin/inquires_report|3||Validate inquireStep 1 - inquire_id:14219
已编辑 **************** 我正在添加docker-compose文件,这是我对logstash的配置
logstash:
build:
context: logstash/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- type: bind
source: ./logstash/config/logstash.yml
target: /usr/share/logstash/config/logstash.yml
read_only: true
- type: bind
source: ./logstash/pipeline
target: /usr/share/logstash/pipeline
read_only: true
volumes:
- ./../../:/usr/share/logstash
ports:
- "5000:5000/tcp"
- "5000:5000/udp"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -xms256m"
networks:
- elk
depends_on:
- elasticsearch
我不确定是什么问题,我尝试了其他解决方案,但是它不起作用。
解决方法
如果这是 - ./../../:/usr/share/logstash
,用于装载日志卷,则您的logstash文件输入路径应指向/usr/share/logstash/*.log
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。