一、ELK介绍
ELK是3个开源软件的缩写,分别为Elasticsearch 、 Logstash和Kibana , 它们都是开源软件。不过现在还新增了一个Beats,它是一个轻量级的日志收集处理工具(Agent),Beats占用资源少,适合于在各个服务器上搜集日志后传输给Logstash,官方也推荐此工具,目前由于原本的ELK Stack成员中加入了Beats工具所以已改名为Elastic Stack。
Elasticsearch是个开源分布式搜索引擎,提供搜集、分析、存储数据3大功能。它的特点有:分布式,零配置,自动发现,索引自动分片,索引副本机制,restful风格接口,多数据源,自动搜索负载等。
Logstash主要是用来日志的搜集、分析、过滤日志的工具,支持大量的数据获取方式。一般工作方式为c/s架构,Client端安装在需要收集日志的主机上,server端负责将收到的各节点日志进行过滤、修改等操作在一并发往Elasticsearch上去。
Kibana也是一个开源和免费的工具,Kibana可以为 Logstash和 ElasticSearch提供的日志分析友好的 Web 界面,可以帮助汇总、分析和搜索重要数据日志。
Beats在这里是一个轻量级日志采集器,其实Beats家族有6个成员,早期的ELK架构中使用Logstash收集、解析日志,但是Logstash对内存、cpu、io等资源消耗比较高。相比 Logstash,Beats所占系统的cpu和内存几乎可以忽略不计。
二、基础环境
1.(虚拟机)规划三个节点,一个为主节点,两个为数据节点:
主机名:elk--1 节点IP:192.168.233.201 节点规划:Elasticsearch+Kibana(主)
主机名:elk--2 节点IP:192.168.233.202 节点规划:Elasticsearch+Logstash(数据)
主机名:elk--3 节点IP:192.168.233.203 节点规划:Elasticsearch(数据)
2.修改主机名:
elk_1节点: [root@localhost ~]# hostnamectl set-hostname elk--1 [root@localhost ~]# bash [root@elk--1 ~]# elk_2节点: [root@localhost ~]# hostnamectl set-hostname elk--2 [root@localhost ~]# bash [root@elk--2 ~]# elk_3节点: [root@localhost ~]# hostnamectl set-hostname elk--3 [root@localhost ~]# bash [root@elk--3 ~]#
3.配置hosts文件:
elk--1节点: [root@elk--1 ~]# vi /etc/hosts [root@elk--1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.100.11 elk--1 192.168.100.12 elk--2 192.168.100.13 elk--3 elk--2节点: [root@elk--2 ~]# vi /etc/hosts [root@elk--2 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.100.11 elk--1 192.168.100.12 elk--2 192.168.100.13 elk--3 elk--3节点: [root@elk--3 ~]# vi /etc/hosts [root@elk--3~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.100.11 elk--1 192.168.100.12 elk--2 192.168.100.13 elk--3
4.yum安装
部署ELK环境需要jdk1.8以上的JDK版本软件环境,我们使用opnejdk1.8,3节点全部安装
[root@elk--1~]# yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel vim net-tools wget [root@elk--1~]# java -version openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
三、部署Elasticserach
1.下载 Elasticserach 三个节点都需要下载
下载 Elasticserach-6.0.0.rpm 包 [root@elk--1 ~]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.0.0.rpm [root@elk--1 ~]# ls anaconda-ks.cfg elasticsearch-6.0.0.rpm
2.安装 Elasticserach 三个节点都需要安装
[root@elk--1 ~]# rpm -ivh elasticsearch-6.0.0.rpm // 参数含义:i表示安装,v表示显示安装过程,h表示显示进度 warning: elasticsearch-6.0.0.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY Preparing... ################################# [100%] Creating elasticsearch group... OK Creating elasticsearch user... OK Updating / installing... 1:elasticsearch-0:6.0.0-1 ################################# [100%] ### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd sudo systemctl daemon-reload sudo systemctl enable elasticsearch.service ### You can start elasticsearch service by executing sudo systemctl start elasticsearch.service
3.配置 Elasticserach
elk--1节点(加蓝标深的是需要修改的地方):
[root@elk_1 ~]# vi /etc/elasticsearch/elasticsearch.yml # ======= Elasticsearch Configuration =========== # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ------------------Cluster -------------------- # Use a descriptive name for your cluster: cluster.name: ELK //配置es的集群名称,默认是elasticsearch,es会自动发现在同一网段下的es,如果在同一网段下有多个集群,就可以用这个属性来区分不同的集群。 # ------------------------Node ----------------- # Use a descriptive name for the node: node.name: elk-1 //节点名,默认随机指定一个name列表中名字,该列表在es的jar包中config文件夹里name.txt文件中,其中有很多作者添加的有趣名字。 node.master: true //指定该节点是否有资格被选举成为node,默认是true,es是默认集群中的第一台机器为master,如果这台机挂了就会重新选举master。 其他两节点为false node.data: false //指定该节点是否存储索引数据,默认为true。其他两节点为true # ----------------- Paths ---------------- # Path to directory where to store the data (separate multiple locations by comma): path.data: /var/lib/elasticsearch //索引数据存储位置(保持默认,不要开启注释) # Path to log files: path.logs: /var/log/elasticsearch //设置日志文件的存储路径,默认是es根目录下的logs文件夹 # --------------- Network ------------------ # Set the bind address to a specific IP (IPv4 or IPv6): network.host: 192.168.233.201 //设置绑定的ip地址,可以是ipv4或ipv6的,默认为0.0.0.0。 # Set a custom port for HTTP: http.port: 9200 //启动的es对外访问的http端口,默认9200 # For more information, consult the network module documentation. # --------------------discovery ---------------- # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] #discovery.zen.ping.unicast.hosts: ["host1", "host2"] discovery.zen.ping.unicast.hosts: ["elk-1","elk-2","elk-3"] //设置集群中master节点的初始列表,可以通过这些节点来自动发现新加入集群的节点。
elk--2节点:
[root@elk--2 ~]# vi /etc/elasticsearch/elasticsearch.yml cluster.name: ELK node.name: elk-2 node.master: false node.data: true path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 192.168.233.202 http.port: 9200 discovery.zen.ping.unicast.hosts: ["elk-1","elk-2","elk-3"]
elk--3节点:
[root@elk--3 ~]# vi /etc/elasticsearch/elasticsearch.yml
cluster.name: ELK node.name: elk-2 node.master: false node.data: true path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 192.168.233.203 http.port: 9200 discovery.zen.ping.unicast.hosts: ["elk-1","elk-2","elk-3"]
4.启动服务
通过命令启动es服务,启动后使用ps命令查看进程是否存在或者使用netstat命令查看是否端口启动
三个节点都需要启动
[root@elk--1 ~]# systemctl start elasticsearch [root@elk--1 ~]# ps -ef |grep elasticsearch elastic+ 19280 1 0 09:00 ? 00:00:54 /bin/java -xms1g -Xmx1g -XX:+UseConcmarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPretouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -dio.netty.noUnsafe=true -dio.netty.noKeySetoptimization=true -dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet root 19844 19230 0 10:54 pts/0 00:00:00 grep --color=auto elasticsearch [root@elk--1 ~]# netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1446/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1994/master tcp6 0 0 192.168.40.11:9200 :::* LISTEN 19280/java tcp6 0 0 192.168.40.11:9300 :::* LISTEN 19280/java tcp6 0 0 :::22 :::* LISTEN 1446/sshd tcp6 0 0 ::1:25 :::* LISTEN 1994/master
有以上端口或者进程存在,证明es服务启动成功
5.检查集群状态
使用curl命令来检查集群状态 elk_1 节点:
[root@elk--1 ~]# curl '192.168.233.201:9200/_cluster/health?pretty' { "cluster_name" : "ELK", "status" : "green", //为green则代表健康没问题,yellow或者red 则是集群有问题 "timed_out" : false, //是否有超时 "number_of_nodes" : 3, //集群中的节点数量 "number_of_data_nodes" : 2, //集群中data节点的数量 "active_primary_shards" : 1, "active_shards" : 2, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 }
四、部署 Kibana
[root@elk-1 ~]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo[root@elk-1 ~]# yum install Nginx -y [root@elk-1 ~]# systemctl start Nginx [root@elk-1 ~]# vim /etc/Nginx/Nginx.conf upstream elasticsearch { zone elasticsearch 64K; server elk-1:9200; server elk-2:9200; server elk-3:9200; } server { listen 8080; server_name 192.168.233.201; location / { proxy_pass http://elasticsearch; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } access_log /var/log/es_access.log; }
1.在主节点下载安装 Kibana
[root@elk--1 ~]# wget https://artifacts.elastic.co/downloads/kibana/kibana-6.0.0-x86_64.rpm [root@elk--1 ~]# rpm -ivh kibana-6.0.0-x86_64.rpm warning: kibana-6.0.0-x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY Preparing... ################################# [100%] Updating / installing... 1:kibana-6.0.0-1 ################################# [100%]
2.配置 Kibana
[root@elk--1 ~]# vim /etc/kibana/kibana.yml server.port: 5601 server.host: "192.168.233.201" elasticsearch.url: "http://192.168.233.201:9200"
4.启动 Kibana
[root@elk--1 ~]# systemctl start kibana [root@elk--1 ~]# ps -ef |grep kibana kibana 19958 1 41 11:26 ? 00:00:03 /usr/share/kibana/bin/../node/bin/node --no-warnings /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml root 19970 19230 0 11:26 pts/0 00:00:00 grep --color=auto kibana [root@elk--1 ~]# netstat -lntp |grep node tcp 0 0 192.168.233.201:5601 0.0.0.0:* LISTEN 19958/node
启动成功后网页访问,可以访问到如下界面。http://192.168.233.201:5601 访问的这个页面
五、部署 Logstash
1. 在elk--2节点下载安装 Logstash
[root@elk--2 ~]# wget https://artifacts.elastic.co/downloads/logstash/logstash-6.0.0.rpm [root@elk--2 ~]# rpm -ivh logstash-6.0.0.rpm warning: logstash-6.0.0.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY Preparing... ################################# [100%] Updating / installing... 1:logstash-1:6.0.0-1 ################################# [100%] Using provided startup.options file: /etc/logstash/startup.options
2. 配置 Logstash
[root@elk--2 ~]# vim /etc/logstash/logstash.yml http.host: "192.168.233.202"
配置logstash收集syslog日志:
[root@elk--2 ~]# vim /etc/logstash/conf.d/syslog.conf input { file { path => "/var/log/messages" type => "systemlog" start_position => "beginning" stat_interval => "3" } } output { elasticsearch { hosts => ["192.168.233.201:9200","192.168.233.202:9200","192.168.233.203:9200"] index => "system-log-%{+YYYY.MM.dd}" } }
[root@elk--2 ~]# ln -s /usr/share/logstash/bin/logstash /usr/bin // 创建软连接,方便使用logstash命令 [root@elk--2 ~]# logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/syslog.conf --config.test_and_exit Sending Logstash's logs to /var/log/logstash which is Now configured via log4j2.properties Configuration OK // 为ok则代表没问题
3. 授权
[root@elk--2 ~]# chmod 755 /var/log/messages [root@elk--2 ~]# chown -R logstash /var/lib/logstash/
4. 启动 logstash
检查配置文件没有问题后,启动Logstash服务:
[root@elk_2 ~]# systemctl start logstash
使用ps命令,查看进程:
[root@elk--2 ~]# ps -ef |grep logstash logstash 21835 1 12 16:45 ? 00:03:01 /bin/java -XX:+UseParNewGC -XX:+UseConcmarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+disableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -xms256m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash root 21957 20367 0 17:10 pts/2 00:00:00 grep --color=auto logstash
使用netstat命令,查看进程端口:
[root@elk--2 ~]# netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 957/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1152/master tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 7390/Nginx: master tcp6 0 0 :::10514 :::* LISTEN 7403/java tcp6 0 0 192.168.233.202:9300 :::* LISTEN 6418/java tcp6 0 0 :::22 :::* LISTEN 957/sshd tcp6 0 0 ::1:25 :::* LISTEN 1152/master tcp6 0 0 192.168.233.202:9600 :::* LISTEN 7403/java tcp6 0 0 192.168.233.202:9200 :::* LISTEN 6418/java
主要查看9600 端口
5. Kibana 上查看日志
之前部署Kibana完成后,还没有检索日志。现在Logstash部署完成,我们回到Kibana服务器上查看日志索引,执行命令如下:
[root@elk--1 ~]# curl '192.168.233.201:9200/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open system-syslog-2022.01 tAd3L8W6RHqDeRDOvOdrTA 5 1 10631 0 2.8mb 1.3mb green open .kibana HJ8wt6BuQaeVcc5Vf1zbcg 1 1 3
6. Web 界面配置
浏览器访问192。168.233.201:5601,到 Kibana 上配置索引
复制下图黄色标注的地方 粘贴到Web界面
按完之后按、按discover。出现的这个是乱码显现
跟着下图走就可以解决乱码了
这就是正常界面
六、Logstash 收集 Nginx 日志
1.下载安装Nginx
[root@elk--2 ~]# wget http://Nginx.org/packages/centos/7/x86_64/RPMS/Nginx-1.16.1-1.el7.ngx.x86_64.rpm [root@elk--2 ~]# rpm -ivh Nginx-1.16.1-1.el7.ngx.x86_64.rpm 警告:Nginx-1.16.1-1.el7.ngx.x86_64.rpm: 头V4 RSA/SHA1 Signature,密钥 ID 7bd9bf62: NOKEY 准备中... ################################# [100%] 正在升级/安装... 1:Nginx-1:1.16.1-1.el7.ngx ( 3%################################# [100%] ---------------------------------------------------------------------- Thanks for using Nginx! Please find the official documentation for Nginx here: * http://Nginx.org/en/docs/ Please subscribe to Nginx-announce mailing list to get the most important news about Nginx: * http://Nginx.org/en/support.html Commercial subscriptions for Nginx are available on: * http://Nginx.com/products/ ----------------------------------------------------------------------
3. 配置 Logstash
[root@elk--2 ~]# vim /etc/logstash/conf.d/Nginx.conf input { file { path => "/tmp/elk_access.log" start_position => "beginning" type => "Nginx" } } filter { grok { match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} - %{USERNAME:remote_user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:http_verb} %{NOTSPACE:HTTP_Request}(?: HTTP/%{NUMBER:http_version})?|%{DATA:raw_HTTP_Request})\" %{NUMBER:response} (?:%{NUMBER:bytes_read}|-) %{QS:referrer} %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time:float}"} } geoip { source => "clientip" } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["192.168.233.202:9200"] index => "Nginx-test-%{+YYYY.MM.dd}" } }
[root@elk--2 ~]# logstash --path.settings /etc/logstash/ -f /etc/logstash/conf.d/Nginx.conf --config.test_and_exit Sending Logstash's logs to /var/log/logstash which is Now configured via log4j2.properties Configuration OK
# 不移走这个文件就 elk.conf 就执不了。因为这个文件也是 80 端口 [root@elk--2 ~]# mv /etc/Nginx/conf.d/default.conf /tmp/ [root@elk--2 ~]#vim /etc/Nginx/conf.d/elk.conf server { listen 80; server_name elk.com; location / { proxy_pass http://192.168.233.201:5601; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } access_log /tmp/elk_access.log main2; }
修改Nginx日志配置文件,增加如下内容(需注意Nginx配置文件格式):
[root@elk--2 ~]# vim /etc/Nginx/Nginx.confhttp { include /etc/Nginx/mime.types; default_type application/octet-stream; # log_format main '$remote_addr - $remote_user [$time_local] "$request" ' # '$status $body_bytes_sent "$http_referer" ' # '"$http_user_agent" "$http_x_forwarded_for"'; #插入这些加蓝代码 log_format main2 '$http_host $remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$upstream_addr" $request_time'; access_log /var/log/Nginx/access.log main2; sendfile on; #tcp_nopush on; keepalive_timeout 65;
[root@elk--2 ~]# Nginx -t Nginx: the configuration file /etc/Nginx/Nginx.conf Syntax is ok Nginx: configuration file /etc/Nginx/Nginx.conf test is successful
启动
[root@elk-2 ~]# systemctl restart logstash
192.168.100.30 elk.com
浏览器访问,检查是否有日志产生
4. Web 页面配置
浏览器访问192.168.233.201:5601,到Kibana上配置索引:
查看elk--1主节点收到没有
[root@elk--1 ~]# curl '192.168.233.201:9200/_cat/indices?v' health status index uuid pri rep d green open system-syslog-2022.01 tAd3L8W6RHqDeRDOvOdrTA 5 1 green open Nginx-test-2022.01.14 vzFiklXgRP-GenN_KMe87A 5 1 green open .kibana HJ8wt6BuQaeVcc5Vf1zbcg 1 1
跟着点击即可,如果出现乱码显现就去修改他的时间
七、Beats采集日志
1.在elk--3节点下载安装 Beats
[root@elk--3 ~]# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-x86_64.rpm --2020-03-30 22:41:52-- https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-x86_64.rpm 正在解析主机 artifacts.elastic.co (artifacts.elastic.co)... 151.101.230.222, 2a04:4e42:1a::734 正在连接 artifacts.elastic.co (artifacts.elastic.co)|151.101.230.222|:443... 已连接。 已发出 HTTP 请求,正在等待回应... 200 OK 长度:11988378 (11M) [binary/octet-stream] 正在保存至: “filebeat-6.0.0-x86_64.rpm.1” 100%[===================================>] 11,988,378 390KB/s 用时 30s 2020-03-30 22:42:24 (387 KB/s) - 已保存 “filebeat-6.0.0-x86_64.rpm.1” [11988378/11988378]) [root@elk_3 ~]# rpm -ivh filebeat-6.0.0-x86_64.rpm
2.配置文件
[root@elk--3 ~]# vim /etc/filebeat/filebeat.yml filebeat.prospectors: paths: - /var/log/elasticsearch/elk.log //此处可自行改为想要监听的日志文件 output.elasticsearch: hosts: ["192.168.100.11:9200"] systemctl start filebeat[root@elk_3 ~]# systemctl start filebeat
在elk--1主机上使用curl '192.168.40.11:9200/_cat/indices?v'命令查看是否监听到elk-3主机上的日志(出现filebeat字样表示成功):
[root@elk--1 ~]# curl '192.168.233.201:9200/_cat/indices?v' health status index uuid pri rep d green open system-syslog-2022.01 tAd3L8W6RHqDeRDOvOdrTA 5 1 green open .kibana HJ8wt6BuQaeVcc5Vf1zbcg 1 1 green open filebeat-6.0.0-2022.01.14 drJ2X8boTR-omVjIqFhc6g 3 1 green open Nginx-test-2022.01.14 vzFiklXgRP-GenN_KMe87A 5 1
3. Web界面配置
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。