{"id":11679,"date":"2019-03-15T21:45:01","date_gmt":"2019-03-15T21:45:01","guid":{"rendered":"http:\/\/www.appservgrid.com\/paw92\/?p=11679"},"modified":"2019-03-28T02:02:46","modified_gmt":"2019-03-28T02:02:46","slug":"how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-rhel-7","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2019\/03\/15\/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-rhel-7\/","title":{"rendered":"How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS\/RHEL 7"},"content":{"rendered":"<p>If you are a person who is, or has been in the past, in charge of inspecting and analyzing system logs in Linux, you know what a nightmare that task can become if multiple services are being monitored simultaneously.<\/p>\n<p>In days past, that task had to be done mostly manually, with each log type being handled separately. Fortunately, the combination of\u00a0<strong>Elasticsearch<\/strong>,\u00a0<strong>Logstash<\/strong>, and\u00a0<strong>Kibana<\/strong>\u00a0on the server side, along with\u00a0<strong>Filebeat<\/strong>\u00a0on the client side, makes that once difficult task look like a walk in the park today.<\/p>\n<p>The first three components form what is called an\u00a0<strong>ELK<\/strong>\u00a0stack, whose main purpose is to collect logs from multiple servers at the same time (also known as centralized logging).<\/p>\n<p><b>Suggested Read:<\/b>\u00a0<a href=\"https:\/\/www.tecmint.com\/best-linux-log-monitoring-and-management-tools\/\" target=\"_blank\" rel=\"noopener\">4 Good Open Source Log Monitoring and Management Tools for Linux<\/a><\/p>\n<p>A built-in java-based web interface allows you to inspect logs quickly at a glance for easier comparison and troubleshooting. These client logs are sent to a central server by\u00a0<strong>Filebeat<\/strong>, which can be described as a log shipping agent.<\/p>\n<p><center><ins class=\"adsbygoogle\" data-ad-client=\"ca-pub-2601749019656699\" data-ad-slot=\"5590002574\" data-ad-format=\"auto\" data-adsbygoogle-status=\"done\"><ins id=\"aswift_4_expand\"><ins id=\"aswift_4_anchor\"><iframe id=\"aswift_4\" name=\"aswift_4\" width=\"780\" height=\"90\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" allowfullscreen=\"allowfullscreen\" data-mce-fragment=\"1\"><\/iframe><\/ins><\/ins><\/ins><\/center>Let\u2019s see how all of these pieces fit together. Our test environment will consist of the following machines:<\/p>\n<pre><strong>Central Server<\/strong>: <strong>CentOS 7<\/strong> (IP address: <strong>192.168.0.29<\/strong>). <strong>2 GB<\/strong> of RAM.\r\n<strong>Client #1<\/strong>: <strong>CentOS 7<\/strong> (IP address: <strong>192.168.0.100<\/strong>). <strong>1 GB<\/strong> of RAM.\r\n<strong>Client #2<\/strong>: <strong>Debian 8<\/strong> (IP address: <strong>192.168.0.101<\/strong>). <strong>1 GB<\/strong> of RAM.\r\n<\/pre>\n<p>Please note that the\u00a0<strong>RAM<\/strong>\u00a0values provided here are not strict prerequisites, but recommended values for successful implementation of the\u00a0<strong>ELK<\/strong>\u00a0stack on the central server. Less\u00a0<strong>RAM<\/strong>\u00a0on clients will not make much difference, if any, at all.<\/p>\n<h3>Installing ELK Stack on the Server<\/h3>\n<p>Let\u2019s begin by installing the\u00a0<strong>ELK<\/strong>\u00a0stack on the server, along with a brief explanation on what each component does:<\/p>\n<ol>\n<li><strong>Elasticsearch<\/strong>\u00a0stores the logs that are sent by the clients.<\/li>\n<li><strong>Logstash<\/strong>\u00a0processes those logs.<\/li>\n<li><strong>Kibana<\/strong>\u00a0provides the web interface that will help us to inspect and analyze the logs.<\/li>\n<\/ol>\n<p>Install the following packages on the central server. First off, we will install\u00a0<strong>Java JDK<\/strong>\u00a0version\u00a0<strong>8<\/strong>\u00a0(update\u00a0<strong>102<\/strong>, the latest one at the time of this writing), which is a dependency of the\u00a0<strong>ELK<\/strong>\u00a0components.<\/p>\n<p>You may want to check first in the Java downloads page\u00a0<a href=\"http:\/\/www.oracle.com\/technetwork\/java\/javase\/downloads\/jre8-downloads-2133155.html\" target=\"_blank\" rel=\"nofollow noopener\">here<\/a>\u00a0to see if there is a newer update available.<\/p>\n<pre># yum update\r\n# cd \/opt\r\n# wget --no-cookies --no-check-certificate --header \"Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie\" \"http:\/\/download.oracle.com\/otn-pub\/java\/jdk\/8u102-b14\/jre-8u102-linux-x64.rpm\"\r\n# rpm -Uvh jre-8u102-linux-x64.rpm\r\n<\/pre>\n<p>Time to check whether the installation completed successfully:<\/p>\n<pre># java -version\r\n<\/pre>\n<div id=\"attachment_22566\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Check-Java-Version-Commandline.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22566\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Check-Java-Version-Commandline.png\" alt=\"Check Java Version from Commandline\" width=\"518\" height=\"92\" aria-describedby=\"caption-attachment-22566\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22566\" class=\"wp-caption-text\">Check Java Version from Commandline<\/p>\n<\/div>\n<p>To install the latest versions of\u00a0<strong>Elasticsearch<\/strong>,\u00a0<strong>Logstash<\/strong>, and\u00a0<strong>Kibana<\/strong>, we will have to create repositories for\u00a0<strong>yum<\/strong>manually as follows:<\/p>\n<h4>Enable Elasticsearch Repository<\/h4>\n<p><strong>1.<\/strong>\u00a0Import the Elasticsearch public GPG key to the rpm package manager:<\/p>\n<pre># rpm --import http:\/\/packages.elastic.co\/GPG-KEY-elasticsearch\r\n<\/pre>\n<p><strong>2.<\/strong>\u00a0Insert the following lines to the repository configuration file\u00a0<code>elasticsearch.repo<\/code>:<\/p>\n<div class=\"code-label\" title=\"Create Elasticsearch Repository\">\/etc\/yum.repos.d\/elasticsearch.repo<\/div>\n<pre>[elasticsearch]\r\nname=Elasticsearch repository\r\nbaseurl=http:\/\/packages.elastic.co\/elasticsearch\/2.x\/centos\r\ngpgcheck=1\r\ngpgkey=http:\/\/packages.elastic.co\/GPG-KEY-elasticsearch\r\nenabled=1\r\n<\/pre>\n<p><strong>3.<\/strong>\u00a0Install the Elasticsearch package.<\/p>\n<pre># yum install elasticsearch\r\n<\/pre>\n<p>When the installation is complete, you will be prompted to start and enable elasticsearch:<\/p>\n<div id=\"attachment_22568\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Install-Elasticsearch-in-Linux.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22568\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Install-Elasticsearch-in-Linux.png\" alt=\"Install Elasticsearch in Linux\" width=\"487\" height=\"204\" aria-describedby=\"caption-attachment-22568\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22568\" class=\"wp-caption-text\">Install Elasticsearch in Linux<\/p>\n<\/div>\n<p><strong>4.<\/strong>\u00a0Start and enable the service.<\/p>\n<pre># systemctl daemon-reload\r\n# systemctl enable elasticsearch\r\n# systemctl start elasticsearch\r\n<\/pre>\n<p><strong>5.<\/strong>\u00a0Allow traffic through TCP port\u00a0<strong>9200<\/strong>\u00a0in your firewall:<\/p>\n<pre># firewall-cmd --add-port=9200\/tcp\r\n# firewall-cmd --add-port=9200\/tcp --permanent\r\n<\/pre>\n<p><strong>6.<\/strong>\u00a0Check if\u00a0<strong>Elasticsearch<\/strong>\u00a0responds to simple requests over\u00a0<strong>HTTP<\/strong>:<\/p>\n<pre># curl -X GET http:\/\/localhost:9200\r\n<\/pre>\n<p>The output of the above command should be similar to:<\/p>\n<div id=\"attachment_22569\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Verify-Elasticsearch-Installation.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22569\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Verify-Elasticsearch-Installation.png\" alt=\"Verify Elasticsearch Installation\" width=\"500\" height=\"243\" aria-describedby=\"caption-attachment-22569\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22569\" class=\"wp-caption-text\">Verify Elasticsearch Installation<\/p>\n<\/div>\n<p>Make sure you complete the above steps and then proceed with\u00a0<strong>Logstash<\/strong>. Since both\u00a0<strong>Logstash<\/strong>\u00a0and\u00a0<strong>Kibana<\/strong>share the\u00a0<strong>Elasticsearch GPG<\/strong>\u00a0key, there is no need to re-import it before installing the packages.<\/p>\n<p><b>Suggested Read:<\/b>\u00a0<a href=\"https:\/\/www.tecmint.com\/manage-linux-system-logs-using-rsyslogd-and-logrotate\/\" target=\"_blank\" rel=\"noopener\">Manage System Logs (Configure, Rotate and Import Into Database) in CentOS 7<\/a><\/p>\n<h4>Enable Logstash Repository<\/h4>\n<p><strong>7.<\/strong>\u00a0Insert the following lines to the repository configuration file\u00a0<code>logstash.repo<\/code>:<\/p>\n<div class=\"code-label\" title=\"Create Elasticsearch Repository\">\/etc\/yum.repos.d\/logstash.repo<\/div>\n<pre>[logstash]\r\nname=Logstash\r\nbaseurl=http:\/\/packages.elasticsearch.org\/logstash\/2.2\/centos\r\ngpgcheck=1\r\ngpgkey=http:\/\/packages.elasticsearch.org\/GPG-KEY-elasticsearch\r\nenabled=1\r\n<\/pre>\n<p><strong>8.<\/strong>\u00a0Install the\u00a0<strong>Logstash<\/strong>\u00a0package:<\/p>\n<pre># yum install logstash\r\n<\/pre>\n<p><strong>9.<\/strong>\u00a0Add a\u00a0<strong>SSL<\/strong>\u00a0certificate based on the IP address of the ELK server at the the following line below the\u00a0<code>[ v3_ca ]<\/code>\u00a0section in\u00a0<code>\/etc\/pki\/tls\/openssl.cnf<\/code>:<\/p>\n<pre>[ v3_ca ]\r\nsubjectAltName = IP: <strong>192.168.0.29<\/strong>\r\n<\/pre>\n<div id=\"attachment_22570\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Add-Elasticsearch-Server-IP.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22570\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Add-Elasticsearch-Server-IP.png\" alt=\"Add Elasticsearch Server IP Address\" width=\"289\" height=\"83\" aria-describedby=\"caption-attachment-22570\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22570\" class=\"wp-caption-text\">Add Elasticsearch Server IP Address<\/p>\n<\/div>\n<p><strong>10.<\/strong>\u00a0Generate a self-signed certificate valid for\u00a0<strong>365<\/strong>\u00a0days:<\/p>\n<pre># cd \/etc\/pki\/tls\r\n# openssl req -config \/etc\/pki\/tls\/openssl.cnf -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private\/logstash-forwarder.key -out certs\/logstash-forwarder.crt<\/pre>\n<p><strong>11.<\/strong>\u00a0Configure\u00a0<strong>Logstash<\/strong>\u00a0input, output, and filter files:<\/p>\n<p><strong>Input<\/strong>: Create\u00a0<code>\/etc\/logstash\/conf.d\/input.conf<\/code>\u00a0and insert the following lines into it. This is necessary for Logstash to \u201c<strong>learn<\/strong>\u201d how to process beats coming from clients. Make sure the path to the certificate and key match the right paths as outlined in the previous step:<\/p>\n<div class=\"code-label\" title=\"Configure Logstash\">\/etc\/logstash\/conf.d\/input.conf<\/div>\n<pre>input {\r\n  beats {\r\n\tport =&gt; 5044\r\n\tssl =&gt; true\r\n\tssl_certificate =&gt; \"\/etc\/pki\/tls\/certs\/logstash-forwarder.crt\"\r\n\tssl_key =&gt; \"\/etc\/pki\/tls\/private\/logstash-forwarder.key\"\r\n  }\r\n}\r\n<\/pre>\n<p>Output (<code>\/etc\/logstash\/conf.d\/output.conf<\/code>) file:<\/p>\n<div class=\"code-label\" title=\"Configure Logstash\">\/etc\/logstash\/conf.d\/output.conf<\/div>\n<pre>output {\r\n  elasticsearch {\r\n\thosts =&gt; [\"localhost:9200\"]\r\n\tsniffing =&gt; true\r\n\tmanage_template =&gt; false\r\n\tindex =&gt; \"%{[@metadata][beat]}-%{+YYYY.MM.dd}\"\r\n\tdocument_type =&gt; \"%{[@metadata][type]}\"\r\n  }\r\n}\r\n<\/pre>\n<p>Filter (<code>\/etc\/logstash\/conf.d\/filter.conf<\/code>) file. We will log syslog messages for simplicity:<\/p>\n<div class=\"code-label\" title=\"Configure Logstash\">\/etc\/logstash\/conf.d\/filter.conf<\/div>\n<pre>filter {\r\nif [type] == \"syslog\" {\r\n\tgrok {\r\n  \tmatch =&gt; { \"message\" =&gt; \"%{SYSLOGLINE}\" }\r\n\t}\r\n\r\n\tdate {\r\nmatch =&gt; [ \"timestamp\", \"MMM  d HH:mm:ss\", \"MMM dd HH:mm:ss\" ]\r\n}\r\n  }\r\n}\r\n<\/pre>\n<p><strong>12.<\/strong>\u00a0Verify the\u00a0<strong>Logstash<\/strong>\u00a0configuration files.<\/p>\n<pre># service logstash configtest\r\n<\/pre>\n<div id=\"attachment_22571\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Check-Logstash-Configuration.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22571\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Check-Logstash-Configuration.png\" alt=\"Verify Logstash Configuration\" width=\"384\" height=\"54\" aria-describedby=\"caption-attachment-22571\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22571\" class=\"wp-caption-text\">Verify Logstash Configuration<\/p>\n<\/div>\n<p><strong>13.<\/strong>\u00a0Start and enable logstash:<\/p>\n<pre># systemctl daemon-reload\r\n# systemctl start logstash\r\n# systemctl enable logstash\r\n<\/pre>\n<p><strong>14.<\/strong>\u00a0Configure the firewall to allow Logstash to get the logs from the clients (TCP port\u00a0<strong>5044<\/strong>):<\/p>\n<pre># firewall-cmd --add-port=5044\/tcp\r\n# firewall-cmd --add-port=5044\/tcp --permanent\r\n<\/pre>\n<h4>Enable Kibana Repository<\/h4>\n<p><strong>14.<\/strong>\u00a0Insert the following lines to the repository configuration file\u00a0<code>kibana.repo<\/code>:<\/p>\n<div class=\"code-label\" title=\"Create Kibana Repository\">\/etc\/yum.repos.d\/kibana.repo<\/div>\n<pre>[kibana]\r\nname=Kibana repository\r\nbaseurl=http:\/\/packages.elastic.co\/kibana\/4.4\/centos\r\ngpgcheck=1\r\ngpgkey=http:\/\/packages.elastic.co\/GPG-KEY-elasticsearch\r\nenabled=1\r\n<\/pre>\n<p><strong>15.<\/strong>\u00a0Install the\u00a0<strong>Kibana<\/strong>\u00a0package:<\/p>\n<pre># yum install kibana\r\n<\/pre>\n<p><strong>16.<\/strong>\u00a0Start and enable Kibana.<\/p>\n<pre># systemctl daemon-reload\r\n# systemctl start kibana\r\n# systemctl enable kibana\r\n<\/pre>\n<p><strong>17.<\/strong>\u00a0Make sure you can access access Kibana\u2019s web interface from another computer (allow traffic on TCP port\u00a0<strong>5601<\/strong>):<\/p>\n<pre># firewall-cmd --add-port=5601\/tcp\r\n# firewall-cmd --add-port=5601\/tcp --permanent\r\n<\/pre>\n<p><strong>18.<\/strong>\u00a0Launch\u00a0<strong>Kibana<\/strong>\u00a0(<code>http:\/\/192.168.0.29:5601<\/code>) to verify that you can access the web interface:<\/p>\n<div id=\"attachment_22572\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Access-Kibana-Web-Interface.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22572\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Access-Kibana-Web-Interface.png\" alt=\"Access Kibana Web Interface\" width=\"635\" height=\"254\" aria-describedby=\"caption-attachment-22572\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22572\" class=\"wp-caption-text\">Access Kibana Web Interface<\/p>\n<\/div>\n<p>We will return here after we have installed and configured\u00a0<strong>Filebeat<\/strong>\u00a0on the clients.<\/p>\n<p><b>Suggested Read:<\/b>\u00a0<a href=\"https:\/\/www.tecmint.com\/linux-server-log-monitoring-with-log-io\/\" target=\"_blank\" rel=\"noopener\">Monitor Server Logs in Real-Time with \u201cLog.io\u201d Tool in Linux<\/a><\/p>\n<h3>Install Filebeat on the Client Servers<\/h3>\n<p>We will show you how to do this for\u00a0<strong>Client #1<\/strong>\u00a0(repeat for\u00a0<strong>Client #2<\/strong>\u00a0afterwards, changing paths if applicable to your distribution).<\/p>\n<p><strong>1.<\/strong>\u00a0Copy the SSL certificate from the server to the clients:<\/p>\n<pre># scp \/etc\/pki\/tls\/certs\/logstash-forwarder.crt root@192.168.0.100:\/etc\/pki\/tls\/certs\/\r\n<\/pre>\n<p><strong>2.<\/strong>\u00a0Import the\u00a0<strong>Elasticsearch<\/strong>\u00a0public GPG key to the rpm package manager:<\/p>\n<pre># rpm --import http:\/\/packages.elastic.co\/GPG-KEY-elasticsearch\r\n<\/pre>\n<p><strong>3.<\/strong>\u00a0Create a repository for\u00a0<strong>Filebeat<\/strong>\u00a0(<code>\/etc\/yum.repos.d\/filebeat.repo<\/code>) in\u00a0<strong>CentOS<\/strong>\u00a0based distributions:<\/p>\n<div class=\"code-label\" title=\"Create Filebeat Repository\">\/etc\/yum.repos.d\/filebeat.repo<\/div>\n<pre>[filebeat]\r\nname=Filebeat for ELK clients\r\nbaseurl=https:\/\/packages.elastic.co\/beats\/yum\/el\/$basearch\r\nenabled=1\r\ngpgkey=https:\/\/packages.elastic.co\/GPG-KEY-elasticsearch\r\ngpgcheck=1\r\n<\/pre>\n<p><strong>4.<\/strong>\u00a0Configure the source to install Filebeat on\u00a0<strong>Debian<\/strong>\u00a0and its derivatives:<\/p>\n<pre># aptitude install apt-transport-https\r\n# echo \"deb https:\/\/packages.elastic.co\/beats\/apt stable main\" &gt; \/etc\/apt\/sources.list.d\/filebeat.list\r\n# aptitude update\r\n<\/pre>\n<p><strong>5.<\/strong>\u00a0Install the\u00a0<strong>Filebeat<\/strong>\u00a0package:<\/p>\n<pre># yum install filebeat        [On <strong>CentOS<\/strong> and based Distros]\r\n# aptitude install filebeat   [On <strong>Debian<\/strong> and its derivatives]\r\n<\/pre>\n<p><strong>6.<\/strong>\u00a0Start and enable Filebeat:<\/p>\n<pre># systemctl start filebeat\r\n# systemctl enable filebeat\r\n<\/pre>\n<h4>Configure Filebeat<\/h4>\n<p>A word of caution here.\u00a0<strong>Filebeat<\/strong>\u00a0configuration is stored in a\u00a0<strong>YAML<\/strong>\u00a0file, which requires strict indentation. Be careful with this as you edit\u00a0<code>\/etc\/filebeat\/filebeat.yml<\/code>\u00a0as follows:<\/p>\n<ol>\n<li>Under\u00a0<strong>paths<\/strong>, indicate which log files should be \u201cshipped\u201d to the ELK server.<\/li>\n<li>Under\u00a0<strong>prospectors<\/strong>:<\/li>\n<\/ol>\n<pre>input_type: log\r\ndocument_type: syslog\r\n<\/pre>\n<ol>\n<li>Under\u00a0<strong>output<\/strong>:\n<ol>\n<li>Uncomment the line that begins with\u00a0<strong>logstash<\/strong>.<\/li>\n<li>Indicate the IP address of your ELK server and port where Logstash is listening in\u00a0<strong>hosts<\/strong>.<\/li>\n<li>Make sure the path to the certificate points to the actual file you created in\u00a0<strong>Step I<\/strong>\u00a0(<strong>Logstash<\/strong>\u00a0section) above.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p>The above steps are illustrated in the following image:<\/p>\n<div id=\"attachment_22575\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Configure-Filebeat-in-Client-Servers.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22575\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Configure-Filebeat-in-Client-Servers.png\" sizes=\"auto, (max-width: 853px) 100vw, 853px\" srcset=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Configure-Filebeat-in-Client-Servers.png 853w, https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Configure-Filebeat-in-Client-Servers-768x114.png 768w\" alt=\"Configure Filebeat in Client Servers\" width=\"853\" height=\"127\" aria-describedby=\"caption-attachment-22575\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22575\" class=\"wp-caption-text\">Configure Filebeat in Client Servers<\/p>\n<\/div>\n<p>Save changes, and then restart\u00a0<strong>Filebeat<\/strong>\u00a0on the clients:<\/p>\n<pre># systemctl restart filebeat\r\n<\/pre>\n<p>Once we have completed the above steps on the clients, feel free to proceed.<\/p>\n<h4>Testing Filebeat<\/h4>\n<p>In order to verify that the logs from the clients can be sent and received successfully, run the following command on the\u00a0<strong>ELK<\/strong>\u00a0server:<\/p>\n<pre># curl -XGET 'http:\/\/localhost:9200\/filebeat-*\/_search?pretty'\r\n<\/pre>\n<p>The output should be similar to (notice how messages from\u00a0<strong>\/var\/log\/messages<\/strong>\u00a0and\u00a0<strong>\/var\/log\/secure<\/strong>\u00a0are being received from\u00a0<strong>client1<\/strong>\u00a0and\u00a0<strong>client2<\/strong>):<\/p>\n<div id=\"attachment_22574\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Testing-Filebeat.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22574\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Testing-Filebeat.png\" sizes=\"auto, (max-width: 793px) 100vw, 793px\" srcset=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Testing-Filebeat.png 793w, https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Testing-Filebeat-768x269.png 768w\" alt=\"Testing Filebeat\" width=\"793\" height=\"278\" aria-describedby=\"caption-attachment-22574\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22574\" class=\"wp-caption-text\">Testing Filebeat<\/p>\n<\/div>\n<p>Otherwise, check the\u00a0<strong>Filebeat<\/strong>\u00a0configuration file for errors.<\/p>\n<pre># journalctl -xe\r\n<\/pre>\n<p>after attempting to restart Filebeat will point you to the offending line(s).<\/p>\n<h4>Testing Kibana<\/h4>\n<p>After we have verified that logs are being shipped by the clients and received successfully on the server. The first thing that we will have to do in\u00a0<strong>Kibana<\/strong>\u00a0is configuring an index pattern and set it as default.<\/p>\n<p>You can describe an index as a full database in a relational database context. We will go with\u00a0<code>filebeat-*<\/code>\u00a0(or you can use a more precise search criteria as explained in\u00a0<a href=\"https:\/\/www.elastic.co\/guide\/en\/kibana\/current\/settings.html\" target=\"_blank\" rel=\"nofollow noopener\">the official documentation<\/a>).<\/p>\n<p>Enter\u00a0<code>filebeat-*<\/code>\u00a0in the\u00a0<strong>Index name<\/strong>\u00a0or\u00a0<strong>pattern field<\/strong>\u00a0and then click\u00a0<strong>Create<\/strong>:<\/p>\n<div id=\"attachment_22576\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Testing-Kibana.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22576\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Testing-Kibana.png\" alt=\"Testing Kibana\" width=\"575\" height=\"632\" aria-describedby=\"caption-attachment-22576\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22576\" class=\"wp-caption-text\">Testing Kibana<\/p>\n<\/div>\n<p>Please note that you will be allowed to enter a more fine-grained search criteria later. Next, click the star inside the green rectangle to configure it as the default index pattern:<\/p>\n<div id=\"attachment_22577\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Configure-Default-Kibana-Index-Pattern.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22577\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Configure-Default-Kibana-Index-Pattern.png\" alt=\"Configure Default Kibana Index Pattern\" width=\"667\" height=\"132\" aria-describedby=\"caption-attachment-22577\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22577\" class=\"wp-caption-text\">Configure Default Kibana Index Pattern<\/p>\n<\/div>\n<p>Finally, in the\u00a0<strong>Discover<\/strong>\u00a0menu you will find several fields to add to the log visualization report. Just hover over them and click\u00a0<strong>Add<\/strong>:<\/p>\n<div id=\"attachment_22578\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Add-Log-Visualization-Report.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22578\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Add-Log-Visualization-Report.png\" alt=\"Add Log Visualization Report\" width=\"506\" height=\"476\" aria-describedby=\"caption-attachment-22578\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22578\" class=\"wp-caption-text\">Add Log Visualization Report<\/p>\n<\/div>\n<p>The results will be shown in the central area of the screen as shown above. Feel free to play around (add and remove fields from the log report) to become familiar with\u00a0<strong>Kibana<\/strong>.<\/p>\n<p>By default,\u00a0<strong>Kibana<\/strong>\u00a0will display the records that were processed during the last\u00a0<strong>15<\/strong>\u00a0minutes (see upper right corner) but you can change that behavior by selecting another time frame:<\/p>\n<div id=\"attachment_22579\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Kibana-Log-Reports.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-22579\" src=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Kibana-Log-Reports.png\" sizes=\"auto, (max-width: 838px) 100vw, 838px\" srcset=\"https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Kibana-Log-Reports.png 838w, https:\/\/www.tecmint.com\/wp-content\/uploads\/2016\/09\/Kibana-Log-Reports-768x339.png 768w\" alt=\"Kibana Log Reports\" width=\"838\" height=\"370\" aria-describedby=\"caption-attachment-22579\" data-lazy-loaded=\"true\" \/><\/a><\/p>\n<p id=\"caption-attachment-22579\" class=\"wp-caption-text\">Kibana Log Reports<\/p>\n<\/div>\n<h3>Summary<\/h3>\n<p>In this article we have explained how to set up an ELK stack to collect the system logs sent by two clients, a CentOS 7 and a Debian 8 machines.<\/p>\n<p>Now you can refer to\u00a0<a href=\"https:\/\/www.elastic.co\/guide\/index.html\" target=\"_blank\" rel=\"noopener\">the official Elasticsearch documentation<\/a>\u00a0and find more details on how to use this setup to inspect and analyze your logs more efficiently.<\/p>\n<p>If you have any questions, don\u2019t hesitate to ask. We look forward to hearing from you.<\/p>\n<p><a href=\"https:\/\/www.tecmint.com\/install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-rhel-7\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you are a person who is, or has been in the past, in charge of inspecting and analyzing system logs in Linux, you know what a nightmare that task can become if multiple services are being monitored simultaneously. In days past, that task had to be done mostly manually, with each log type being &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw92\/index.php\/2019\/03\/15\/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-rhel-7\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS\/RHEL 7&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-11679","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11679","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/comments?post=11679"}],"version-history":[{"count":3,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11679\/revisions"}],"predecessor-version":[{"id":12574,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/11679\/revisions\/12574"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/media?parent=11679"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/categories?post=11679"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/tags?post=11679"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}