Install and Configure Filebeat on clients Ubuntu 18.04 or centos 7

In this step, we will configure the Ubuntu 18.04 client NGINX50 by installing the Elastic Beats data shippers Filebeat on it.

Before installing the filebeat to the system, we need to edit the /etc/hosts and/or add a DNS entry so that the client can resolve the elk host, we also need to download the logstash certificate file logstash-forwarder.crt file to the NGINX50 server.

Edit the /etc/hosts file using vim editor. Paste the following configuration there.

Alternatively, place a DNS entry and let DNS work its magic

# sudo vim /etc/hosts

192.168.20.15      elk.t3st.org elk

Install the Elastic Beats ‘Filebeat’

see:

[Repositories for APT and YUM | Filebeat Reference [6.6] Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html)
[Step 1: Install Filebeat | Filebeat Reference [6.6] Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html)

Ubuntu / Debian

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.0-amd64.deb
sudo dpkg -i filebeat-6.6.0-amd64.deb

RHEL / Centos

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.0-x86_64.rpm
sudo rpm -vi filebeat-6.6.0-x86_64.rpm

Copy the logstash certificate

Copy the logstash certificate file logstash-forwarder.crt using scp command:

# scp from client machine
scp root@elk:/etc/logstash/ssl/logstash-forwarder.crt .

or the other way around, scp from elk host to the client:

# scp from elk host
scp /etc/logstash/ssl/logstash-forwarder.* [email protected]:/home/armand/

Move Cert file into the ‘/etc/filebeat/’ directory:

mv /home/armand/logstash-forwarder.* /etc/filebeat                  

ls /etc/filebeat
logstash-forwarder.crt
logstash-forwarder.key 

Configure filebeat

After the installation is complete, go to the /etc/filebeat directory and edit the configuration file filebeat.yml:

# sudo vim /etc/filebeat/filebeat.yml
# Now enable the filebeat prospectors by changing the 'enabled' line value to 'true'.
enabled: true


# Define system log files to be sent to the logstash server. For this guide, we will add the ssh log file 'auth.log' and the syslog file.

  paths:
    - /var/log/nginx/*.log
  exclude_files: ['\.gz$']

#comment out elasticsearch output
#output.elasticsearch:
  # Array of hosts to connect to.
  #  hosts: ["localhost:9200"]

# Setup the output to logstash by commenting the default 'elasticsearch' output and uncomment the logstash output line as below.

output.logstash:
  # The Logstash hosts
  hosts: ["elk:5400"]
  ssl.certificate_authorities: ["/etc/filebeat/ca.crt"]
  ssl.certificate: "/etc/filebeat/beat.crt"
  ssl.key: "/etc/filebeat/beat.key"

Enable modules

see [Specify which modules to run | Filebeat Reference [master]

Elastic](https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-modules.html#enable-modules-d-configs)

Enable module configs in the modules.d directory:

filebeat modules enable nginx system                           

Enabled nginx
Enabled system

To see a list of enabled and disabled modules, run filebeat modules list

filebeat modules list

Enabled:
nginx
system

Disabled:
apache2
auditd
elasticsearch
haproxy
icinga
iis
kafka
kibana
logstash
mongodb
mysql
osquery
postgresql
redis
suricata
traefik

Filebeat installation and configuration have been completed. Now start the filebeat service and enable it to launch every time at system boot.

service filebeat stop
service filebeat start
systemctl enable filebeat
tail -f /var/log/filebeat/filebeat

Troublehooting

You can check if data is contained in a filebeat-YYYY.MM.dd index in Elasticsearch using a curl command that will print the event count.

curl http://localhost:9200/filebeat-*/_count?pretty

Check the filebeat service using commands below.

tail -f /var/log/filebeat/filebeat