I recently set up Elasticsearch, Kibana, and Suricata in my homelab in docker containers utilizing filebeat to ship the logs from Suricata’s eve.json to Elasticsearch. This was the process.

Why

First off, why did I do this?

This was a project I wanted to do as a precursor to bigger things. The benefits of running containers are great for development environments and at scale. My next project is to tear this down and set it up on a larger scale using Kubernetes. My long term goal is to get my Certified Kubernetes Administrator cert.

This was a way to validate what was in my head and get more comfortable with Docker. I’ll let it run for awhile while I work through some pluralsight kubernetes courses and then I’ll tear it down and do it again in Kubernetes.

The Benefits of Containerization

  1. Portability: You can run a Docker image almost anywhere as long as Docker is installed on the host.
  2. Better resource utilization: When running VM’s, each VM needs a host OS which utilizes resources. Again, on a small scale this doesn’t matter, but in a large scale environment this is a lot of resources lost. With containers, you don’t need to waste resources on multiple OS’s. This site has a good image to explain what I’m talking about.
  3. Microservices: Containers are great for developing Microservices.
There are a lot more benefits to Containerization than I don’t want to take the time go into here. If you’re interested here are a few links to go deeper: Here and Here

Containers really shine in the cloud. All cloud platforms support container orchestration with Kubernetes or a proprietary orchestration method (AWS has ECS and EKS, Azure and GCP have their own version as well).

Downloading Docker

To start, you need to make sure you have docker.

If on Debian:

First you need to remove the any old docker installs and set up the docker repository:

Copy to Clipboard

Now you can install the docker engine. 

Copy to Clipboard

The following command will test if your docker install was successful:

Copy to Clipboard

If on Red Hat:

We first have to uninstall old versions and add the repository

Copy to Clipboard

We also need to remove the podman and buildah repositories if we want to use Docker instead of podman (podman is RHELs default install for running containers)

Copy to Clipboard

Now we can install Docker:

Copy to Clipboard

Start Docker and test the installation:

Copy to Clipboard

Housekeeping

I did this on an ubuntu server that I’m running on ESXI. Keep in mind the OS you’re running on when looking at my commands.

Capture Interface

First you need to set up a capture interface (ex. a SPAN port off of a managed switch). You need two interfaces on your box for this: your capture interface and a management interface. On ESXI when you create the capture interface you need to create a new virtual switch and enable promiscous mode. Then in the VM you need to run:

Copy to Clipboard

Now check to see if you’re collecting traffic on the interface with:

Copy to Clipboard

You should see traffic after a few seconds. CTRL+C to exit the tcpdump.

Create a Directory

In my home directory I have a Docker directory for my Docker projects.. Inside that Docker directory I created an ELK directory and inside there I created a suricata directory. Once you have that done and you’ve installed docker you should be ready to start.

VM Memory Count

First, you have to run:

Copy to Clipboard

If you don’t do this your containers will crash.

Elasticsearch & Kibana

I installed elasticsearch and kibana in docker containers as shown here.

The steps from the link above (assuming you have docker downloaded already) are:

Copy to Clipboard

Instead of running the containers with the -it flag, which runs them in the terminal, you should instead run them with the -d flag which will run them in the background.

You should now be able to navigate to kibana at http://<yourip>:5601

Then, to get the enrollment token for Kibana you can run a docker logs es01 If that doesn’t work you can run the following commands to get the enrollment token and the elastic password.

Copy to Clipboard
Copy to Clipboard

Suricata

Once Elasticsearch and Kibana are up and running, you can install suricata. I did that here. The steps from the link above are:

Copy to Clipboard

One note: you want to use the -v option as it’s shown because that will save the eve.json logs to your local box (in my case my ubuntu server). This allows your logs to persist through container failurs/restarts/shutdowns/etc.

Filebeat as a service

Now for filebeat. The documentation on installing filebeat is here. You can run filebeat in a container but since the logs are saved to the localhost I decided to run filebeat directly on the ubuntu server for now.

The filebeat.yml should be in /etc/filebeat

When configuring filebeat, make sure you edit the filebeat.yml and pass it a username and password for elastic as well as make sure you specify the path as https and enable ssl. An example filebeat.yml:

Copy to Clipboard

To generate the fingerprint needed for ssl. Refer to this documentation specifically:

Copy to Clipboard

 You should be able to run this command in the directory you ran the following command:

docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .

You then need to cd into modules.d and edit the suricata.yml to enable it (set it to true) and give it the path of your eve.json (the path on your local host).

Sample suricata.yml:

Copy to Clipboard

You now have to run

Copy to Clipboard
 to set up the index template and some dashboards.

Once thats done you should be able to start the service. On ubuntu the command is

Copy to Clipboard

If you cat eve.json and there’s logs in there you should now be seeing those logs in Kibana.

One Comment

  1. Very good article. I definitely appreciate this website. Continue the good work!

Leave A Comment