I will walk through how to build your own fluentd container which can be used to send logs to VMware Log Intelligence
Earlier blogs had procedure listed how to install fluentd on linux and windows using rpms.
Linux – Click Here
Windows – Click Here
Use Cases
This container helps with following use cases
- Centralized Syslog Receiver on standard syslog port (514) and forwards to VMware Log Intelligence
- Centralized App Logs Receiver on HTTP port 9880 and forwards to VMware Log Intelligence
- Centralized Server to receive events from other fluentd agents over tcp port 24224 and forwards to VMware Log Intelligence
VMware Log Intelligence is a Cloud based service which means you will need to send logs over WAN and users may not want to send it from all the Machines hence having centralized server which sends out to Cloud would be a good idea
Steps
We will be using https://github.com/fluent/fluentd-docker-image and modify to install http-out-ext plugin which is needed to send logs to VMware Log Intelligence
Execute the following command to clone the fluentd-docker-image repo
git clone https://github.com/fluent/fluentd-docker-image.git
You will notice it will create a folder \”fluentd-docker-image\”
I will be using fluentd version 1.2 which latest with Debian image as it is recommended one
Navigate to cd fluentd-docker-image/v1.2/Debian
You will see Dockerfile
The only part which we will be adding is to install the http-out-ext plugin in the docker image
Add following line in the Docker file
&&gem install fluent-plugin-out-http-ext -v 0.1.10 \\
Next, we will run docker build to build the container image
docker build -t docker.io/mmakhija/fluentdcontainer:v0 ./
A couple of points to note
- The reason I am choosing this name is to that I can push the container name to public docker hub. You don\’t need to do it unless you want to share it with others. If you have your own private repo you can use that
- mmakhija – docker hub username
- fluentdcontainer – repo name on docker hub
- It will take a couple of mins to build the docker. It will show some red color. Don\’t be scared it like I was when I did it for the first time ☺
You can view the image by executing the following command
docker images
Next, you will need to copy fluentd.conf listed below on the docker host. Please remember the path where you are saving it because you will need to specify that when running docker run command.
I have saved the file at which I have specified below \”/fluent/fluentd.conf\”
https://github.com/munishpalmakhija/fluentd/blob/master/fluent.conf
Execute the following command to run the fluentd docker container.
docker run -d -p 24224:24224 -p 24224:24224/udp -p 514:5140 -p 514:5140/udp -p 9880:9880 -v /fluent:/fluentd/etc/ mmakhija/fluentdcontainer:v0
Please ensure to match the name of the image which you had it when you ran docker build command and the path of the fluent.conf file
If everything goes you will see a fluentd container running as below and you can configure ESXi host to forward logs to ip of the docker host where this container is running on port 514 and you should see logs flowing
Please feel free to leave comments or suggestions of if something not working
I really admired your post, such great and useful information that you have mentioned here. AWS Training in Chennai AWS course in Chennai Cloud Computing Training in Chennai Cloud Computing courses in Chennai VMware Training in Chennai VMware course in Chennai DevOps Training in Chennai AWS Training in Chennai AWS course in Chennai
I gather more information through this article, so please keep me informed of any update in this regard. For more relevant articles click on: how to recover deleted datacustom filedisk image softwareretrieve deleted files windowsdiskdiggerdiskdigger prousb drivedownload diskdigger
This is a great article. Loved your efforts on it buddy. Thanks for sharing this with us. Vmware training.