Data Collection

Getting Data From Nginx Into Elasticsearch Using Fluentd

Looking to get data out of nginx into elasticsearch? You can do that with fluentd in 10 minutes!

![](/images/plugin_icon/nginx.png)

→

![](/images/plugin_icon/elasticsearch.png)

Here is how:

$ gem install fluentd
$ gem install fluent-plugin-elasticsearch
$ touch fluentd.conf

fluentd.conf should look like this (just copy and paste this into fluentd.conf):

<source>
  type tail
  path /var/log/httpd-access.log #...or where you placed your Apache access log
  pos_file /var/log/td-agent/httpd-access.log.pos # This is where you record file position
  tag nginx.access #fluentd tag!
  format nginx # Do you have a custom format? You can write your own regex.
</source>

<match **>
  type elasticsearch
  logstash_format true
  host <hostname> #(optional; default="localhost")
  port <port> #(optional; default=9200)
  index_name <index name> #(optional; default=fluentd)
  type_name <type name> #(optional; default=fluentd)
</match>

After that, you can start fluentd and everything should work:

$ fluentd -c fluentd.conf

Of course, this is just a quick example. If you are thinking of running fluentd in production, consider using td-agent, the enterprise version of Fluentd packaged and maintained by Treasure Data, Inc..