Effortless monitoring with collectd, Graphite and Docker

At dotCloud, we are very happy users of collectd. collectd is a metrics collection daemon, an alternative (much superior, in my opinion) to Munin that you might know. At dotCloud we use collectd to collect system metrics, but it could also be used for business or application oriented metrics quite easily.

collectd is basically a big loop, which read (i.e: collect) and/or write (in a storage backend like RRDs, on the network, etc.) metrics at a configurable interval. collectd is very efficient at that, and since it’s modular, a collectd daemon can be configured to collect metrics, store them or simply route them to other collectd servers using its efficient binary protocol; all of that from a large list of plugins.

However, collectd doesn’t do any graphing at all and you have to use something on top of the storage backend you picked up. So, collectd is fairly easy to set up but getting graphs —the coolest part— ends up being a pain each time. But, there is another project called Graphite, which is good at storing and displaying time-series metrics but cannot collect them. That makes it a great combo with collectd.

Graphite is still a pain to install compared to collectd, but at least you only need to set it up once (whereas you want collectd on each host/container where you collect metrics), but thanks to this Dockerfile, this is a solved issue! And in the process, we added UDP support in Docker since UDP is used by collectd’s network protocol.

So, two things:

  • You can now open/redirect UDP ports in Docker containers, e.g: 25826/udp;
  • You can now deploy a collectd/Graphite receiver/graphing frontend easily.

If you have Docker (from master, UDP support hasn’t been released yet) installed, you can try that out with:

docker pull lopter/collectd-graphite
docker run lopter/collectd-graphite

Then, using docker ps, write down which port has been assigned to collectd and which one has been assigned to the web interface:

ports

The other ports are for SSH and for Graphite (in case you want to send data to Graphite directly instead of going through collectd).

Install collectd using your favorite package manager, open collectd.conf and add:


    Server "" ""

Restart collectd and point your browser to http://<your-docker-host>:<the-other-port-in-docker-ps>, and you should see this:

graphite_empty

By navigating in the tree on the left, you can start to build your own graphs:

graphite_load_avg

If you want to save configured graphs, you can login into graphite with the username “graphite” and the password “admin”.

I’m very excited about this, because I always wanted to use collectd more, even during development/test phases (collectd can collect metrics down to a one second resolution which makes it interesting for that), but I never did it because of the high cost of setting up the graphing. With Docker I can deploy my own, disposable, collectd/graphite receiver in a few secs and get some insightful infos right away.

The sources for this Docker image are on GitHub, in it you’ll find the Dockerfile, but also the configuration files used for collectd and Graphite, which you might want to tune!

About Louis OpterJerome

Louis Opter is a Platform engineer at dotCloud. He is working with us since day 1 in 2009. He’s passionate about systems programming and specialized in Python. He likes to code while listening to music and is a Vietnamese martial arts enthusiast (Tay Son Vo Dao).

Connect with Louis on Twitter! @1opter

 

5 Responses to “Effortless monitoring with collectd, Graphite and Docker”

  1. Geert-Jan Brits

    Interesting. I’m just reading up on Docker and it looks great.

    I’ve got a question for the opposite case: using collectd to collect info from a process running in a Docker Daemon (since I’m reading a Daemon is the way forward from now on)

    Say I’m running Elasticsearch (without Docker) and I want to collect some metrics using CollectD. I’d use the collectd elasticsearch plugin: https://github.com/phobos182/collectd-elasticsearch or roll my own.

    However, how does this work when Elasticsearch were to run in a Docker Daemon? Are all metrics still exposed somehow? Can the original plugin still be used (by hooking into some sort of feed, as if Elasticsearch wasn’t running in a Daemon, just thinking out loud here) or does it have to be rewritten entirely?

    Best,
    Geert-Jan

    Reply
    • Louis Opter

      You can perfectly run collectd side-by-side with elasticsearch in the same container. So, yes you can use an unmodified version of collectd to monitor elasticsearch running in a Docker container.

      Reply
      • Geert-Jan Brits

        Just trying to wrap my head around the general flow. So just to confirm, say that I have multiple dockers on the same box. I could run collectd in each of them and push data to the outside world (i.e: outside of the docker instance) to aggregate the data in Graphite and/or Nagios for example?

        Reply
  2. J. Javier Maestro

    Hi there!
    In case you find it useful, I recently added UDP support to the graphite plugin in collectd:
    https://github.com/collectd/collectd/commit/514a9990b21a3450f815f346ea6a191531934265

    It’s merged so I guess it will make it to the next release but it’s probably not available right now through packages. If you want it in the Docker, I guess you’ll have to modify the Dockerfile to do a build :-?

    Anyway, hope you find it useful! :)

    Reply
    • Louis Opter

      Neat! Indeed, to get advantage of that the Dockerfile will need modifications. Right now, it’s simply using the package shipped with Ubuntu 13.04.

      Reply

Leave a Reply