Kairosdb vs opentsdb how to query

Monitoring with Netdata

It doesn't happen every day that a "C-Level Executive" programs free software himself, but Costa Tsaousis, the COO of the payment provider VivaWallet, obviously finds the time. Under the umbrella of "FireHOL", he has not only worked on the firewall frontend of the same name for more than ten years, but also on the traffic shaper FireQoS and the monitoring package Netdata, which this article is about.

Netdata [1] offers performance and health monitoring, whereby it attaches particular importance to the high temporal resolution of the monitoring with one event per second. This goes hand in hand with an architectural specialty, namely the fact that Netdata keeps all data in memory by default and does not write continuously to the hard disk. Netdata only saves the current data when it is closed so that it does not start from scratch when restarting. In any case, that was the original idea, which is still the main focus of Netdata, even if the possibility of saving the history data has been available since last year.

Real-time graphs

With regard to real-time performance monitoring, Netdata is comparable to the familiar command line tools such as top, sysstat and so on, except that Netdata presents its data in an appealing way in real time in the browser. At the same time, without further configuration, it offers too many rather than too few metrics such as various CPU indicators, memory consumption, network utilization, hardware monitoring and much more.

Last year, the program author asked users on GitHub which features they would like to have implemented, and the most frequent answer was alerting in the event of an error, a classic monitoring feature such as is known from Nagios. And so Netdata now also offers that. In addition, Netdata can be integrated into numerous other systems. There are backends for the time series databases Graphite, OpenTSDB, KairosDB, InfluxDB, ElasticSearch, Prometheus and Blueflood. Netdata supports many monitorable software components via plug-ins. Netdata itself is written in the C programming language, but the plug-ins can be written in Python or JavaScript (Node.js). An impressive interactive graphic, which refers to further documentation, shows the diverse connection options that Netdata offers (Fig. 1).

Image 1: The complete ecosystem in which Netdata operates. This graphic is interactive on the web and the individual components are linked to other help pages.

Convenient installation

The latest version 1.10.0 was released at the end of March, but the older version 1.9 can often be found in the package repositories of Linux distributions. Packages for Arch Linux, Alpine, Debian, Gentoo, OpenSUSE, Solus and Ubuntu (18.04) are available. There is a third-party repository for Ubuntu 16.04 users, but we had problems with the package - some Python-based plug-ins didn't work like this. An installation via Docker is also possible, but using the full range of functions is difficult, and the configuration is more complicated than with a "native" installed Netdata.

In contrast, Netdata can be installed very easily with an install script on all possible Linux distributions, FreeBSD and special distributions derived therefrom such as pfSense and FreeNAS as well as macOS. The following commands download and run the installer script:

$ wget https://my-netdata.io/kickstart.sh$ chmod + x kickstart.sh $ ./kickstart.sh

You don't need to call the script with sudo, it asks for the password itself if necessary. The installation works perfectly, shows the paths used and even creates a script for removing Netdata (Illus. 2). During the installation, the script fetches the necessary third-party packages from the distribution repository, checks out the Netdata code from GitHub (in "/usr/src/netdata.git"), compiles it and then distributes the files the hard drive. In addition, the installer configures a startup script for the init system on the system, be it Systemd, Init-V or OpenRC.

Fig. 2: The Netdata installation script outputs all important paths and creates a script for the deinstallation.

The original idea was that Netdata would run on every system to be monitored. There is no distributed architecture with servers and agents at Netdata. Instead, the web dashboard in the menu saves every Netdata instance that the admin can select for monitoring in a "registry". In the meantime, this concept has been softened a bit because it is not so well suited, for example, to dynamic environments with emerging and disappearing VMs and containers. It is now possible to use individual Netdata instances only to collect data and to send them to a central Netdata server.

Listing 1: HAProxy authentication

frontend https-in use_backend netdata if {hdr_end (host) -i netdata.mydomain.com}… userlist netdata_users user ofrommel insecure-password T0Ps3Cret… backend netdata acl AuthOkNetdata http_auth (netdata_users) http-request auth realm NetdataRealm 127! .0.1: 19999
comments powered by Disqus