Netdata a tool for monitoring servers and datacenters.

We know better than anyone how difficult it can be to follow the trend, the state of health, the availability of services and servers when they grow out of all proportion.

You always start with a handful of servers, you end up managing thousands of them, in multiple datacenters, in multiple locations and regions around the world, on different suppliers and technologies.

Often the nightmare and challenge of every self-respecting system administrator, devops, sysadmin is to have everything under control, manage and tame the services and not be overwhelmed by them.

Over time, various metrics and data monitoring and collection systems have been devised, among which we mention for professional and informative purposes ZABBIX and NAGIOS, the two most popular solutions on the market.

However, both solutions have many problems if you are looking for bundled all in one solutions that allow you to do their job very well quickly, quickly and in which the commissioning and installation and configuration on each system takes 30 seconds / 1 minute at the most.

If you are looking for a highly professional solution capable of collecting the data of the entire fleet on a single screen, the solution we would like to recommend is definitely Netdata.

What is Netdata?

Network data is a highly optimized Linux utility that provides real-time (per second) performance monitoring for Linux systems , FreeBSD, applications, SNMP devices, etc. and shows complete interactive graphs that render absolutely all the values ​​collected on the web browser to analyze them.

Netdata helps system administrators, SREs, DevOps engineers and IT professionals collect all possible metrics from systems and applications, visualize these metrics in real time, and solve complex performance problems.

Netdata's solution uses two components, NetdataAgent e Netdata Cloud, to provide real-time performance and health monitoring for both individual nodes and the entire infrastructure.

It was developed to be installed on any Linux system, without interrupting the current applications running on it. You can use this tool to monitor and get an overview of what is happening in real time, and what just happened, on your Linux systems and applications.

This is what it monitors:

1. Total and per-core CPU utilization, interruptions, softirq and frequency.
2. Total usage of memory, RAM, swap and kernel.
3. Disk I / O (per disk: bandwidth, operations, backlog, usage, etc.).
4. Monitor network interfaces including: bandwidth, packets, errors, drops, etc.).
5. Monitor connections, events, errors, etc. Linux Netfilter / iptables firewall.
6. Processes (running, frozen, forking, active, etc.).
7. Process tree system applications (CPU, memory, swap, disk reads / writes, threads, etc.).
8. Apache and Nginx status monitoring with mod_status.
9. MySQL database monitoring: queries, updates, freezes, problems, threads, etc.
10. Postfix mail server message queue.
11. Squid proxy server bandwidth and request monitoring.
12. Hardware sensors (temperature, voltage, fans, power, humidity, etc.).
13. SNMP devices.

netdata

Netdata's Distributed Monitoring Agent collects thousands of metrics from systems, hardware and applications without any configuration. It works permanently on all your physical / virtual servers, containers, cloud deployments and edge / IoT devices.

You can install Netdata on most Linux distributions (Ubuntu, Debian, CentOS and others), container / microservice platforms (Kubernetes cluster, Docker) and many other operating systems (FreeBSD, macOS).

Netdata Cloud

Netdata Cloud is a web application that gives you real-time visibility for the entire infrastructure. With Netdata Cloud, you can view key metrics, in-depth graphs and active alarms from all your nodes in a single web interface. When an anomaly occurs, seamlessly log into any node to troubleshoot and find the root cause with Netdata's familiar dashboard.

Netdata Cloud is free ! You can add an entire infrastructure of nodes, invite all of your colleagues, and view any number of metrics, graphs, and alerts - all for free.

While Netdata Cloud offers a centralized method to monitor your agents, the metrics data is not stored or centralized in any way. The metric data remains with your nodes and is streamed to your browser, via the Cloud, only when you view the Netdata Cloud interface.

What can you do with Netdata Cloud?

Netdata is designed to be simple to use and flexible for every monitoring, viewing and troubleshooting use case:

• Collect : Netdata collects all available metrics from your system and applications with over 300 collectors, Kubernetes service detection and deep container monitoring, all using just 1% CPU and a few MB of RAM. It also collects metrics from Windows machines.
• Immagine - The dashboard features significant graphs to help you understand the relationships between hardware, operating system, running apps / services, and the rest of your infrastructure. Add nodes to Netdata Cloud for a complete view of your infrastructure from a single pane of glass.
• Monitor : Netdata's health watchdog uses hundreds of pre-configured alarms to alert you via Slack, email, PagerDuty and more when an anomaly occurs. Customize with dynamic thresholds, hysteresis, alarm patterns and role-based notifications.
• Troubleshooting - 1s granularity allows you to detect and analyze anomalies that other monitoring platforms may not have detected. Interactive visualizations reduce your dependency on the console and historical metrics help you trace the root cause of the problems.
• Store : Netdata's efficient database engine efficiently stores metrics per second for days, weeks or even months. Each distributed node stores metrics locally, simplifying implementation, reducing costs and enriching Netdata's interactive dashboards.
• Export - Integrate metrics per second with other time series databases like Graphite, Prometheus, InfluxDB, TimescaleDB and others with Netdata's interoperable and extensible core.
• Stream - Aggregates metrics from any number of distributed nodes into one location for in-depth analysis, including temporary nodes in a Kubernetes cluster.

Netdata takes a different approach to helping people build extraordinary infrastructure. It was created out of frustration with existing monitoring tools which are too complex, too expensive and they don't help their users solve complex performance and health problems.

Netdata is:

Simple to deploy

• Distribution on one line for Linux distributions, as well as support for Kubernetes / Docker infrastructures.
• No configuration and maintenance request to collect thousands of parameters, every second, from the underlying operating system and running applications.
• Predefined graphics and alarms report common anomalies and performance problems without manual configuration.
• Distributed storage to simplify the cost and complexity of archiving metric data from any number of nodes.

Powerful and scalable

• 1% CPU utilization, a few MB of RAM, and minimal disk I / O to run the monitoring agent on bare metal, virtual machines, containers and even IoT devices.
• Granularity per second for an unlimited number of metrics based on the hardware and applications running on the nodes.
• The interoperable exporters allow you to link Netdata's metrics per second with an existing monitoring stack and other time series databases.

Optimized for troubleshooting

• Detection of visual anomalies with UI / UX emphasizing relationships between graphs.
• Customizable dashboards to identify related metrics, respond to incidents, and help you streamline workflows.
• Metrics distributed in a centralized interface to help users or teams trace complex problems across distributed nodes.

Comparison with other solutions

Netdata offers many advantages over the existing monitoring landscape, be it expensive SaaS products or other open source tools.

Prices and versions

Starting March 13, 2024, NetData takes on a new business strategy for businesses that require the connection of numerous nodes. With the introduction of three distinct pricing plans, NetData aims to comprehensively respond to the needs of different user segments.

Monitoring and troubleshooting with Netdata remains free, forever, for everyone. By carefully analyzing the feature list, you can observe that every single tracking feature is available for free to all users. This is possible thanks to the design of the Netdata solution, where all monitoring capabilities, including artificial intelligence (ML), are implemented within the Netdata Agent, which is open source and installed on-premise.

Netdata's policy is to not charge for monitoring itself, but rather for better integration of their monitoring solution with users' business processes. The development of this integration proceeds hand in hand with that of the monitoring solution itself. Netdata is convinced that users will find their monitoring method not only easier and faster, but also more efficient, modern and attractive. They therefore expect that as the user team grows, there will be a desire to integrate the Netdata solution more closely with existing business processes.

In this regard, NetData has developed three different plans hoping to satisfy all types of users.

Homelab

• Price:
  • $90 if billed annually
  • $10 if billed monthly
• Features:
  • Use by home lab users or students for non-commercial purposes.
  • Unlimited number of connected nodes or custom dashboards.
  • All the features of a complete business level.
  • Fixed price and governed by a fair usage policy.

Business (Recommended)

• Price:
  • $3 per node per month, when billed annually.
  • $4 per node per month, when billed monthly.
• Features:
  • Inclusion of all community features.
  • Nodes connected without limits.
  • Unlimited custom dashboards.
  • Event and topology history up to 90 days.
  • Notifications of critical events during troubleshooting operations.
  • Ability to unlock all user roles to maximize team efficiency.
  • Integration with advanced notification systems like Slack, PagerDuty, Opsgenie and others.

Enterprise On-Premise

• Price:
  • $3 per node per month, when billed annually.
  • Additional cost of $900 per month for each installation.
• Features:
  • Netdata Cloud and all its components hosted entirely on its own infrastructures.
  • Ideal solution for monitoring critical infrastructures and isolated systems.
  • Complete customization to meet your organization's specific needs.

For Business and Enterprise On-Premise options you can contact for volume discounts or specific premium support.

Although the HomeLab price is absolutely adequate and in line with the spending limits for a Hobbyist, the Business and On-Premise plan can be very expensive especially with pricing policies per single node. For a hosting provider like ours, for example, 4 euros of monthly expenditure for monitoring a single node would entail around 10% of the costs on a dedicated server and around 20% of the costs on a medium-sized Cloud VPS. In short, on a fleet of 1000 cars, we would have a cost of 4000 dollars + VAT, a non-trivial cost if we consider Italian costs and pricing policies that are extremely different from those in the United States or in any case outside Europe.

In this regard, we have already examined within 2 months the complete switch to Prometheus and Grafana in order to obtain a completely self-hosted environment at extremely lower costs with an assumed saving of around 95% compared to a commercial solution like NetData which still remains certainly a valid solution.

Conclusions

We have just introduced and examined a very fast and powerful system for obtaining a lot of metrics and displaying them on a fast, performing and captivating dashboard. The validity of the tool that is absolutely adequate for almost all the needs of those who work in the server sector and above all web servers that offer web services is immediately understood.

We as a Hosting and systems engineering company found it faster, more accurate, and easier to install than the previous Zabbix which today is probably the most complete system if you want to invest time (waste time) to install and configure it.

To give an idea of ​​the commissioning of a Zabbix system, let's say that only the installation of the master node requires an hour by an expert systems engineer and each additional machine at least 15 minutes if you go very fast and you have mastery as well as some degree of manual automation.

The implementation of Netdata and the Netdata Cloud visualization requires an average of 1 hour of work for every 50 machines installed. Obviously, if you work with multiple terminals, in a heterogeneous environment the time can be drastically reduced.