Linux Blimp

There are all kinds of Linux system monitoring tools such as top, htop, atop and many more that provide different output of system data such as resource utilization, running processes, CPU temperature and others.

In this article, we are going to review a modular monitoring tool called Hegemon. It’s an open source project written in Rust, which works are still in progress.

Hegemon includes the following features:

• Monitor CPU, memory and swap usage
• Monitor system temperatures and fan speeds
• Adjustable update interval
• Unit tests
• Expand data stream for more detailed graphic visualization

How to Install Hegemon in Linux

Hegemon is currently available for Linux only and requires Rust and the development files for libsensors. The latter can be found in the default package repository and can be installed using the following commands.

# yum install lm_sensors-devel   [On CentOS/RHEL] 
# dnf install lm_sensors-devel   [On Fedora 22+]
# apt install libsensors4-dev    [On Debian/Ubuntu]

Detailed instructions how to install Rust programming language on your system are provided in the following article.

1. How to Install Rust Programming Language in Linux

Once you have install Rust, you can proceed with installing Hegemon by using Rust’s package manager called cargo.

# cargo install hegemon

When the installation is complete run hegemon, by simply issuing the following command.

# hegemon

The hegemon graph will appear. You will have to give it a few seconds to collect data and update its information.

You will see the following sections:

• CPU – Shows the CPU utilization
• Core Num – Utilization of the CPU core
• Mem – memory utilization
• Swap – swap memory usage

You can expand each section by pressing “Space” button on your keyboard. This will provide a little more detailed information about the utilization of the resource you have selected.

If you wish to increase or decrease the update interval, you can use the + and - buttons on your keyboard.

How to Add New Streams

Hegemon uses data streams to visualize its data. Their behavior is defined in the stream trait here. Streams only need to provide basic data such as name, description and a method for retrieving numeric data value.

Hegemon will manage the rest – updating the information, rendering layout and computation stats. To learn more how to create data streams and learn how to create your own, you would need to dive deeper into the Hegemon project on git. A good starting point would be the project readme file.

Conclusion

Hegemon is a simple, easy to use tool to help you collect quick stats about your system status. While it’s functionality is rather basic compared to other monitoring tools, it does its job very well and is a reliable source for collecting system information. Future releases are expected to have network monitoring support, which may come quite handy.

Source

Monitorix is an open source, free and most powerful lightweight tool designed to monitor system and network resources in Linux. It regularly collects system and network data and display the information in graphs using its own web interface. Monitorix allows to monitor overall system performance and also help in detecting bottlenecks, failures, unwanted long response times and other abnormal activities.

It is written in Perl language and licensed under the terms of GNU (General Public License) as published by the FSP (Free Software Foundation). It uses RRDtool to generate graphs and display them using web interface.

This tool is specifically created for monitoring Red Hat, CentOS, Fedora based Linux systems, but today it runs on many different flavors of GNU/Linux distributions and even it runs on UNIX systems like OpenBSD, NetBSDand FreeBSD.

The development of Monitorix is currently in active state and adding new features, new graphs, new updates and fixing bugs to offer a great tool for Linux system/network administration.

Monitorix Features

1. System load average, active processes, per-processor kernel usage, global kernel usage and memory allocation.
2. Monitors Disk drive temperatures and health.
3. Filesystem usage and I/O activity of filesystems.
4. Network traffic usage up to 10 network devices.
5. System services including SSH, FTP, Vsftpd, ProFTP, SMTP, POP3, IMAP, POP3, VirusMail and Spam.
6. MTA Mail statistics including input and output connections.
7. Network port traffic including TCP, UDP, etc.
8. FTP statistics with log file formats of FTP servers.
9. Apache statistics of local or remote servers.
10. MySQL statistics of local or remote servers.
11. Squid Proxy Web Cache statistics.
12. Fail2ban statistics.
13. Monitor remote servers (Multihost).
14. Ability to view statistics in graphs or in plain text tables per day, week, month or year.
15. Ability to zoom graphs for better view.
16. Ability to define the number of graphs per row.
17. Built-in HTTP server.

For a full list of new features and updates, please check out the official feature page.

Installing Monitorix on a RHEL/CentOS/Fedora Linux

First, install following required packages.

# yum install rrdtool rrdtool-perl perl-libwww-perl perl-MailTools perl-MIME-Lite perl-CGI perl-DBI perl-XML-Simple perl-Config-General perl-HTTP-Server-Simple perl-IO-Socket-SSL wget

If in case yum fails to installing one or more of above packages, then you could enable following additional repositories to install them.

1. Enable EPEL repository
2. Enable RPMforge repository

Next, download the latest version of ‘Monitorix‘ package using wget command.

# wget http://www.monitorix.org/monitorix-3.10.1-1.noarch.rpm

Once successfully downloaded, install it using the rpm command.

# rpm -ivh monitorix-3.10.1-1.noarch.rpm

Preparing...                ########################################### [100%]
   1:monitorix              ########################################### [100%]

Once successfully installed, please have a look at the main configuration file ‘/etc/monitorix.conf‘ to add some extra settings according to your system and enable or disable graphs.

Finally, add Monitorix service to system start-up and start the service with following commands.

# chkconfig --level 35 monitorix on
# service monitorix start        
# systemctl start monitorix       [On RHEL/CentOS 7 and Fedora 22+ versions ]

Once, you’ve started service, the program will start collecting system information according to configuration set in ‘/etc/monitorix.conf‘ file, and after few minutes you will start seeing system graphs from your browser at.

http://localhost:8080/monitorix/

If you have SELinux in enabled state, then graphs are not visible and you will get tons of error messages in ‘/var/log/messages‘ or ‘/var/log/audit/audit.log‘ file about access denied to RRD database files. To get rid of such errors messages and visible graphs, you need to disable SELinux.

To Turn Off SELinux, simple changing the line “enforcing” to “disabled” in ‘/etc/selinux/config’ file.

SELINUX=disabled

The above will disable SELinux temporarily, until you reboot the machine. If you want the system to start in always disable mode, you need to reboot the system.

Installing Monitorix on a Ubuntu/Debian/Linux Mint

The Monitorix installation can be done in two-ways, using Izzy repository for automatic installation/updates and another using manually download and install .deb package.

The Izzy repository is an experimental repository but the packages from this repository should work on all versions of Ubuntu, Debian, etc. However, no warranties are given – So, the risk is all yours. If you still want to add this repository for automatic updates via apt-get, simply follow the steps provided below for automatic installation.

Automatic Installation Using Izzy Repository

Add the following line to your ‘/etc/apt/sources.list’ file.

deb http://apt.izzysoft.de/ubuntu generic universe

Get GPG key for this repository, you can get it using wget command.

# wget http://apt.izzysoft.de/izzysoft.asc

Once downloaded, add this GPG key to apt configuration by using the command ‘apt-key‘ as shown below.

# apt-key add izzysoft.asc

Finally, install the package via the repository.

# apt-get update
# apt-get install monitorix

Manual Installation Using .Deb Package

Manually, downloading latest version of .deb package and install it with taking care of required dependencies as shown below.

# apt-get update
# apt-get install rrdtool perl libwww-perl libmailtools-perl libmime-lite-perl librrds-perl libdbi-perl libxml-simple-perl libhttp-server-simple-perl libconfig-general-perl libio-socket-ssl-perl
# wget http://www.monitorix.org/monitorix_3.10.1-izzy1_all.deb
# dpkg -i monitorix_3.10.1-izzy1_all.deb

During installation, a web server configuration takes place. So, you need to reload the Apache web server to reflect new configuration.

# service apache2 restart         [On SysVinit]
# systemctl restart apache2       [On SystemD]

Monitorix comes with a default configuration, if you wish to change or adjust some settings take a look at the configuration file at ‘/etc/monitorix.conf‘. Once you’ve done changes reload the service for new configuration to take effect.

# service monitorix restart         [On SysVinit]
# systemctl restart monitorix       [On SystemD]

Now point your browser to ‘http://localhost:8080/monitorix‘ and start watching graphs of your system. It can be accessed from localhost only, if you wish to allow access to remote IP’s. Simply open the ‘/etc/apache2/conf.d/monitorix.conf‘ file and add IP’s to the ‘Allow from‘ clause. For example see below.

<Directory /usr/share/monitorix/cgi-bin/>
        DirectoryIndex monitorix.cgi
        Options ExecCGI
        Order Deny,Allow
        Deny from all
        Allow from 172.16.16.25
</Directory>

After you made changes to above configuration, do not forget to restart Apache.

# service apache2 restart         [On SysVinit]
# systemctl restart apache2       [On SystemD]

Monitorix Screenshots

Please check out the following are some screenshots.

Monitorix Homepage

Monitor Linux Load Average

Monitor Linux Kernel Usage

Monitor Linux Kernel Processor

Monitor Linux Disk Health

Monitor Linux Filesystem and Disk I/O Read

Monitor Linux Network Traffic

Monitor Linux System Services

Monitor Linux Network Port Traffic

Monitor Linux Apache Statistics

Monitor MySQL/MariaDB Statistics

Reference Links:

1. Monitorix Homepage
2. Monitorix Documentation

Source

Some of the popular and frequently used system resource generating tools available on the Linux platform include vmstat, netstat, iostat, ifstat and mpstat. They are used for reporting statistics from different system components such as virtual memory, network connections and interfaces, CPU, input/output devices and more.

As a system administrator, you may be looking for that one tool that can give your a good amount of the information provided by above tools, even more, a single and powerful tool that has additional features and capabilities, then look no further than dstat.

Suggested Read: 20 Command Line Tools to Monitor Linux Performance

dstat is a powerful, flexible and versatile tool for generating Linux system resource statistics, that is a replacement for all the tools mentioned above. It comes with extra features, counters and it is highly extensible, users with Python knowledge can build their own plugins.

Features of dstat:

1. Joins information from vmstat, netstat, iostat, ifstat and mpstat tools
2. Displays statistics simultaneously
3. Orders counters and highly-extensible
4. Supports summarizing of grouped block/network devices
5. Displays interrupts per device
6. Works on accurate timeframes, no timeshifts when a system is stressed
7. Supports colored output, it indicates different units in different colors
8. Shows exact units and limits conversion mistakes as much as possible
9. Supports exporting of CSV output to Gnumeric and Excel documents

How to Install dstat in Linux Systems

dstat is available to install from default repositories on most Linux distributions, you can install and use it for monitoring a Linux system in the process of performance tuning tests or troubleshooting exercises.

# yum install dstat             [On RedHat/CentOS and Fedora]
$ sudo apt-get install dstat    [On Debian, Ubuntu and Linux Mint]

It works in real-time, outputting selective information in columns, including the magnitude and units for stats displayed after every one second, by default.

Note: The dstat output is aimed specifically for human interpretation, not as input for other tools to process.

Below is an output seen after running the dstat command without any options and arguments (similar to using -cdngy (default) options or -a option).

$ dstat 

The output above indicates:

1. CPU stats: cpu usage by a user (usr) processes, system (sys) processes, as well as the number of idle (idl) and waiting (wai) processes, hard interrupt (hiq) and soft interrupt (siq).
2. Disk stats: total number of read (read) and write (writ) operations on disks.
3. Network stats: total amount of bytes received (recv) and sent (send) on network interfaces.
4. Paging stats: number of times information is copied into (in) and moved out (out) of memory.
5. System stats: number of interrupts (int) and context switches (csw).

To display information provided by vmstat, use the -v or --vmstat option:

$ dstat --vmstat

In the image above, dstat displays:

1. Process stats: number of running (run), blocked (blk) and new (new) spawned processes.
2. Memory stats: amount of used (used), buffered (buff), cached (cach) and free (free) memory.

I already explained at the last three sections (paging, disk and system stats) in the previous example.

Suggested Read: Sysstat – All-in-One System Performance and Usage Activity Monitoring Tool For Linux

Let us dive into some advanced dstat system monitoring commands. In the next example, we want to monitor a single program that is using the most CPU and consuming the most amount of memory.

The options in the command are:

1. -c – cpu usage
2. --top-cpu – process using most CPU
3. -dn – disk and network stats
4. --top-mem – process consuming the most memory

$ dstat -c --top-cpu -dn --top-mem

Additionally, you can also store the output of dstat in a .csv file for analysis at a latter time by enabling the --output option as in the example below.

0Here, we are displaying the time, cpu, mem, system load stats with a one second delay between 5 updates (counts).

$ dstat --time --cpu --mem --load --output report.csv 1 5 

There are several internal (such as options used in previous example) and external dstat plugins you can use with dstat, to view a list of all available plugins, run the command below:

$ dstat --list

It reads plugins from the paths below, therefore, add external plugins in these directories:

~/.dstat/
(path of binary)/plugins/
/usr/share/dstat/
/usr/local/share/dstat/

For more usage information, look through the dstat man page or visit the homepage at: http://dag.wiee.rs/home-made/dstat/.

Suggested Read: Collectl: An Advanced All-in-One Performance Monitoring Tool for Linux

dstat is a versatile, all-in-one system resources statistics generating tool, it combines information from several other tools such as vmstat, mpstat, iostat, netstat and ifstat.

I hope this review will be helpful to you, most importantly, you can share with us any suggestions, supplementary ideas to improve the article and also give us feedback about your experience using of dstatthrough the comment section below.

 
Source

Osquery is a free open source, powerful and cross-platform SQL-based operating system instrumentation, monitoring, and analytics framework for Linux, FreeBSD, Windows, and Mac/OS X systems, built by Facebook. It is a simple and easy-to-use operating system explorer.

It combines a number of tools which perform low-level OS analytics and monitoring; these tools reveal an operating system as a high-performance relational database such as MySQL/MariaDB, PostgreSQL and more, where OS concepts are represented in tabular form, thus allowing users to employ SQL commands to carry out system monitoring and analytics.

Osquery use a simple plugin and extensions API to implement SQL tables, there is a collection of tables in existence ready for use, and more are being written. Some tables can only be found on a specific operating system, for instance, you only find the kernel_modules table on Linux systems.

Additionally, you can run queries to monitor and analyze OS state on a single host via the osqueryi shell, or on several hosts on a network via a scheduler or execute them from any of your custom applications using osquery Thrift APIs.

How to Install Osquery in Linux

The Osquery can be installed from the official repository using apt yum or dnf package management tool on your respective Linux distribution as shown.

On Debian/Ubuntu

$ export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys $OSQUERY_KEY
$ sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main'
$ sudo apt update
$ sudo apt install osquery

On RHEL/CentOS

$ curl -L https://pkg.osquery.io/rpm/GPG | sudo tee /etc/pki/rpm-gpg/RPM-GPG-KEY-osquery
$ sudo yum-config-manager --add-repo https://pkg.osquery.io/rpm/osquery-s3-rpm.repo
$ sudo yum-config-manager --enable osquery-s3-rpm-repo
$ sudo yum install osquery

On Fedora 22+

$ curl -L https://pkg.osquery.io/rpm/GPG | sudo tee /etc/pki/rpm-gpg/RPM-GPG-KEY-osquery
$ dnf config-manager --add-repo --add-repo https://pkg.osquery.io/rpm/osquery-s3-rpm.repo
$ sudo dnf config-manager --set-enabled osquery-s3-rpm
$ sudo dnf install osquery

How to Monitor and Analyze Linux Using Osquery

Once you have successfully installed Osquery on your system, launch the osqueryi shell to start querying the state of your OS as shown.

$ osqueryi

Using a virtual database. Need help, type '.help'
osquery> 

To get a summarized Linux system information run the following command.

osquery> SELECT  * FROM system_info;

To get a well formated list of all users on the Linux system, run the following query.

osquery> SELECT * FROM users;

To get a list of all Linux kernel modules and their status, run the following query.

osquery> SELECT * FROM kernel_modules;

To get a list of all installed RPM packages on CentOS, RHEL and Fedora, run the following query.

osquery> .all rpm_packages;

To get a informatin about running Linux processes, run the following query.

osquery> SELECT DISTINCT processes.name, listening_ports.port, processes.pid FROM listening_ports JOIN processes USING (pid) WHERE listening_ports.address = '0.0.0.0';

If you are running osquery on a desktop and have Firefox or Chrome installed, you can list all your add-ons using the following query.

osquery> .all firefox_addons;
osquery> .all  chrome_extensions;

To display a list of all implemented tables in Linux, use the .tables command as shown.

osquery> .tables;	#list all implemented tables
osquery> .help; 	#view help message

Osquery also provides file integrity monitoring (FIM), and process and socket auditing features and more, thus it is an intrusion detection tool, but this calls for certain configurations before you can deploy it for such a purpose. You can find more information from the Osquery Github repository.

Source

BCC (BPF Compiler Collection) is a powerful set of appropriate tools and example files for creating resourceful kernel tracing and manipulation programs. It utilizes extended BPF (Berkeley Packet Filters), initially known as eBPF which was one of the new features in Linux 3.15.

Practically, most of the components used by BCC require Linux 4.1 or above, and its noteworthy features include:

1. Requires no 3rd party kernel module, since all the tools work based on BPF which is built into the kernel and BCC uses features added in Linux 4.x series.
2. Enables observation of software execution.
3. Comprises of several performance analysis tools with example files and man pages.

Suggested Read: 20 Command Line Tools to Monitor Linux Performance

Best suited for advanced Linux users, BCC makes it easy to write BPF programs using kernel instrumentation in C, and front-ends in Python and lua. Additionally, it supports multiple tasks such as performance analysis, monitoring, network traffic control plus lots more.

How To Install BCC in Linux Systems

Remember that BCC uses features added in Linux kernel version 4.1 or above, and as a requirement, the kernel should have been compiled with the flags set below:

CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
# [optional, for tc filters]
CONFIG_NET_CLS_BPF=m
# [optional, for tc actions]
CONFIG_NET_ACT_BPF=m
CONFIG_BPF_JIT=y
CONFIG_HAVE_BPF_JIT=y
# [optional, for kprobes]
CONFIG_BPF_EVENTS=y

To check your kernel flags, view the file /proc/config.gz or run the commands as in the examples below:

tecmint@TecMint ~ $ grep CONFIG_BPF= /boot/config-`uname -r`
CONFIG_BPF=y
tecmint@TecMint ~ $ grep CONFIG_BPF_SYSCALL= /boot/config-`uname -r`
CONFIG_BPF_SYSCALL=y
tecmint@TecMint ~ $ grep CONFIG_NET_CLS_BPF= /boot/config-`uname -r`
CONFIG_NET_CLS_BPF=m
tecmint@TecMint ~ $ grep CONFIG_NET_ACT_BPF= /boot/config-`uname -r`
CONFIG_NET_ACT_BPF=m
tecmint@TecMint ~ $ grep CONFIG_BPF_JIT= /boot/config-`uname -r`
CONFIG_BPF_JIT=y
tecmint@TecMint ~ $ grep CONFIG_HAVE_BPF_JIT= /boot/config-`uname -r`
CONFIG_HAVE_BPF_JIT=y
tecmint@TecMint ~ $ grep CONFIG_BPF_EVENTS= /boot/config-`uname -r`
CONFIG_BPF_EVENTS=y

After verifying kernel flags, it’s time to install BCC tools in Linux systems.

On Ubuntu 16.04

Only the nightly packages are created for Ubuntu 16.04, but the installation instructions are very straightforward. No need of kernel upgrade or compile it from source.

$ echo "deb [trusted=yes] https://repo.iovisor.org/apt/xenial xenial-nightly main" | sudo tee /etc/apt/sources.list.d/iovisor.list
$ sudo apt-get update
$ sudo apt-get install bcc-tools

On Ubuntu 14.04

Begin by installing a 4.3+ Linux kernel, from http://kernel.ubuntu.com/~kernel-ppa/mainline.

As an example, write a small shell script “bcc-install.sh” with the content below.

Note: update PREFIX value to the latest date, and also browse the files in the PREFIX url provided to get the actual REL value, substitute them in the shell script.

#!/bin/bash
VER=4.5.1-040501
PREFIX=http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5.1-wily/
REL=201604121331
wget ${PREFIX}/linux-headers-${VER}-generic_${VER}.${REL}_amd64.deb
wget ${PREFIX}/linux-headers-${VER}_${VER}.${REL}_all.deb
wget ${PREFIX}/linux-image-${VER}-generic_${VER}.${REL}_amd64.deb
sudo dpkg -i linux-*${VER}.${REL}*.deb

Save the file and exit. Make it executable, then run it as shown:

$ chmod +x bcc-install.sh
$ sh bcc-install.sh

Afterwards, reboot your system.

$ reboot

Next, run the commands below to install signed BCC packages:

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys D4284CDD
$ echo "deb https://repo.iovisor.org/apt trusty main" | sudo tee /etc/apt/sources.list.d/iovisor.list
$ sudo apt-get update
$ sudo apt-get install binutils bcc bcc-tools libbcc-examples python-bcc

On Fedora 24-23

Install a 4.2+ kernel from http://alt.fedoraproject.org/pub/alt/rawhide-kernel-nodebug, if your system has a version lower than what is required. Below is an example of how to do that:

$ sudo dnf config-manager --add-repo=http://alt.fedoraproject.org/pub/alt/rawhide-kernel-nodebug/fedora-rawhide-kernel-nodebug.repo
$ sudo dnf update
$ reboot

After that, add the BBC tools repository, update your system and install the tools by executing the next series of commands:

$ echo -e '[iovisor]\nbaseurl=https://repo.iovisor.org/yum/nightly/f23/$basearch\nenabled=1\ngpgcheck=0' | sudo tee /etc/yum.repos.d/iovisor.repo
$ sudo dnf update
$ sudo dnf install bcc-tools

On Arch Linux – AUR

You should start by upgrading your kernel to at least version 4.3.1-1, subsequently install the packages below using any Arch package managers such as pacaur, yaourt, cower, etc.

bcc bcc-tools python-bcc python2-bcc

How To Use BCC Tools in Linux Systems

All the BCC tools are installed under /usr/share/bcc/tools directory. However, you can alternatively run them from the BCC Github repository under /tools where they end with a .py extension.

$ ls /usr/share/bcc/tools 

argdist       capable     filetop         offwaketime  stackcount  vfscount
bashreadline  cpudist     funccount       old          stacksnoop  vfsstat
biolatency    dcsnoop     funclatency     oomkill      statsnoop   wakeuptime
biosnoop      dcstat      gethostlatency  opensnoop    syncsnoop   xfsdist
biotop        doc         hardirqs        pidpersec    tcpaccept   xfsslower
bitesize      execsnoop   killsnoop       profile      tcpconnect  zfsdist
btrfsdist     ext4dist    mdflush         runqlat      tcpconnlat  zfsslower
btrfsslower   ext4slower  memleak         softirqs     tcpretrans
cachestat     filelife    mysqld_qslower  solisten     tplist
cachetop      fileslower  offcputime      sslsniff     trace

We shall cover a few examples under – monitoring general Linux system performance and networking.

Trace open() syscalls

Let’s start by tracing all open() syscalls using opensnoop. This enable us tell us how various applications work by identifying their data files, config files and many more:

$ cd /usr/share/bcc/tools 
$ sudo ./opensnoop

PID    COMM               FD ERR PATH
1      systemd            35   0 /proc/self/mountinfo
2797   udisksd            13   0 /proc/self/mountinfo
1      systemd            35   0 /sys/devices/pci0000:00/0000:00:0d.0/ata3/host2/target2:0:0/2:0:0:0/block/sda/sda1/uevent
1      systemd            35   0 /run/udev/data/b8:1
1      systemd            -1   2 /etc/systemd/system/sys-kernel-debug-tracing.mount
1      systemd            -1   2 /run/systemd/system/sys-kernel-debug-tracing.mount
1      systemd            -1   2 /run/systemd/generator/sys-kernel-debug-tracing.mount
1      systemd            -1   2 /usr/local/lib/systemd/system/sys-kernel-debug-tracing.mount
2247   systemd            15   0 /proc/self/mountinfo
1      systemd            -1   2 /lib/systemd/system/sys-kernel-debug-tracing.mount
1      systemd            -1   2 /usr/lib/systemd/system/sys-kernel-debug-tracing.mount
1      systemd            -1   2 /run/systemd/generator.late/sys-kernel-debug-tracing.mount
1      systemd            -1   2 /etc/systemd/system/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /etc/systemd/system/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /run/systemd/system/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /run/systemd/system/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /run/systemd/generator/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /run/systemd/generator/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /usr/local/lib/systemd/system/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /usr/local/lib/systemd/system/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /lib/systemd/system/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /lib/systemd/system/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /usr/lib/systemd/system/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /usr/lib/systemd/system/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /run/systemd/generator.late/sys-kernel-debug-tracing.mount.wants
1      systemd            -1   2 /run/systemd/generator.late/sys-kernel-debug-tracing.mount.requires
1      systemd            -1   2 /etc/systemd/system/sys-kernel-debug-tracing.mount.d
1      systemd            -1   2 /run/systemd/system/sys-kernel-debug-tracing.mount.d
1      systemd            -1   2 /run/systemd/generator/sys-kernel-debug-tracing.mount.d
....

Summarize Block Device I/O Latency

In this example, it shows a summarized distribution of disk I/O latency using biolatecncy. After executing the command, wait for a few minutes and hit Ctrl-C to end it and view the output.

$ sudo ./biolatecncy

Tracing block device I/O... Hit Ctrl-C to end.
^C
     usecs               : count     distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 3        |****************************************|
       256 -> 511        : 3        |****************************************|
       512 -> 1023       : 1        |*************                           |

Trace New Processes via exec() Syscalls

In this section, we shall move to tracing new processes in execution using execsnoop tool. Each time a process is forked by fork() and exec() syscalls, it is shown in the output. However, not all processes are captured.

$ sudo ./execsnoop

PCOMM            PID    PPID   RET ARGS
gnome-screensho  14882  14881    0 /usr/bin/gnome-screenshot --gapplication-service
systemd-hostnam  14892  1        0 /lib/systemd/systemd-hostnamed
nautilus         14897  2767    -2 /home/tecmint/bin/net usershare info
nautilus         14897  2767    -2 /home/tecmint/.local/bin/net usershare info
nautilus         14897  2767    -2 /usr/local/sbin/net usershare info
nautilus         14897  2767    -2 /usr/local/bin/net usershare info
nautilus         14897  2767    -2 /usr/sbin/net usershare info
nautilus         14897  2767    -2 /usr/bin/net usershare info
nautilus         14897  2767    -2 /sbin/net usershare info
nautilus         14897  2767    -2 /bin/net usershare info
nautilus         14897  2767    -2 /usr/games/net usershare info
nautilus         14897  2767    -2 /usr/local/games/net usershare info
nautilus         14897  2767    -2 /snap/bin/net usershare info
compiz           14899  14898   -2 /home/tecmint/bin/libreoffice --calc
compiz           14899  14898   -2 /home/tecmint/.local/bin/libreoffice --calc
compiz           14899  14898   -2 /usr/local/sbin/libreoffice --calc
compiz           14899  14898   -2 /usr/local/bin/libreoffice --calc
compiz           14899  14898   -2 /usr/sbin/libreoffice --calc
libreoffice      14899  2252     0 /usr/bin/libreoffice --calc
dirname          14902  14899    0 /usr/bin/dirname /usr/bin/libreoffice
basename         14903  14899    0 /usr/bin/basename /usr/bin/libreoffice
...

Trace Slow ext4 Operations

Using ext4slower to trace the ext4 file system common operations that are slower than 10ms, to help us identify independently slow disk I/O via the file system.

Suggested Read: 13 Linux Performance Monitoring Tools

It only outputs those operations that exceed a threshold:

$ sudo ./execslower

Tracing ext4 operations slower than 10 ms
TIME     COMM           PID    T BYTES   OFF_KB   LAT(ms) FILENAME
11:59:13 upstart        2252   W 48      1          10.76 dbus.log
11:59:13 gnome-screensh 14993  R 144     0          10.96 settings.ini
11:59:13 gnome-screensh 14993  R 28      0          16.02 gtk.css
11:59:13 gnome-screensh 14993  R 3389    0          18.32 gtk-main.css
11:59:25 rs:main Q:Reg  1826   W 156     60         31.85 syslog
11:59:25 pool           15002  R 208     0          14.98 .xsession-errors
11:59:25 pool           15002  R 644     0          12.28 .ICEauthority
11:59:25 pool           15002  R 220     0          13.38 .bash_logout
11:59:27 dconf-service  2599   S 0       0          22.75 user.BHDKOY
11:59:33 compiz         2548   R 4096    0          19.03 firefox.desktop
11:59:34 compiz         15008  R 128     0          27.52 firefox.sh
11:59:34 firefox        15008  R 128     0          36.48 firefox
11:59:34 zeitgeist-daem 2988   S 0       0          62.23 activity.sqlite-wal
11:59:34 zeitgeist-fts  2996   R 8192    40         15.67 postlist.DB
11:59:34 firefox        15008  R 140     0          18.05 dependentlibs.list
11:59:34 zeitgeist-fts  2996   S 0       0          25.96 position.tmp
11:59:34 firefox        15008  R 4096    0          10.67 libplc4.so
11:59:34 zeitgeist-fts  2996   S 0       0          11.29 termlist.tmp
...

Trace Block Device I/O with PID and Latency

Next off, let’s dive into printing a line per disk I/O each second, with details such as process ID, sector, bytes, latency among others using biosnoop:

$ sudo ./biosnoop

TIME(s)        COMM           PID    DISK    T  SECTOR    BYTES   LAT(ms)
0.000000000    ?              0              R  -1        8          0.26
2.047897000    ?              0              R  -1        8          0.21
3.280028000    kworker/u4:0   14871  sda     W  30552896  4096       0.24
3.280271000    jbd2/sda1-8    545    sda     W  29757720  12288      0.40
3.298318000    jbd2/sda1-8    545    sda     W  29757744  4096       0.14
4.096084000    ?              0              R  -1        8          0.27
6.143977000    ?              0              R  -1        8          0.27
8.192006000    ?              0              R  -1        8          0.26
8.303938000    kworker/u4:2   15084  sda     W  12586584  4096       0.14
8.303965000    kworker/u4:2   15084  sda     W  25174736  4096       0.14
10.239961000   ?              0              R  -1        8          0.26
12.292057000   ?              0              R  -1        8          0.20
14.335990000   ?              0              R  -1        8          0.26
16.383798000   ?              0              R  -1        8          0.17
...

Trace Page Cache hit/miss Ratio

Thereafter, we proceed to using cachestat to displays one line of summarized statistics from the system cache every second. This enables for system tuning operations by pointing out low cache hit ratio and high rate of misses:

$ sudo ./cachestat

 HITS   MISSES  DIRTIES  READ_HIT% WRITE_HIT%   BUFFERS_MB  CACHED_MB
       0        0        0       0.0%       0.0%           19        544
       4        4        2      25.0%      25.0%           19        544
    1321       33        4      97.3%       2.3%           19        545
    7476        0        2     100.0%       0.0%           19        545
    6228       15        2      99.7%       0.2%           19        545
       0        0        0       0.0%       0.0%           19        545
    7391      253      108      95.3%       2.7%           19        545
   33608     5382       28      86.1%      13.8%           19        567
   25098       37       36      99.7%       0.0%           19        566
   17624      239      416      96.3%       0.5%           19        520
...

Trace TCP Active Connections

Monitoring TCP connections every second using tcpconnect. Its output includes source and destination address, and port number. This tool is useful for tracing unexpected TCP connections, thereby helping us to identify inefficiencies in application configurations or an attacker.

$ sudo ./tcpconnect

PID    COMM         IP SADDR            DADDR            DPORT
15272  Socket Threa 4  10.0.2.15        91.189.89.240    80  
15272  Socket Threa 4  10.0.2.15        216.58.199.142   443 
15272  Socket Threa 4  10.0.2.15        216.58.199.142   80  
15272  Socket Threa 4  10.0.2.15        216.58.199.174   443 
15272  Socket Threa 4  10.0.2.15        54.200.62.216    443 
15272  Socket Threa 4  10.0.2.15        54.200.62.216    443 
15272  Socket Threa 4  10.0.2.15        117.18.237.29    80  
15272  Socket Threa 4  10.0.2.15        216.58.199.142   80  
15272  Socket Threa 4  10.0.2.15        216.58.199.131   80  
15272  Socket Threa 4  10.0.2.15        216.58.199.131   443 
15272  Socket Threa 4  10.0.2.15        52.222.135.52    443 
15272  Socket Threa 4  10.0.2.15        216.58.199.131   443 
15272  Socket Threa 4  10.0.2.15        54.200.62.216    443 
15272  Socket Threa 4  10.0.2.15        54.200.62.216    443 
15272  Socket Threa 4  10.0.2.15        216.58.199.132   443 
15272  Socket Threa 4  10.0.2.15        216.58.199.131   443 
15272  Socket Threa 4  10.0.2.15        216.58.199.142   443 
15272  Socket Threa 4  10.0.2.15        54.69.17.198     443 
15272  Socket Threa 4  10.0.2.15        54.69.17.198     443 
...

All the tools above can also be used with various options, to enable the help page for a given tool, make use of the -h option, for example:

$ sudo ./tcpconnect -h

usage: tcpconnect [-h] [-t] [-p PID] [-P PORT]

Trace TCP connects

optional arguments:
  -h, --help            show this help message and exit
  -t, --timestamp       include timestamp on output
  -p PID, --pid PID     trace this PID only
  -P PORT, --port PORT  comma-separated list of destination ports to trace.

examples:
    ./tcpconnect           # trace all TCP connect()s
    ./tcpconnect -t        # include timestamps
    ./tcpconnect -p 181    # only trace PID 181
    ./tcpconnect -P 80     # only trace port 80
    ./tcpconnect -P 80,81  # only trace port 80 and 81

Trace Failed exec()s Syscalls

To trace failed exec()s syscalls, employ the -x option with opensnoop as below:

$ sudo ./opensnoop -x

PID    COMM               FD ERR PATH
15414  pool               -1   2 /home/.hidden
15415  (ostnamed)         -1   2 /sys/fs/cgroup/cpu/system.slice/systemd-hostnamed.service/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/cpu/system.slice/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/cpuacct/system.slice/systemd-hostnamed.service/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/cpuacct/system.slice/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/blkio/system.slice/systemd-hostnamed.service/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/blkio/system.slice/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/memory/system.slice/systemd-hostnamed.service/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/memory/system.slice/cgroup.procs
15415  (ostnamed)         -1   2 /sys/fs/cgroup/pids/system.slice/systemd-hostnamed.service/cgroup.procs
2548   compiz             -1   2 
15416  systemd-cgroups    -1   2 /run/systemd/container
15416  systemd-cgroups    -1   2 /sys/fs/kdbus/0-system/bus
15415  systemd-hostnam    -1   2 /run/systemd/container
15415  systemd-hostnam    -1  13 /proc/1/environ
15415  systemd-hostnam    -1   2 /sys/fs/kdbus/0-system/bus
1695   dbus-daemon        -1   2 /run/systemd/users/0
15415  systemd-hostnam    -1   2 /etc/machine-info
15414  pool               -1   2 /home/tecmint/.hidden
15414  pool               -1   2 /home/tecmint/Binary/.hidden
2599   dconf-service      -1   2 /run/user/1000/dconf/user
...

Trace Particular Process Functions

The last example below demonstrates how to execute a custom trace operation. We are tracing a particular process using its PID.

Suggested Read: Netdata – A Real-Time Performance Monitoring Tool for Linux

First determine the process ID:

$ pidof firefox

15437

Later on, run the custom trace command. In the command below: -p specifies the process ID, do_sys_open() is a kernel function that is traced dynamically including its second argument as a string.

$ sudo ./trace -p 4095 'do_sys_open "%s", arg2'

TIME     PID    COMM         FUNC             -
12:17:14 15437  firefox      do_sys_open      /run/user/1000/dconf/user
12:17:14 15437  firefox      do_sys_open      /home/tecmint/.config/dconf/user
12:18:07 15437  firefox      do_sys_open      /run/user/1000/dconf/user
12:18:07 15437  firefox      do_sys_open      /home/tecmint/.config/dconf/user
12:18:13 15437  firefox      do_sys_open      /sys/devices/system/cpu/present
12:18:13 15437  firefox      do_sys_open      /dev/urandom
12:18:13 15437  firefox      do_sys_open      /dev/urandom
12:18:14 15437  firefox      do_sys_open      /usr/share/fonts/truetype/liberation/LiberationSans-Italic.ttf
12:18:14 15437  firefox      do_sys_open      /usr/share/fonts/truetype/liberation/LiberationSans-Italic.ttf
12:18:14 15437  firefox      do_sys_open      /usr/share/fonts/truetype/liberation/LiberationSans-Italic.ttf
12:18:14 15437  firefox      do_sys_open      /sys/devices/system/cpu/present
12:18:14 15437  firefox      do_sys_open      /dev/urandom
12:18:14 15437  firefox      do_sys_open      /dev/urandom
12:18:14 15437  firefox      do_sys_open      /dev/urandom
12:18:14 15437  firefox      do_sys_open      /dev/urandom
12:18:15 15437  firefox      do_sys_open      /sys/devices/system/cpu/present
12:18:15 15437  firefox      do_sys_open      /dev/urandom
12:18:15 15437  firefox      do_sys_open      /dev/urandom
12:18:15 15437  firefox      do_sys_open      /sys/devices/system/cpu/present
12:18:15 15437  firefox      do_sys_open      /dev/urandom
12:18:15 15437  firefox      do_sys_open      /dev/urandom
....

Summary

BCC is a powerful and easy-to-use toolkit for various System administration tasks such as tracing system performance monitoring, tracing block device I/O, TCP functions, file system operations, syscalls, Node.js probes, plus lots more. Importantly, it ships in with several example files and man pages for the tools to guide you, making it user friendly and reliable.

Last but not least, you can get back to us by sharing your thoughts about the subject, ask questions, make useful suggestions or any constructive feedback via the comment section below.

Source

If you are looking for a low resource, speedy server statistics monitoring script, look no further than linux-dash. Linux Dash’s claim to popular is its slick and responsive web dashboard that works better on large and small screens.

linux dash is a memory efficient, low resource, easy to install, server statistics monitoring script written in PHP. The web statistics page allows you to drag and drop the various widgets and rearrange the display as you desire. The script displays live statistics of your server, including RAM, CPU, Disk Space, Network Information, Installed Software’s, Running Processes and much more.

Linux Dash’s interface provides information in a organized fashion, which makes us easy to switch between specific sections using buttons in the main toolbar. Linux Dash is not an advanced monitoring tool like Collectlor Glances, but still it’s a good monitoring application for users who are looking for lightweight and easy to deploy.

linux-dash Demo

Please have a quick look at the demo page set up by the developer of linux-dash.

1. Watch Demo at: linux-dash: Server Monitoring

Linux Dash Features

1. A responsive web based interface for monitoring server resources.
2. A real-time monitoring of CPU, RAM, Disk Usage, Load, Uptime, Users and many more system statistics.
3. Easy install for servers with Apache/Nginx + PHP.
4. Click and drag to re-organize widgets.
5. Support for wide range of Linux server flavours.

Pre-requisites for Installation

1. A Linux server with Apache/Nginx installed.
2. A PHP and php-json extension installed.
3. A unzip utility installed on server.
4. Optionally, you need htpasswd installed, to password protect the statistics page on your server.

After all, you do not want to be displaying your statistics to the whole world, as it is a security risk.

Note: htpasswd is just one of the ways to protect your server. There are others such as denying access to certain IPs for instance. Use whichever way you are comfortable.

However, in this article, I’ve used Apache web server to show you how to setup linux-dash on Linux servers. I’ve also tested this nifty tool on other browsers such as Firefox, Midori and Chrome and it works fine.

Installing “linux-dash” in RedHat and Debian Based Systems

As I said above, that linux-dash is created in PHP for Linux with Apache. So, you must have these two packages installed on the server along with php-json module. Let’s install them using package manager tool called yum or apt-get according to your server distribution.

Step 1: Install Apache, PHP and PHP Modules

Install on Red Hat based systems using yum command.

# yum install httpd httpd-tools
# yum install php php-xml php-common php-json
# service httpd start

Install on Debian based systems using apt-get command.

# apt-get install apache2 apache2-utils
# apt-get install php5 curl php5-curl php5-json
# service apache2 start

Step 2: Download and Install linux-Dash

Proceed to ‘GitHub‘ repository, download linux-dash and extract contents into a sub-directory called ‘linux-dash‘ in your Apache public folder (i.e. /var/www or /var/www/html).

# git clone https://github.com/afaqurk/linux-dash.git

Step 3: Monitor Server using linux-dash

Open your browser and navigate to the folder where you have ‘linux-dash‘ installed. On mine it ishttp://localhost/linux-dash.

The following are some screenshots of linux-dash dashboard taken from my CentOS 6.5 server.

General Info

Disk Usage

CPU Usage

RAM Usage

Users

Network Statistics

Full linux-dash Preview

Step 4: Password Protect linux-dash

To password protect your statistics page, you need to generate an ‘.htaccess’ and ‘.htpasswd‘ file. The following command will create a user ‘admin‘, sets password ‘admin123‘ and creates new ‘htpasswd‘ file under ‘/var‘ folder.

# htpasswd -c /var/.htpasswd admin admin123

Note: The ‘htpasswd‘ file stores the user ‘admin‘ password in encrypted format and this file should be placed in a non public folder to protect from viewing in the browser.

Now create a ‘.htaccess‘ file under ‘linux-dash‘ directory and add the following content to it. Save and close the file.

AuthName "Restricted Area" 
AuthType Basic 
AuthUserFile /var/.htpasswd 
AuthGroupFile /dev/null 
require valid-user

Clear your browser’s cache. The next time you navigate to the statistics page, you will be greeted with a login prompt. Login with the username and password you used in the htpasswd command.

Reference Links

https://github.com/afaqurk/linux-dash

Enjoy your low resource, server statistics monitoring application.

Source

In this topic we’ll learn how to install and configure the latest official version of Nagios Core from sources in Debian and Ubuntu servers.

Nagios Core is a free Open Source network monitoring application designed for monitoring network applications, devices and their related services and in a network.

Nagios can remotely monitor specific operating system parameters via agents deployed on nodes and send alerts via mail or SMS in order to notify administrators in case critical services in a network, such as SMTP, HTTP, SSH, FTP and other fails.

Requirements

1. Debian 9 Minimal Installation
2. Ubuntu 16.04 Minimal Installation

Step 1: Install Pre-requirements for Nagios

1. Before installing Nagios Core from sources in Ubuntu or Debian, first install the following LAMP stack components in your system, without MySQL RDBMS database component, by issuing the below command.

# apt install apache2 libapache2-mod-php7.0 php7.0

2. On the next step, install the following system dependencies and utilities required to compile and install Nagios Core from sources, by issuing the follwoing command.

# apt install wget unzip zip  autoconf gcc libc6 make apache2-utils libgd-dev

Step 2: Install Nagios 4 Core in Ubuntu and Debian

3. On the first step, create nagios system user and group and add nagios account to the Apache www-data user, by issuing the below commands.

# useradd nagios
# usermod -a -G nagios www-data

4. After all dependencies, packages and system requirements for compiling Nagios from sources are present in your system, go to Nagios webpage and grab the latest version of Nagios Core stable source archive by issuing the following command.

# wget https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.3.4.tar.gz

5. Next, extract Nagios tarball and enter the extracted nagios directory, with the following commands. Issue ls command to list nagios directory content.

# tar xzf nagios-4.3.4.tar.gz 
# cd nagios-4.3.4/
# ls

6. Now, start to compile Nagios from sources by issuing the below commands. Make sure you configure Nagios with Apache sites-enabled directory configuration by issuing the below command.

# ./configure --with-httpd-conf=/etc/apache2/sites-enabled

7. In the next step, build Nagios files by issuing the following command.

# make all

8. Now, install Nagios binary files, CGI scripts and HTML files by issuing the following command.

# make install

9. Next, install Nagios daemon init and external command mode configuration files and make sure you enable nagios daemon system-wide by issuing the following commands.

# make install-init
# make install-commandmode
# systemctl enable nagios.service

10. Next, run the following command in order to install some Nagios sample configuration files needed by Nagios to run properly by issuing the below command.

# make install-config

11. Also, install Nagios configuration file for Apacahe web server, which can be fount in /etc/apacahe2/sites-enabled/ directory, by executing the below command.

# make install-webconf

12. Next, create nagiosadmin account and a password for this account necessary by Apache server to log in to Nagios web panel by issuing the following command.

# htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

13. To allow Apache HTTP server to execute Nagios cgi scripts and to access Nagios admin panel via HTTP, first enable cgi module in Apache and then restart Apache service and start and enable Nagios daemon system-wide by issuing the following commands.

# a2enmod cgi
# systemctl restart apache2
# systemctl start nagios
# systemctl enable nagios

14. Finally, log in to Nagios Web Interface by pointing a browser to your server’s IP address or domain name at the following URL address via HTTP protocol. Log in to Nagios with nagiosadmin user the password setup with htpasswd script.

http://IP-Address/nagios
OR
http://DOMAIN/nagios

15. To view your hosts status, navigate to Current Status -> Hosts menu where you will notice that some errors are displayed for localhost host, as illustrated in the below screenshot. The error appears because Nagios has no plugins installed to check hosts and services status.

Step 3: Install Nagios Plugins in Ubuntu and Debian

16. To compile and install Nagios Plugins from sources in Debian or Ubuntu, at the first stage, install the following dependencies in your system, by issuing the below command.

# apt install libmcrypt-dev make libssl-dev bc gawk dc build-essential snmp libnet-snmp-perl gettext libldap2-dev smbclient fping libmysqlclient-dev qmail-tools libpqxx3-dev libdbi-dev 

17. Next, visit Nagios Plugins repositories page and download the latest source code tarball by issuing the following command.

# wget https://github.com/nagios-plugins/nagios-plugins/archive/release-2.2.1.tar.gz 

18. Go ahead and extract the Nagios Plugins source code tarball and change path to the extracted nagios-plugins directory by executing the following commands.

# tar xfz release-2.2.1.tar.gz 
# cd nagios-plugins-release-2.2.1/

19. Now, start to compile and install Nagios Plugins from sources, by executing the following series of commands in your server console.

# ./tools/setup 
# ./configure 
# make
# make install

20. The compiled and installed Nagios plugins can be located in /usr/local/nagios/libexec/ directory. List this directory to view all available plugins in your system.

# ls /usr/local/nagios/libexec/

21. Finally, restart Nagios daemon in order to apply the installed plugins, by issuing the below command.

# systemctl restart nagios.service

22. Next, log in to Nagios web panel and go to Current Status -> Services menu and you should notice all hosts services are checked now by Nagios plugins.

From the color code you should see the current services status: green color is for OK status, yellow for Warningand red for Critical status.

23. Finally, to access Nagios admin web interface via HTTPS protocol, issue the following commands to enable Apache SSL configurations and restart the Apache daemon to reflect changes.

# a2enmod ssl 
# a2ensite default-ssl.conf
# systemctl restart apache2

24. After you’ve enabled Apache SSL configurations, open /etc/apache2/sites-enabled/000-default.conf file for editing and add the following block of code after DocumentRoot statement as shown in the below excerpt.

RewriteEngine on
RewriteCond %{HTTPS} off
RewriteRule ^(.*) https://%{HTTP_HOST}/$1

25. You need to restart Apache daemon to apply the configured rules, by issuing the below command.

# systemctl restart apache2.service 

26. Finally, refresh the browser in order to be redirected to Nagios admin panel via HTTPS protocol. Accept the wanting message that gets displayed in the browser and log in to Nagios again with the your credentials.

Congratulations! You have successfully install and configured Nagios Core monitoring system from sources in Ubuntu server or Debian 9.

Source

rtop is a straightforward and interactive, remote system monitoring tool based on SSH that collects and shows important system performance values such as CPU, disk, memory, network metrics.

It is written in Go Language and does not require any extra programs to be installed on the server that you want to monitor except SSH server and working credentials.

rtop basically functions by launching an SSH session, and executing certain commands on the remote server to gather various system performance information.

Once an SSH session is been established, it keeps refreshing the information collected from the remote server every few seconds (5 seconds by default), similar to all other top-like utilities (like htop) in Linux.

Prerequisites:

Make sure you have installed Go (GoLang) 1.2 or higher on your Linux system in order to install rtop, otherwise click on the link below to follow the GoLang installation steps:

1. Install GoLang (Go Programming Language) in Linux

How to Install rtop in Linux Systems

If you have Go installed, run the command below to build rtop:

$ go get github.com/rapidloop/rtop

The rtop executable binary will be saved in $GOPATH/bin or $GOBIN once the command completes executing.

Note: You do not need any runtime dependencies or configurations to start using rtop.

How to Use rtop in Linux Systems

Try to run rtop without any flags and arguments as below, it will display a usage message:

$ $GOBIN/rtop

Sample Output

rtop 1.0 - (c) 2015 RapidLoop - MIT Licensed - http://rtop-monitor.org
rtop monitors server statistics over an ssh connection

Usage: rtop [-i private-key-file] [user@]host[:port] [interval]

	-i private-key-file
		PEM-encoded private key file to use (default: ~/.ssh/id_rsa if present)
	[user@]host[:port]
		the SSH server to connect to, with optional username and port
	interval
		refresh interval in seconds (default: 5)

Now let’s monitor the remote Linux server using rtop as follows, while refreshing the information gathered after an interval of 5 seconds by default:

$ $GOBIN/rtop   aaronkilik@10.42.0.1 

The command below will refresh the system performance metrics collected after every 10 seconds:

$ $GOBIN/rtop aaronkilik@10.42.0.1 10

rtop can also connect using ssh-agent, private keys or password authentication.

Visit rtop Github repository: https://github.com/rapidloop/rtop

As a concluding remark, rtop is a simple and easy-to-use remote server monitoring tool, it uses very few and direct options. You can as well read about several other command line tools to monitor your system so as to improve your Linux performance monitoring skills.

Lastly, get in touch with us via the comment section below for any questions or remarks.

Source

CloudStats.me is a new linux server monitoring tool which is very easy to use yet capable of providing most vital information about your server. CloudStats was developed with simplicity in mind. Most existing server monitoring tools are either quite expensive or have a tedious installation procedure. In contrast, once you sign up for a CloudStats account, you will be asked to run only 1 command on your server using SSH command line tool. That command will install the monitoring Agent on your server which will start sending statistics to your CloudStats account.

From now on, no matter where you are – you can always login to your CloudStats account using a home PC or a mobile phone and see full information about your server, including CPU, Disk, RAM, Network usage etc. In addition, CloudStats.me can monitor Apache, DNS, MySQL, Mail, FTP and other services on your server. Once your server or a service goes down, you will receive an Alert via an email. Alternatively, you can receive alerts via Skype chat message. This makes monitoring your server a stress-free job whereas you can focus on your project and be sure that you will know immediately if something happens to your server.

The Email and Skype alerts are fully configurable so that you can define when you receive them. For example, you can configure to be notified when your free Disk Space is very low, or when RAM Usage on your server is too high. This would indicate that you need to login to your server and investigate or contact your server technical support team for them to check this for you. It is vital to know what is happening inside your server to make sure the project that you are working on is successful.

Whether you are running your server on CentOS, Debian, Ubuntu or Fedora, CloudStats.me is the best choice for monitoring it. What’s more, CloudStats can even monitor your Windows-based servers, which makes it a swiss-knife like tool for server monitoring.

CloudStats Features

Here is a list of features that CloudStats includes:

1. Linux and Windows Server Monitoring – network usage, cpu usage, disk usage, running processes, uptime, io usage, server load, memory usage etc.
2. Support of CentOS, Debian, Ubuntu and Windows servers.
3. Support of VPS, Dedicated and Cloud servers.
4. One click agent installation.
5. Your personal account with “youraccount.cloudstats.me” subdomain.
6. URL Monitoring and Pingmap.
7. Service monitoring – HTTP, DNS etc.
8. Port monitoring – 80, 443 etc.
9. Intuitive Dashboard.
10. Cloud-based, creating no load on your servers, no need to host anything by yourself.
11. Powerful charts.
12. Suitable for single and multi-users.
13. Suitable for hundreds of servers, as well as for small VPS servers.
14. Configurable EMAIL, Skype Alerts.
15. Free Server UP/Down Alerts.

So, Why CloudStats?

CloudStats.me is an easy to use server monitoring platform which is much cheaper compared to other similar products. Yet CloudStats boasts many features that are not always present in competitive software tools. Furthermore, you don’t need to host it on your server – everything is done from the Cloud, which minimizes your expenses and lets your focus on your work.

Special Deal for TecMint readers!

TecMint team has worked with CloudStats to offer visitors a nice deal – only $1 unlimited server monitoring package during the first month of it’s use.

The promocode should be used during sign up procedure for CloudStats Enterprise account, the second month price will be $29.95 which will allow you to keep monitoring as many servers and websites as you like. Similar packages at other companies may easily cost you $100+, so it’s best to go for CloudStats. You can always downgrade the package to suit your needs at any time.

CloudStats also has a completely free account for those who plan to monitor 3 servers or less.

CloudStats Screen Shot Tour

Here are some screen shots of CloudStats:

Server List Overview

Dashboard Overview

Agent Installation Overview

This article describes how to monitor Windows machines “private” services such as CPU load, Disk usage, Memory usage, Services, etc. For this, we required to install an NSClient++ addon on the Windows machine. The addon acts a proxy between the Windows machine and Nagios and monitors actual services by communicating with the check_nt plugin. The check_nt plugin already installed on the Nagios Monitoring Server, if you followed our Nagios installation guide.

We assume that you’ve already installed and configured Nagios server according to our following guides.

1. How to Install Nagios 4.0.1 on RHEL/CentOS 6.x/5.x and Fedora 19/18/17
2. Add Linux Host to Nagios Monitoring Server

To monitor Windows Machines you will need to follow several steps and they are:

1. Install NSClient++ addon on the Windows Machine.
2. Configure Nagios Server for monitoring Windows Machine.
3. Add new host and service definitions for Windows machine monitoring.
4. Restart the Nagios Service.

To make this guide simple and easier, a few of configuration already done for you in the Nagios installation.

1. A check_nt command definition already added to the command.cfg file. This definition command is used by check_nt plugin to monitor Windows services.
2. A windows-server host template already created in the templates.cfg file. This template allows you to add new Windows host definitions.

The above two files “command.cfg” and “templates.cfg” files can be found at /usr/local/nagios/etc/objects/directory. You can modify and add your own definitions that suits your requirement. But, I’d recommend you to follow the instructions described in this article and you will be successfully monitoring your windows host in less than 20 minutes.

Step 1: Installing NSClient++ Agent on Windows Machine

Please use the below instructions to install NSClient++ Agent on the Remote Windows Host. First download the latest stable version NSClient++ 0.3.1 addon source files, which can be found at below link.

1. http://sourceforge.net/projects/nscplus/

Once you’ve downloaded latest stable version, unzip the NSClient++ files into a new C:\NSClient++ directory.

Now open a MS-DOS command prompt from the Start Screen –> Run –> type ‘cmd‘ and press enter and change to the C:\NSClient++ directory.

C:\NSClient++

Next, register the NSClient++ service on the system with the following command.

nsclient++ /install

Finally, install the NSClient++ systray with the following command.

nsclient++ SysTray

Open the Windows Services Manager and right click on NSClient go to Properties and then ‘Log On‘ tab and click the check box that says “Allow service to interact with the desktop“. If it isn’t already allowed, please check the box to allow it to.

Open NSC.INI file located at C:\NSClient++ directory and uncomment all the modules defined in the “modules” section, except for CheckWMI.dll and RemoteConfiguration.dll.

[modules]
;# NSCLIENT++ MODULES
;# A list with DLLs to load at startup.
;  You will need to enable some of these for NSClient++ to work.
; ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
; *                                                               *
; * N O T I C E ! ! ! - Y O U   H A V E   T O   E D I T   T H I S *
; *                                                               *
; ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !
FileLogger.dll
CheckSystem.dll
CheckDisk.dll
NSClientListener.dll
NRPEListener.dll
SysTray.dll
CheckEventLog.dll
CheckHelpers.dll
;CheckWMI.dll
;
; RemoteConfiguration IS AN EXTREM EARLY IDEA SO DONT USE FOR PRODUCTION ENVIROMNEMTS!
;RemoteConfiguration.dll
; NSCA Agent is a new beta module use with care!
;NSCAAgent.dll
; LUA script module used to write your own "check deamon" (sort of) early beta.
;LUAScript.dll
; Script to check external scripts and/or internal aliases, early beta.
;CheckExternalScripts.dll
; Check other hosts through NRPE extreme beta and probably a bit dangerous! :)
;NRPEClient.dll

Uncomment the “allowed_hosts” in the “Settings” section and define the IP address of your Nagios Monitoring Server or leave it blank to allow any hosts to connect.

[Settings]
;# ALLOWED HOST ADDRESSES
;  This is a comma-delimited list of IP address of hosts that are allowed to talk to the all daemons.
;  If leave this blank anyone can access the deamon remotly (NSClient still requires a valid password).
;  The syntax is host or ip/mask so 192.168.0.0/24 will allow anyone on that subnet access
allowed_hosts=172.16.27.41

Uncomment the “port” in the “NSClient” section and set to default port ‘12489‘. Make sure to open ‘12489‘ port on Windows Firewall.

[NSClient]
;# NSCLIENT PORT NUMBER
;  This is the port the NSClientListener.dll will listen to.
port=12489

Finally start the NSClient++ service with the following command.

nsclient++ /start

If your properly installed and configured, you should see a new icon in the system tray in yellow circle with a black ‘M‘ inside.

Step 2: Configuring Nagios Server and Add Windows Hosts

Now Login into Nagios Server and add some object definitions in Nagios configuration files to monitor new Windows machine. Open windows.cfg file for editing with Vi editor.

[root@tecmint]# vi /usr/local/nagios/etc/objects/windows.cfg

A sample Windows host definition already defined for the Windows machine, you can simply change the host definition like host_name, alias, and address fields to appropriate values of your Windows machine.

###############################################################################
###############################################################################
#
# HOST DEFINITIONS
#
###############################################################################
###############################################################################

# Define a host for the Windows machine we'll be monitoring
# Change the host_name, alias, and address to fit your situation

define host{
        use             windows-server  ; Inherit default values from a template
        host_name       winserver       ; The name we're giving to this host
        alias           My Windows Server       ; A longer name associated with the host
        address         172.31.41.53    ; IP address of the host
        }

Following services are already added and enabled in windows.cfg file. If you wish to add some more other service definitions that needs to be monitored, you can simple add those definitions to same configuration file. Make sure to change the host_name for these all services with host_name defined in the above step.

define service{
	use			generic-service
	host_name		winserver
	service_description	NSClient++ Version
	check_command		check_nt!CLIENTVERSION
	}

Add the following service definition to monitor the uptime of the Windows server.

define service{
	use			generic-service
	host_name		winserver
	service_description	Uptime
	check_command		check_nt!UPTIME
	}

Add the following service definition to monitor the CPU utilization on the Windows server and generate a CRITICAL alert if the 5-minute CPU load is 90% or more or a WARNING alert if the 5-minute load is 80% or greater.

define service{
	use			generic-service
	host_name		winserver
	service_description	CPU Load
	check_command		check_nt!CPULOAD!-l 5,80,90
	}

Add the following service definition to monitor memory usage on the Windows server and generate a CRITICAL alert if memory usage is 90% or more or a WARNING alert if memory usage is 80% or greater.

define service{
	use			generic-service
	host_name		winserver
	service_description	Memory Usage
	check_command		check_nt!MEMUSE!-w 80 -c 90
	}

Add the following service definition to monitor usage of the C:\ drive on the Windows server and generate a CRITICAL alert if disk usage is 90% or more or a WARNING alert if disk usage is 80% or greater.

define service{
	use			generic-service
	host_name		winserver
	service_description	C:\ Drive Space
	check_command		check_nt!USEDDISKSPACE!-l c -w 80 -c 90
	}

Add the following service definition to monitor the W3SVC service state on the Windows machine and generate a CRITICAL alert if the service is stopped.

define service{
	use			generic-service
	host_name		winserver
	service_description	W3SVC
	check_command		check_nt!SERVICESTATE!-d SHOWALL -l W3SVC
	}

Add the following service definition to monitor the Explorer.exe process on the Windows machine and generate a CRITICAL alert if the process is not running.

define service{
	use			generic-service
	host_name		winserver
	service_description	Explorer
	check_command		check_nt!PROCSTATE!-d SHOWALL -l Explorer.exe
	}

Lastly, uncomment the windows.cfg file in /usr/local/nagios/etc/nagios.cfg.

[root@tecmint]# vi /usr/local/nagios/etc/nagios.cfg

# Definitions for monitoring a Windows machine
cfg_file=/usr/local/nagios/etc/objects/windows.cfg

Finally, verify the Nagios configuration files for any erros.

[root@tecmint]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check

If the verification process throws any error messages, fix those errors until the verification process completes without any error messages. Once’ you fix those errors, restart the Nagios service.

[root@tecmint]# service nagios restart

Running configuration check...done.
Stopping nagios: done.
Starting nagios: done.

That’s it. Now go to Nagios Monitoring Web interface at “http://Your-server-IP-address/nagios” or “http://FQDN/nagios” and Provide the username “nagiosadmin” and password. Check that the Remote Windows Host was added and is being monitored.

That’s it! for now, in my up-coming article I will show you how to add Printer and Switches to Nagios Monitoring Server. If you’re having any difficulties while adding Windows host to Nagios. Please do comment your queries via comment section, till then stay tuned to Tecmint.com for more such kind of valuable articles.

Source