Xymon is an agent-based1 monitoring solution. That means that a xymon-client is installed on the monitored system. For an unixoide OS hat client collects various metrics via a simple shell script and sends that data to the Xymon server using the xymon-binary2 with a simple clear-text protocol.

On some systems it is not possible for technical (no installation permissions, client not available, no compile environment) or legal (appliance vendors often forbid installing 3rd party applications) reasons to install the xymon client.

But these systems often have remote access possibilities — nowadays mostly SSH. In that case these can be monitored anyway using the server-side extension xymon-rclient3. This article describes the setup using QNap NAS appliances and SSH for remote access as an example.

As a server-side extension xymon-rclient is installed on the Xymon server. All configuration is done via tags in hosts.cfg. Except for ensuring non-interactive (password-less) access to the monitored system by adding an ssh public key to the authorized_keys-file nothing is changed on the client side (the QNap system in our example).

Installation

The installation of xymon-rclient is very simple and non-intrusive:

  • download the xymon-rclient extension
  • install according to the instructions

    1. copy the xymon-rclient.sh script to /usr/lib/xymon/server/ext/
    2. create the task to run the script periodically by adding the following to tasks.cfg (or task.d/<file>, see also How to Modularize the Configuration Files of Xymon

        [xymon-rclient]
        ENVFILE $XYMONHOME/etc/xymonserver.cfg
        CMD /$XYMONHOME/ext/xymon-rclient.sh
        LOGFILE $XYMONSERVERLOGS/xymon-rclient.log
        INTERVAL 5m
      
  • the xymon-user on the server should have an ssh-key, you may want to generate one with ssh-keygen

Configuration

Prepare the QNap system

Login to the QNap (this requires ssh-access to be enabled on the QNap) or use ssh-copy-id to prepare the authorized_keys-file. Note that only the user admin can connect with ssh (this is QNap specific), for all other usernames the connection is disconnected immediately).

Using ssh-copy-id to prepare password-less login: (make sure your are logged in as the xymon-user locally)

    ssh-copy-id admin@qnap.local

Manually prepare the authorized_keys-file on the client:

    ssh admin@qnap.local
    mkdir ~/.ssh && chmod 700 ~/.ssh
    vi ~/.ssh/authorized_keys // paste contents of the `~xymon/.ssh/id_rsa.pub`-file

Check that you can login to the QNap from the xymon user on the server without password now. This also ensures that the remote-hostkey ist added to the local known-hosts file.

Server-side config in hosts.cfg

To start monitoring systems with rclient we only need to add the RCLIENT-tag to the qnap.local-host and specify connection command (ssh) the username (admin) and the OS-type. For our QNap system this may look as follows:

1.2.3.4 qnap.local  # https://qnap.local/ "RCLIENT:cmd(ssh -T admin@%{H}),ostype(linux)"

This is the minimal configuration that already collects some data (cpu, uptime, ifconfig, …) — more on that below. After at least 10 minutes the configured system should show up on the Xymon web interface. If no data is reported there may be something wrong with the “task part” on the server or the ssh connection fails. The logfiles xymonlaunch.log and xymon-rclient.log may contains useful pointers.

There are quite some ways for debugging connection problems by running manually that are shown below.

First: You can get the full help of the rclient-script with

root@xymon:/usr/lib/xymon/server/ext# xymoncmd /usr/lib/xymon/server/ext/xymon-rclient.sh -h

Troubleshooting

Running the script manually with some debugging may give us some hints

root@xymon:/usr/lib/xymon/server/ext# xymoncmd /usr/lib/xymon/server/ext/xymon-rclient.sh -m qnap -y -d 1
2015-11-20 12:49:13.491252 Using default environment file /usr/lib/xymon/client/etc/xymonserver.cfg
Fri Nov 20 12:49:13 MSK 2015: starting /usr/lib/xymon/server/ext/xymon-rclient.sh
Fri Nov 20 12:49:13 MSK 2015
Adding hosts from hosts.cfg
Dry-run mode enabled
Fri Nov 20 12:49:13 MSK 2015: finished /usr/lib/xymon/server/ext/xymon-rclient.sh (completed 0 out of 0)

The options used are:

  • -m <hostmatch> selects the RCLIENT-lines from hosts.cfg where the hostname matches <hostmatch>.
  • -y prevents data from being sent to the server
  • -d 1 enables debugging. With debug level 1 you get information about the matching host(s) and the remote-connection command.

Here no matching system is found (completed 0 out of 0). In fact no RCLIENT-line is configured at all in hosts.cfg and thus no host matching qnap was found.

After adding our host along with two other (fake) entries to hosts.cfg

1.2.3.4 qnap.local  # https://qnap.local/ "RCLIENT:cmd(ssh -T admin@%{H}),ostype(linux)"
1.2.3.5 panq.local  # "RCLIENT:cmd(ssh -T admin@%{H}),ostype(linux)"
1.2.3.6 foo.local   # "RCLIENT:cmd(ssh -T admin@%{H}),ostype(linux)"

We get the following output

root@xymon:/usr/lib/xymon/server/ext# xymoncmd /usr/lib/xymon/server/ext/xymon-rclient.sh -m qnap -y -d 1
2015-11-20 12:54:44.158487 Using default environment file /usr/lib/xymon/client/etc/xymonserver.cfg
Fri Nov 20 12:54:44 MSK 2015: starting /usr/lib/xymon/server/ext/xymon-rclient.sh
Fri Nov 20 12:54:44 MSK 2015
Adding hosts from hosts.cfg
Skipping host panq.local, doesn't match /qnap/
Skipping host foo.local, doesn't match /qnap/
Dry-run mode enabled
Server 1 qnap.local(linux)
Command: ssh -T admin@qnap.local
Fri Nov 20 12:54:50 MSK 2015: Failed to collect data for qnap.local
Fri Nov 20 12:54:50 MSK 2015: finished /usr/lib/xymon/server/ext/xymon-rclient.sh (completed 0 out of 1)

We see the non-matching host-entries skipped and the ssh-command used. As the system qnap.local does not exist the remote connection fails. If everything is configured correctly you would get a “completed”-count greater than zero.

Conclusion

We can do “agent-less” monitoring with Xymon using the rclient-extension. Except for the remote-access configuration nothing has to be changed on the monitored host and all configuration is centralised on the Xymon server.

The remote connection with ssh was used as an example but rsh or even netcat connections are possible too (beware of the security implications though) by adjusting the cmd(...)-part of the RCLIENT-tag.

As noted QNap systems provide a limited (or: different) userland based on busybox: So some commands are missing, some have different output and thus cannot be parsed by the Xymon default xymonclient-linux.sh-script. This includes disk- and memory-usage which do not work out-of-the-box.

In the next article we extend the xymonclient-script to play better with busybox-based systems to provide us with a more complete monitoring of the QNap and other busybox based systems.


  1. Strictly speaking there is no such thing like agent-less monitoring for OS metrics. But the agent may be included in the default installation (like it is often the case for SNMP). The pure checking of network services can be considered agent-less (icmp-ping, http, ssh, …) as the agent is inherently the monitored service. XXX: Apologies for the rather catchy title.

  2. The example in this article uses SSH as the transport instead of the xymon-binary and system-tools called by the client-shell-script. This can be considered the (pre installed) agent.

  3. Created and made available by Jeremy Laidman.