tutorial

Monitoring Nagios with Vigilmon

Nagios monitors your infrastructure — but who monitors Nagios? Here's how to set up external Vigilmon monitoring for the Nagios web interface, CGI layer, NRPE agent port, SSL certificates, and service check heartbeats.

Nagios has been the gold standard of IT infrastructure monitoring since 1999. Nagios Core and Nagios XI watch thousands of hosts and services, alerting on failures across your entire environment. But Nagios has a blind spot: it can't monitor itself. When Nagios goes down — the web interface becomes unreachable, the CGI layer stops responding, or the monitoring daemon crashes — you lose visibility across everything it watches. Vigilmon solves this by providing external, independent monitoring for Nagios itself.

What You'll Set Up

  • HTTP uptime monitor for the Nagios web interface (/nagios/)
  • CGI availability check for the Nagios CGI layer
  • TCP port monitor for the NRPE agent (port 5666)
  • SSL certificate expiry alerts for Nagios HTTPS setup
  • Heartbeat monitoring for Nagios service check execution

Prerequisites

  • Nagios Core 4.x or Nagios XI installed
  • Nagios web interface accessible at http://yourhost/nagios/
  • NRPE installed on monitored hosts (port 5666)
  • A free Vigilmon account

Step 1: Monitor the Nagios Web Interface

The Nagios web interface is the operational dashboard your team uses to view host and service states. Monitoring it with Vigilmon confirms Apache/nginx is running and the Nagios web assets are being served.

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: http://yourhost/nagios/
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

The Nagios web interface typically requires HTTP Basic Authentication. To monitor authenticated pages with Vigilmon:

  1. Open the monitor you just created.
  2. Add a request header:
    • Name: Authorization
    • Value: Basic followed by the Base64-encoded nagiosadmin:yourpassword

Generate the Base64 value:

echo -n "nagiosadmin:yourpassword" | base64
# Output: bmFnaW9zYWRtaW46eW91cnBhc3N3b3Jk

Set the header value to Basic bmFnaW9zYWRtaW46eW91cnBhc3N3b3Jk. Now Vigilmon authenticates and confirms the full web interface is accessible.

Step 2: Monitor CGI Availability

Nagios uses CGI scripts to render its web interface dynamically. The CGI layer depends on Apache's mod_cgi or mod_cgid being enabled and nagios.cgi being executable. CGI failures produce a web interface that partially loads but shows no host or service data.

Add a CGI-specific monitor:

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter the URL: http://yourhost/nagios/cgi-bin/status.cgi?hostgroup=all&style=summary
  3. Add the same Authorization header from Step 1.
  4. Set Expected HTTP status to 200.
  5. Set Expected response body to contain Current Network Status.
  6. Set Check interval to 5 minutes.
  7. Click Save.

This checks the CGI status page rather than the static web interface. If the CGI layer breaks (e.g. a Nagios upgrade breaks the CGI binary permissions), the static interface still loads but status.cgi returns a 500 or empty page.

Step 3: Monitor the NRPE Agent Port (TCP 5666)

NRPE (Nagios Remote Plugin Executor) runs on monitored hosts and lets Nagios execute plugins remotely. Port 5666 must be reachable from the Nagios server for active checks to work. A TCP monitor from Vigilmon verifies NRPE is listening on your critical hosts.

For each important monitored host:

  1. Click Add Monitor → TCP Port.
  2. Enter the hostname or IP of the monitored host.
  3. Set Port to 5666.
  4. Set Check interval to 5 minutes.
  5. Click Save.

This catches situations where NRPE stops accepting connections (e.g. xinetd or nrpe daemon crashes) before Nagios itself detects it through its own check cycle — which may not run for 5–30 minutes depending on your check interval.

For the Nagios server itself, also add a TCP monitor for the Nagios daemon's listening port if you're running NSClient++ or NSCA for passive check submission:

  • NSCA (passive check receiver): TCP port 5667

Step 4: SSL Certificate Alerts for Nagios HTTPS

Production Nagios installations typically run behind HTTPS with Apache or nginx. Certificate expiry on the monitoring interface blocks access for your ops team at exactly the moment they might need it most.

Enable SSL monitoring on your HTTPS monitor:

  1. Open the HTTP / HTTPS monitor created in Step 1 (update the URL to https://yourhost/nagios/).
  2. Enable Monitor SSL certificate under the SSL section.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

For Nagios XI installations that use self-signed certificates, you may need to disable strict SSL verification in Vigilmon and rely on the certificate expiry date check rather than chain validation.

Also verify your Nagios Apache config is serving HTTPS correctly:

apachectl -M | grep ssl
# ssl_module (shared)

# Check certificate expiry manually
openssl s_client -connect yourhost:443 -servername yourhost 2>/dev/null | \
  openssl x509 -noout -dates

Step 5: Heartbeat Monitoring for Nagios Service Checks

Nagios executes service checks on a schedule. If the Nagios daemon (nagios process) crashes or hangs, no checks run — but the web interface might still load cached state from the status file, making the failure invisible until someone notices stale timestamps.

Use Vigilmon's heartbeat to verify Nagios is actively executing checks.

Option A: cron-based heartbeat

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set the expected ping interval to 5 minutes.
  3. Copy the heartbeat URL (e.g. https://vigilmon.online/heartbeat/abc123).
  4. Create a Nagios event handler or cron job:
# /etc/cron.d/nagios-heartbeat
*/5 * * * * nagios /usr/sbin/nagios -v /etc/nagios/nagios.cfg > /dev/null 2>&1 && curl -s https://vigilmon.online/heartbeat/abc123

This validates the Nagios config and then pings — if nagios isn't running or the config is broken, the curl never executes.

Option B: Nagios event handler

Add a service check that pings Vigilmon on success:

# /usr/lib/nagios/plugins/check_vigilmon_heartbeat.sh
#!/bin/bash
curl -s https://vigilmon.online/heartbeat/abc123
echo "HEARTBEAT OK - pinged Vigilmon"
exit 0

chmod +x /usr/lib/nagios/plugins/check_vigilmon_heartbeat.sh

Define the service check in Nagios:

# /etc/nagios/conf.d/vigilmon-heartbeat.cfg
define service {
    use                     generic-service
    host_name               localhost
    service_description     Vigilmon Heartbeat
    check_command           check_vigilmon_heartbeat
    check_interval          5
    max_check_attempts      1
}

define command {
    command_name            check_vigilmon_heartbeat
    command_line            /usr/lib/nagios/plugins/check_vigilmon_heartbeat.sh
}

Reload Nagios:

systemctl reload nagios
# or
nagios -v /etc/nagios/nagios.cfg && systemctl restart nagios

Now if the Nagios daemon stops scheduling checks, the heartbeat service never executes, and Vigilmon alerts you after 5 minutes.

Step 6: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add email, Slack, PagerDuty, or a webhook.
  2. Set Consecutive failures before alert to 2 on the web interface monitor — Apache may briefly restart during Nagios reloads.
  3. Route Vigilmon's Nagios-down alerts to a separate escalation channel from your normal Nagios alerts, since your entire alert chain depends on Nagios being up.
  4. Use Maintenance windows during Nagios upgrades:
curl -X POST https://vigilmon.online/api/maintenance \
  -H "Authorization: Bearer YOUR_VIGILMON_API_KEY" \
  -d '{"monitor_id": "abc123", "duration_minutes": 30}'

yum update nagios
# or
apt-get upgrade nagios4

Summary

| Monitor | Target | What It Catches | |---|---|---| | Web interface | http://yourhost/nagios/ | Apache down, web assets broken | | CGI availability | /nagios/cgi-bin/status.cgi | CGI layer broken, wrong permissions | | NRPE port | TCP :5666 | NRPE daemon stopped, firewall change | | SSL certificate | HTTPS domain | Certificate expiry, renewal failure | | Service check heartbeat | Heartbeat URL | Nagios daemon crashed, checks stalled |

The irony of infrastructure monitoring is that your monitoring system is the last thing you'd know about if it failed — because it's the system that would tell you. With Vigilmon providing independent external monitoring for the Nagios web interface, CGI layer, NRPE ports, SSL certificates, and check execution heartbeat, you close Nagios's one blind spot: itself.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles