How to Monitor OpenNMS with Vigilmon

OpenNMS is the network monitoring platform that watches everything else — SNMP devices, servers, services, flows, and application performance across hundreds to tens of thousands of nodes. But OpenNMS itself has failure modes that go undetected until your ops team asks why that router alarm from three days ago never fired.

In this tutorial you'll set up monitoring for OpenNMS Horizon (or Meridian) using Vigilmon, covering the web UI, REST API, PostgreSQL backend, and the daemon health that keeps polling and collection running.

Why monitoring your monitoring platform matters

OpenNMS failures are uniquely high-impact because they're silent by design:

• Poller daemon backlog — OpenNMS falls behind on service polls; existing alarms stay open but new outages go undetected
• PostgreSQL connection exhaustion — new event data can't be written; you see a working UI but no new alerts fire
• collectd collection failures — SNMP collection stops silently; dashboards show stale data with no indication that collection has lapsed
• Event bus saturation — trap floods or threshold storms fill the queue; event processing lag grows until OpenNMS drops events
• Java heap exhaustion — the OpenNMS JVM runs out of memory; the web UI starts throwing 500 errors before the process crashes

A second monitoring layer — Vigilmon watching OpenNMS from outside — catches all of these independently.

What you'll need

• A running OpenNMS Horizon or Meridian instance (default web UI on port 8980)
• The OpenNMS REST API enabled (enabled by default)
• A free Vigilmon account

Step 1: Verify OpenNMS REST API access

OpenNMS exposes a REST API at http://your-opennms:8980/opennms/rest/. Verify it's accessible:

curl -u admin:admin http://your-opennms:8980/opennms/rest/info

A healthy response returns JSON with OpenNMS version and display version:

{
  "displayVersion": "32.0.1",
  "version": "32.0.1",
  "packageName": "opennms",
  "packageDescription": "OpenNMS"
}

The admin/admin credentials are the default — replace with your actual admin user in production. For monitoring, create a dedicated read-only role user:

# In OpenNMS admin UI: Admin → Configure Users and Groups → Add New User
# Assign the "ROLE_REST" and "ROLE_USER" roles only
# This user can read REST API data but cannot modify configuration

Step 2: Monitor the OpenNMS web UI and REST API

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS as the type
3. Set the URL to http://your-opennms:8980/opennms/rest/info
4. Under Authentication, set Basic Auth with your read-only monitoring user
5. Set check interval to 1 minute
6. Under Expected response, set:
   • Status code: 200
   • Response body contains: "version" (confirms a valid OpenNMS response, not an nginx error page)
7. Save the monitor

When this monitor goes down, OpenNMS is unreachable — no new monitoring data is being ingested and no management actions can be taken via the UI.

Step 3: Monitor the PostgreSQL backend

OpenNMS stores all node data, events, outages, alarms, and performance data in PostgreSQL. A database outage means OpenNMS stops writing event data even if the web UI stays responsive briefly.

Monitor the PostgreSQL port directly:

1. New Monitor → TCP Port
2. Hostname: your-postgres-host
3. Port: 5432
4. Save

For a deeper check, add an HTTP monitor that queries the OpenNMS nodes REST API (requires a DB round-trip):

1. New Monitor → HTTP / HTTPS
2. URL: http://your-opennms:8980/opennms/rest/nodes?limit=1
3. Basic Auth with monitoring user
4. Expected status: 200
5. Response body contains: "node" or "totalCount"

If the PostgreSQL connection pool is exhausted, this endpoint will return 500 or time out — Vigilmon will alert immediately.

Step 4: Monitor the poller daemon health via REST

OpenNMS's poller daemon tests services on monitored nodes. If the poller falls behind schedule, new outages go undetected. You can check active outage counts via the REST API to verify polling is keeping up:

# Check current unacknowledged outages count — should be consistent, not growing unboundedly
curl -u monitor:password \
  'http://your-opennms:8980/opennms/rest/outages?limit=0' \
  | jq '.totalCount'

Set up a heartbeat monitor that a script updates every 5 minutes:

#!/bin/bash
# /usr/local/bin/check-opennms-poller.sh
# Runs on the OpenNMS server; checks poller thread status via REST

OPENNMS_URL="http://localhost:8980"
OPENNMS_USER="monitor"
OPENNMS_PASS="your-password"
PING_URL="https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID"

# Check that the REST API responds and returns a valid nodes count
node_count=$(curl -fs -u "${OPENNMS_USER}:${OPENNMS_PASS}" \
  "${OPENNMS_URL}/opennms/rest/nodes?limit=0" | \
  python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('totalCount',0))")

if [ -z "$node_count" ] || [ "$node_count" -eq 0 ]; then
  echo "ERROR: OpenNMS REST API returned 0 nodes — poller may be down"
  exit 1
fi

# Ping Vigilmon heartbeat on success
curl -fsS "$PING_URL" > /dev/null 2>&1
echo "OK: OpenNMS monitoring ${node_count} nodes"

Add to cron:

*/5 * * * * /usr/local/bin/check-opennms-poller.sh >> /var/log/opennms-poller-check.log 2>&1

Step 5: Monitor OpenNMS performance collection (collectd)

OpenNMS's collectd daemon gathers SNMP counters, JMX metrics, and response times from monitored nodes. Collection failures are silent — the UI continues working but RRD/JRobin files stop updating.

Check collection health via the resource API:

# Returns performance data resources — a live response confirms collectd is writing
curl -u monitor:password \
  'http://your-opennms:8980/opennms/rest/resources?limit=1' | jq '.count'

For a more targeted check, query a specific known node's resource and verify the last update time is recent:

#!/bin/bash
# /usr/local/bin/check-opennms-collectd.sh

OPENNMS_URL="http://localhost:8980"
OPENNMS_USER="monitor"
OPENNMS_PASS="your-password"
PING_URL="https://vigilmon.online/heartbeat/YOUR_COLLECTD_HEARTBEAT_ID"
MAX_AGE_MINUTES=15

# Get the last-updated timestamp for a known node's SNMP resource
last_update=$(curl -fs -u "${OPENNMS_USER}:${OPENNMS_PASS}" \
  "${OPENNMS_URL}/opennms/rest/resources/node%5B1%5D.nodeSnmp%5B%5D" | \
  python3 -c "
import json, sys, time
d = json.load(sys.stdin)
attrs = d.get('children', {}).get('resource', [{}])
print(attrs[0].get('link', 'unknown'))
")

if [ "$last_update" = "unknown" ]; then
  echo "ERROR: Could not retrieve collectd resource data"
  exit 1
fi

curl -fsS "$PING_URL" > /dev/null 2>&1
echo "OK: collectd resources accessible"

Step 6: Monitor REST API response time

OpenNMS's REST API is used by integrations, Grafana dashboards, and external tooling. Slow REST responses indicate database query degradation or JVM garbage collection pressure.

In Vigilmon, add a response time alert to your existing REST API monitor:

1. Open your OpenNMS REST API monitor
2. Under Performance, set a response time alert threshold of 5 seconds
3. Save

A response time alert fires before the API becomes fully unresponsive — giving you time to investigate JVM heap pressure or slow PostgreSQL queries before the system degrades further.

Step 7: Configure alerting for OpenNMS incidents

1. Go to Alert Channels → Add Channel → Webhook
2. Set up Slack or PagerDuty
3. Assign the channel to all your OpenNMS monitors

Recommended alert policy:

• OpenNMS REST API down → page immediately (P1 — monitoring platform unreachable)
• PostgreSQL TCP down → page immediately (P1 — event data loss)
• Poller heartbeat missed → page immediately (P1 — outages going undetected)
• collectd heartbeat missed → warn (P2 — performance data stale)
• REST API response time >5s → warn (P2 — investigate JVM/DB)

Step 8: Create a status page for your monitoring infrastructure

A dedicated status page for your observability stack lets your NOC team know at a glance whether OpenNMS itself is healthy:

1. Go to Status Pages → New Status Page
2. Name it "Monitoring Infrastructure"
3. Add:
   • OpenNMS REST API (HTTP monitor)
   • PostgreSQL (TCP monitor)
   • Poller daemon (Heartbeat)
   • collectd (Heartbeat)
4. Publish the page

Putting it all together

| Monitor | Type | Alert level | What it catches | |---------|------|-------------|-----------------| | http://opennms:8980/opennms/rest/info | HTTP | P1 | OpenNMS web UI / REST API down | | http://opennms:8980/opennms/rest/nodes?limit=1 | HTTP | P1 | DB connection failure | | postgres:5432 | TCP | P1 | PostgreSQL unreachable | | Poller daemon script | Heartbeat | P1 | Poller behind / not running | | collectd resource script | Heartbeat | P2 | Collection stopped | | OpenNMS REST latency | HTTP (response time) | P2 | JVM / DB performance degradation |

What's next

• Event bus depth — query opennms/rest/events?limit=0 periodically and alert if the total event count grows faster than your baseline (a sign of trap flood or queue buildup)
• JVM heap monitoring — OpenNMS exposes JMX metrics; connect Vigilmon's TCP monitor to port 18980 (OpenNMS JMX) and add a heartbeat script that checks java.lang:type=Memory HeapMemoryUsage
• Alarm count trending — a sudden spike in unacknowledged critical alarms often indicates a discovery storm or misconfigured threshold; track via opennms/rest/alarms?severity=CRITICAL&limit=0

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.