Monitoring Netdisco with Vigilmon

Netdisco is the gold standard for answering questions like "what device is connected to switch X port Y?" and "where is this MAC address in the network?" — but when its backend daemon silently dies or PostgreSQL slows down, you lose that visibility without any notification. Vigilmon closes that gap, giving you uptime alerts for the web UI, backend daemon health checks, discovery job watchdogs, and database performance monitoring across your entire Netdisco deployment.

What You'll Set Up

• HTTP uptime monitor for the Netdisco web UI and REST API (port 5000)
• Cron heartbeat to detect backend daemon (netdisco-backend) crashes
• Discovery job failure watchdog via the Netdisco API
• Device inventory freshness alert for stale SNMP poll data
• PostgreSQL database health monitor
• MAC address tracking freshness watchdog
• Web UI response time monitoring

Prerequisites

• Netdisco installed and running (web UI accessible on port 5000)
• netdisco-backend daemon running as a systemd service or similar
• A free Vigilmon account

Step 1: Monitor the Netdisco Web UI and REST API

The Netdisco web UI and REST API share port 5000. When this goes down, your NOC team loses all access to network topology and device inventory. Add a health probe using the built-in ping endpoint:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter http://your-netdisco-host:5000/netdisco/api/v1/ping.
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Under Response body, set Contains to pong (the API returns {"ping":"pong"}).
7. Click Save.

This endpoint is Netdisco's official health probe — a 200 response confirms the PSGI application server (Starman/Twiggy) is up and the application is initialised.

Step 2: Watchdog the Netdisco Backend Daemon

netdisco-backend is the worker process that runs all SNMP polling and discovery jobs. If it crashes, no device data is ever refreshed — but the web UI stays up, giving you a false sense of health. Use a cron heartbeat to detect a stopped daemon:

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 5 minutes.
3. Copy the heartbeat URL (e.g. https://vigilmon.online/heartbeat/abc123).

Create a wrapper script that pings Vigilmon only when the daemon is confirmed running:

#!/bin/bash
# /usr/local/bin/netdisco-backend-watchdog.sh
if systemctl is-active --quiet netdisco-backend; then
  curl -s https://vigilmon.online/heartbeat/abc123 > /dev/null
fi

Schedule it with cron every 5 minutes:

# crontab -e
*/5 * * * * /usr/local/bin/netdisco-backend-watchdog.sh

If the daemon crashes and the cron job stops pinging, Vigilmon alerts within 5 minutes.

Step 3: Monitor Discovery Job Health

Netdisco schedules SNMP discovery jobs against every network device. A spike in discovery failures indicates SNMP credential issues, unreachable devices, or backend overload. Query the Netdisco REST API to track this:

#!/bin/bash
# /usr/local/bin/netdisco-discovery-check.sh
NETDISCO_URL="http://localhost:5000"
API_KEY="your-netdisco-api-key"
HEARTBEAT_URL="https://vigilmon.online/heartbeat/your-discovery-heartbeat-id"

# Fetch recent jobs — alert if failure rate > 10%
RESPONSE=$(curl -s -H "Authorization: Bearer $API_KEY" \
  "$NETDISCO_URL/netdisco/api/v1/jobs?status=error&since=1h" 2>/dev/null)

ERROR_COUNT=$(echo "$RESPONSE" | python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d.get('list',[])))" 2>/dev/null || echo "0")

if [ "$ERROR_COUNT" -lt 10 ]; then
  curl -s "$HEARTBEAT_URL" > /dev/null
fi

Schedule this check every 15 minutes:

*/15 * * * * /usr/local/bin/netdisco-discovery-check.sh

Set the heartbeat interval to 20 minutes so Vigilmon alerts if the check stops running or if too many discoveries are failing.

Step 4: Alert on Stale Device Inventory

Netdisco polls each device via SNMP on a schedule. Devices that haven't been polled in over 24 hours mean your topology data is stale — you may be looking at an out-of-date picture of the network. Add a heartbeat that only pings if all devices have been polled recently:

#!/bin/bash
# /usr/local/bin/netdisco-freshness-check.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/your-freshness-heartbeat-id"

# Check for devices with last_discover older than 24 hours
STALE=$(psql -U netdisco -d netdisco -t -c \
  "SELECT COUNT(*) FROM device WHERE last_discover < NOW() - INTERVAL '24 hours';" 2>/dev/null | tr -d ' ')

if [ "$STALE" = "0" ]; then
  curl -s "$HEARTBEAT_URL" > /dev/null
fi

Set the cron interval to 60 minutes and the heartbeat interval to 90 minutes. If any devices go stale, the ping stops and Vigilmon alerts.

Step 5: Monitor PostgreSQL Database Health

All of Netdisco's device inventory, topology, and interface data lives in PostgreSQL. A connectivity loss or slow query storm means the web UI will time out on every page. Add a database reachability check:

1. In Vigilmon, click Add Monitor → TCP Port.
2. Enter your PostgreSQL host and port 5432.
3. Set Check interval to 1 minute.
4. Click Save.

For query latency, add a cron heartbeat that only pings when the DB responds quickly:

#!/bin/bash
# /usr/local/bin/netdisco-db-latency.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/your-db-latency-heartbeat-id"

# Run a lightweight query and measure response time
START=$(date +%s%N)
psql -U netdisco -d netdisco -c "SELECT 1;" > /dev/null 2>&1
END=$(date +%s%N)
LATENCY_MS=$(( (END - START) / 1000000 ))

# Alert if query takes more than 500ms
if [ "$LATENCY_MS" -lt 500 ]; then
  curl -s "$HEARTBEAT_URL" > /dev/null
fi

Schedule every 5 minutes with a 10-minute heartbeat interval.

Step 6: Monitor MAC Address Tracking Freshness

Netdisco tracks MAC-to-port-to-IP mappings — the most operationally critical data for "where is this device connected?" queries. Add a staleness check:

#!/bin/bash
# /usr/local/bin/netdisco-mac-freshness.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/your-mac-freshness-heartbeat-id"

# Check that MAC tracking ran within the last 6 hours
LAST_MAC=$(psql -U netdisco -d netdisco -t -c \
  "SELECT EXTRACT(EPOCH FROM (NOW() - MAX(time_last))) FROM node;" 2>/dev/null | tr -d ' ')

# LAST_MAC is seconds since last MAC update; alert if > 21600 (6 hours)
if [ -n "$LAST_MAC" ] && (( $(echo "$LAST_MAC < 21600" | bc -l) )); then
  curl -s "$HEARTBEAT_URL" > /dev/null
fi

Schedule every 30 minutes with a 45-minute heartbeat interval.

Step 7: Monitor Web UI Response Time

Netdisco topology and device search queries hit PostgreSQL with complex joins. Slow UI response time is an early indicator of database performance degradation before it becomes a full outage.

1. Open the HTTP monitor you created in Step 1.
2. Under Performance, enable Alert on slow response.
3. Set Alert when response time exceeds 5000 ms (Netdisco topology pages can legitimately take 2–3s on large networks).
4. Click Save.

For the device search endpoint specifically, add a second monitor targeting a lightweight search query:

http://your-netdisco-host:5000/netdisco/search?q=test&tab=node

Set the response time threshold to 3000 ms. This proxy check for search performance will alert you to index degradation before it impacts real user queries.

Step 8: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add your preferred channel — email, Slack, PagerDuty, or a webhook to your NMS ticketing system.
2. For the web UI monitor, set Consecutive failures before alert to 2 — transient PSGI hiccups can cause a single probe to fail.
3. For the backend daemon heartbeat, set Consecutive failures before alert to 1 — a stopped daemon is always a real incident.
4. For discovery job and inventory freshness heartbeats, set Consecutive failures before alert to 2 to absorb job queue delays.

Recommended Alert Thresholds

| Monitor | Alert After | Notes | |---|---|---| | Web UI (port 5000) | 2 failures | PSGI restart takes ~10s | | Backend daemon heartbeat | 1 failure | No false positives expected | | Discovery job health | 2 failures | Queue may back up briefly | | Device inventory freshness | 1 failure | 24h threshold is already conservative | | MAC tracking freshness | 1 failure | 6h is a wide window | | PostgreSQL TCP | 2 failures | Transient connection errors |

Conclusion

With these monitors in place, Vigilmon gives you full observability across the Netdisco stack: web UI availability, backend daemon liveness, discovery job health, device inventory freshness, and database performance. When your NOC team asks "why is the topology map out of date?", you'll already have an alert in your inbox telling you exactly which layer failed.

For more self-hosted monitoring guides, visit vigilmon.online.