Monitoring Limnoria IRC Bot with Vigilmon

Limnoria has been keeping IRC channels running since the Supybot days — factoid databases, RSS feeds, trivia bots, and channel management all live in a single Python process. When that process crashes or its IRC connection silently drops, your channel loses all automated functions without any visible signal. Vigilmon gives you process health monitoring, IRC connection heartbeats, database checks, and web server availability so you catch failures the moment they happen.

What You'll Set Up

• Cron heartbeat for the Limnoria Python process
• IRC connection health via a command-response heartbeat
• HTTP monitor for the Limnoria web server (if enabled)
• Database connectivity check (SQLite or PostgreSQL)
• RSS feed fetch health check
• Plugin availability monitoring via log scanning

Prerequisites

• Limnoria running (via supybot command, systemd, or Docker)
• A free Vigilmon account

Step 1: Monitor the Limnoria Process with a Heartbeat

A process heartbeat is the most reliable signal that Limnoria is alive. Python exceptions that aren't caught properly can silently kill the process without leaving a port open:

1. Log in to vigilmon.online and click Add Monitor → Cron Job / Heartbeat.
2. Set Expected interval to 5 minutes.
3. Copy the heartbeat URL (e.g., https://vigilmon.online/ping/your-heartbeat-id).

Add a cron job:

# /etc/cron.d/limnoria-heartbeat
*/5 * * * * limnoria pgrep -f "supybot.*conf/supybot.conf" > /dev/null && \
  curl -fsS --retry 3 https://vigilmon.online/ping/your-heartbeat-id

This only pings when Limnoria's Python process is actually running. Adjust the pgrep pattern to match how you launch Limnoria (the config file path is a reliable distinguishing pattern).

For automatic restart, ensure Limnoria runs under systemd:

[Unit]
Description=Limnoria IRC Bot
After=network.target

[Service]
User=limnoria
ExecStart=/usr/local/bin/supybot /home/limnoria/conf/supybot.conf
Restart=on-failure
RestartSec=15

[Install]
WantedBy=multi-user.target

Step 2: Monitor IRC Network Connection Health

Limnoria can connect to multiple IRC networks simultaneously. A dropped connection may not restart immediately, leaving channels unmonitored. Create a heartbeat that a Limnoria plugin pings from inside the bot:

Option A: Using Limnoria's Scheduler plugin

Add a Limnoria command that pings your heartbeat URL on a schedule. In your bot's DM, run:

owner scheduler add 300 shell curl -fsS https://vigilmon.online/ping/your-irc-heartbeat-id

This uses Limnoria's built-in scheduler to send a heartbeat every 300 seconds (5 minutes) via the Shell plugin. The ping only fires when the bot is connected and processing commands — a missed ping means either the bot is down or the IRC connection has dropped.

Option B: Write a custom Limnoria plugin

# plugins/VigilmonHeartbeat/__init__.py
from supybot.commands import wrap
import supybot.callbacks as callbacks
import supybot.schedule as schedule
import urllib.request

class VigilmonHeartbeat(callbacks.Plugin):
    """Pings Vigilmon to confirm the bot is alive and connected."""
    PING_URL = "https://vigilmon.online/ping/your-irc-heartbeat-id"

    def __init__(self, irc):
        super().__init__(irc)
        schedule.addPeriodicEvent(self._ping, 300, 'vigilmon_heartbeat')

    def _ping(self):
        try:
            urllib.request.urlopen(self.PING_URL, timeout=5)
        except Exception:
            pass  # Don't let network errors crash the plugin

    def die(self):
        schedule.removeEvent('vigilmon_heartbeat')
        super().die()

Class = VigilmonHeartbeat

Load the plugin with load VigilmonHeartbeat in the bot console.

Step 3: Monitor the Limnoria Web Server

Limnoria includes an optional built-in HTTP server (enabled via the HttpServer plugin) for web-based plugin interfaces like the web configuration panel. If this is enabled:

1. Click Add Monitor → HTTP / HTTPS.
2. Set the URL to http://your-server:8080 (or the port configured in supybot.servers.http.port).
3. Set Expected HTTP status to 200.
4. Set Check interval to 2 minutes.
5. Click Save.

If you reverse-proxy the Limnoria HTTP server through nginx or Caddy, monitor the public URL instead so the check validates the full stack.

Step 4: Monitor the Database

Limnoria stores factoids, user accounts, channel data, and plugin state in SQLite (default) or PostgreSQL. A corrupted or inaccessible database degrades most plugins silently.

SQLite monitoring

Add a cron heartbeat that validates the SQLite database:

1. Create a Vigilmon Cron Job / Heartbeat with Expected interval 10 minutes.
2. Copy the heartbeat URL.

# /etc/cron.d/limnoria-db-check
*/10 * * * * limnoria \
  sqlite3 /home/limnoria/data/factoids.db "SELECT count(*) FROM factoids;" > /dev/null 2>&1 && \
  curl -fsS https://vigilmon.online/ping/your-db-heartbeat-id

Replace the path and table name with your actual database file and a table that's always present. The query failing means the database is corrupt or the file is missing.

PostgreSQL monitoring

# /etc/cron.d/limnoria-pg-check
*/10 * * * * limnoria \
  psql -U limnoria -d limnoriadb -c "SELECT 1;" > /dev/null 2>&1 && \
  curl -fsS https://vigilmon.online/ping/your-db-heartbeat-id

Step 5: Monitor RSS Feed Fetch Health

If you use Limnoria's RSS or Announce plugin to relay RSS feeds into channels, broken feed URLs silently stop delivering news. Monitor the external feed URLs directly:

1. For each RSS feed URL in your Limnoria config, click Add Monitor → HTTP / HTTPS.
2. Set the URL to the feed endpoint.
3. Set Expected HTTP status to 200.
4. Set Check interval to 10 minutes.
5. Click Save.

Limnoria polls feeds on its own schedule, but a Vigilmon monitor will catch a permanently broken feed URL before your users notice the silence.

Step 6: Monitor Plugin Load Health via Log Scanning

Limnoria logs plugin load failures to its log file. Failed plugins disable features silently — a Critical or Error level message about a plugin import is a signal worth monitoring:

1. Create a Vigilmon Cron Job / Heartbeat with Expected interval 30 minutes.
2. Copy the heartbeat URL.

# /etc/cron.d/limnoria-plugin-check
*/30 * * * * limnoria \
  ERRORS=$(grep -c "CRITICAL\|load.*Error\|import.*Error" /home/limnoria/logs/messages.log 2>/dev/null); \
  [ "${ERRORS:-0}" -eq 0 ] && curl -fsS https://vigilmon.online/ping/your-plugin-heartbeat-id

This only pings Vigilmon when there are no critical errors in the log. A missed ping triggers an alert so you can investigate the plugin failure.

Step 7: Monitor Configuration File Integrity

Limnoria's conf/supybot.conf controls all bot behavior — an accidental overwrite or permissions change can cause unpredictable behavior on the next restart. Track file integrity with a simple hash check:

#!/bin/bash
# /etc/cron.d/limnoria-config-check
CONF="/home/limnoria/conf/supybot.conf"
HASH_FILE="/home/limnoria/.supybot-conf-hash"

CURRENT=$(sha256sum "$CONF" | awk '{print $1}')

if [ ! -f "$HASH_FILE" ]; then
  echo "$CURRENT" > "$HASH_FILE"
  exit 0
fi

STORED=$(cat "$HASH_FILE")
if [ "$CURRENT" != "$STORED" ]; then
  logger "Limnoria config changed: $CONF"
  # Optionally update the hash and alert
  echo "$CURRENT" > "$HASH_FILE"
fi

If config changes should be tracked rather than alerting, commit supybot.conf to a git repository and use git diff in the cron check.

Step 8: Configure Alerts

1. Go to Settings → Alerts in Vigilmon.
2. Add a Slack webhook or email alert channel.
3. Set the failure threshold — 2 consecutive failures before alerting prevents transient noise.
4. Apply to all Limnoria monitors.

Recommended alert priority:

| Monitor | Priority | Impact | |---|---|---| | Process heartbeat | Critical | All bot functions stop | | IRC connection heartbeat | Critical | Bot silent in channels | | Database connectivity | High | Factoids, user data inaccessible | | Web server | Medium | Admin UI only | | Plugin load check | Medium | Feature degradation | | RSS feed URLs | Low | Content delivery only |

What to Watch

| Monitor | Type | Check | Alert threshold | |---|---|---|---| | Limnoria process | Heartbeat | Cron ping every 5 min | 2 missed | | IRC connection | Heartbeat | Bot-generated ping every 5 min | 2 missed | | HTTP web server | HTTP | Port 8080 responds 200 | 2 failures | | SQLite/PostgreSQL DB | Heartbeat | Query succeeds every 10 min | 2 missed | | RSS feed URLs | HTTP | Feed URL responds 200 | 3 failures | | Plugin load log | Heartbeat | No errors in log | 2 missed |

Conclusion

Limnoria's longevity as an IRC bot platform means your setup could be running for years — and years of uptime means subtle failures often go undetected until a user notices the bot isn't responding. Vigilmon's free tier covers all the monitors configured here, giving you process health, connection health, and database integrity monitoring without extra infrastructure.

Sign up for Vigilmon free →