Monitoring Trac with Vigilmon

Trac is a battle-tested open-source project management environment combining a wiki, issue tracker, and version control browser in a single Python/WSGI application. Development teams use it to manage tickets, document their codebase, and browse Subversion or Git history — but Trac's WSGI stack, SQLite/PostgreSQL/MySQL database, and plugin ecosystem can fail silently in ways that leave developers unable to file bugs or access the wiki. Vigilmon gives you external uptime checks for the WSGI server, ticket system, repository browser, search indexer, and email notifications so you catch failures before your developers hit a 500 error mid-sprint.

What You'll Set Up

• WSGI server availability check
• Wiki page rendering service health
• Ticket system API health
• Version control repository browser probe
• Database connectivity check
• Search indexing service health
• Email notification delivery heartbeat
• Plugin service health
• Timeline feed endpoint monitoring

Prerequisites

• Trac running via Apache mod_wsgi, gunicorn, uwsgi, or tracd (default port 80 or 8000)
• Access to the Trac environment directory for probe scripts
• A free Vigilmon account

Step 1: Monitor the Trac WSGI Server

Trac's login page exercises the WSGI stack, database connectivity, and template rendering in a single request. It is the most comprehensive single-endpoint health check available.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your Trac URL: https://trac.yourdomain.com/login
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Set Expected response body contains to Trac.
7. Click Save.

The login page renders even without valid credentials, returning 200 with the login form. A 500 or blank response indicates a WSGI crash, database failure, or broken Trac plugin.

If Trac is behind a reverse proxy (nginx, Apache), also add a TCP port monitor:

1. Click Add Monitor → TCP Port.
2. Enter trac.yourdomain.com:80 (or :443).
3. Set Check interval to 1 minute.

Step 2: Monitor Wiki Page Rendering

The wiki is Trac's core feature. A broken wiki renderer (caused by a faulty plugin, missing macro, or database issue) shows a 500 error on wiki pages but may leave the ticket system working normally.

Add a separate HTTP monitor for the wiki front page:

1. Add a Vigilmon HTTP monitor for:

   https://trac.yourdomain.com/wiki/WikiStart
   

2. Set Expected HTTP status to 200.
3. Set Expected response body contains to WikiStart (the page title).
4. Set Check interval to 2 minutes.

If your wiki front page requires login, create a read-only monitor user in Trac and use HTTP Basic Auth in the monitor URL:

https://monitor:probe_pass@trac.yourdomain.com/wiki/WikiStart

Step 3: Monitor the Ticket System

Trac's ticket system is the most business-critical component for development teams. Monitor the ticket query endpoint, which exercises ticket storage and rendering:

1. Add an HTTP monitor for:

   https://trac.yourdomain.com/query
   

2. Set Expected HTTP status to 200.
3. Set Expected response body contains to Query.
4. Set Check interval to 1 minute.

Also monitor ticket creation access with the new ticket form:

https://trac.yourdomain.com/newticket

Expected status: 200, expected body contains: Create New Ticket. If the ticket database table is corrupted or locked, the new ticket form returns a database error while the wiki may still render normally.

Step 4: Monitor the Repository Browser

Trac's repository browser (/browser) renders version control history from Subversion or Git. A broken VCS integration causes 500 errors in the browser while leaving tickets and wiki pages unaffected.

Add an HTTP monitor for the repository browser root:

https://trac.yourdomain.com/browser

Expected status: 200, expected body contains: Repository.

For installations with multiple repositories, add a monitor for each:

https://trac.yourdomain.com/browser/repo-name

If the repository path is unreachable (unmounted NFS, missing Subversion server, broken Git bare repo), these monitors fire independently from the WSGI availability check, letting you pinpoint the VCS layer specifically.

Step 5: Monitor Database Connectivity

Trac supports SQLite, PostgreSQL, and MySQL. Database failures cause 500 errors across all Trac pages simultaneously. For SQLite (the default), disk-full or permission issues are the most common failure modes. For PostgreSQL/MySQL, connection pool exhaustion or network failures are more likely.

Create a lightweight database probe using trac-admin:

#!/bin/bash
# /opt/scripts/trac-db-probe.sh
DB_RESULT=$(trac-admin /path/to/trac/env session list 2>&1 | head -1)

if echo "$DB_RESULT" | grep -q "SID"; then
  curl -s https://vigilmon.online/heartbeat/TRAC_DB_KEY
else
  echo "Database probe failed: $DB_RESULT"
fi

trac-admin session list executes a read query against the Trac session table — a reliable database connectivity check. Set the Vigilmon heartbeat interval to 5 minutes and schedule the script every 2 minutes.

For PostgreSQL or MySQL backends, also add a TCP port monitor:

postgresql.internal:5432

or

mysql.internal:3306

Step 6: Monitor Search Indexing

Trac's built-in search indexes tickets, wiki pages, and changesets. The Whoosh-based full-text index can fall out of sync with the database when the index directory runs out of disk space or becomes corrupted.

Add an HTTP monitor for the search endpoint:

https://trac.yourdomain.com/search?q=test&noquickjump=1

Expected status: 200, expected body contains: Search.

For deeper index health, add a probe script that verifies the index is writable:

#!/usr/bin/env python3
# /opt/scripts/trac-search-probe.py
import os
import sys

index_dir = "/path/to/trac/env/index"

if not os.path.exists(index_dir):
    print("Search index directory missing")
    sys.exit(1)

# Check index directory is writable
if not os.access(index_dir, os.W_OK):
    print("Search index not writable")
    sys.exit(1)

print("ok")

Wrap in a cron heartbeat and set the Vigilmon interval to 30 minutes.

Step 7: Monitor Email Notification Delivery

Trac sends email notifications for ticket changes, wiki edits, and new registrations. Silent email failures mean developers miss ticket updates and managers miss escalations.

Trac's email notifications are synchronous (sent during the request), so a broken SMTP configuration causes slow requests and partial 500 errors rather than a visible queue.

Add a probe that verifies SMTP connectivity:

#!/bin/bash
# Test SMTP connectivity to Trac's configured mail server
SMTP_HOST="mail.yourdomain.com"
SMTP_PORT=25

if nc -zw5 "$SMTP_HOST" "$SMTP_PORT" 2>/dev/null; then
  curl -s https://vigilmon.online/heartbeat/TRAC_MAIL_KEY
else
  echo "SMTP unreachable: $SMTP_HOST:$SMTP_PORT"
fi

Set the Vigilmon heartbeat interval to 30 minutes and schedule the script every 15 minutes.

You can also add a TCP port monitor directly in Vigilmon for your SMTP server:

1. Click Add Monitor → TCP Port.
2. Enter mail.yourdomain.com:25 (or 587 for submission).
3. Set Check interval to 5 minutes.

Step 8: Monitor Plugin Service Health

Trac's plugin system is powerful but fragile — a misbehaving plugin can break specific pages while leaving the rest of Trac functional. The most common failure mode is a plugin that raises an exception on component initialization.

Add monitors for pages that exercise your most critical plugins:

# Example: monitor the AccountManager plugin login page
https://trac.yourdomain.com/account

# Example: monitor the MasterTickets dependency graph
https://trac.yourdomain.com/mastertickets

# Example: monitor the Agilo sprint board
https://trac.yourdomain.com/sprints

Set Expected HTTP status to 200 and Expected response body contains to a key phrase from each page. When a plugin breaks, that specific monitor fires while the core WSGI monitor stays green, letting you isolate the failing component immediately.

Step 9: Monitor the Timeline Feed

Trac's timeline feed (/timeline) aggregates recent activity across tickets, wiki, and repository. It is commonly consumed by CI systems, dashboards, and RSS clients. A broken timeline feed often indicates database join performance issues before they become user-visible slowdowns.

1. Add an HTTP monitor for:

   https://trac.yourdomain.com/timeline?format=rss
   

2. Set Expected HTTP status to 200.
3. Set Expected response body contains to <rss.
4. Set Response time alert to 5000 ms — the timeline query joins tickets, wiki pages, and changesets; a response over 5 seconds signals query performance issues or index maintenance blocking.

Step 10: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
2. Set Consecutive failures before alert to 2 on all HTTP monitors — WSGI worker restarts are brief and should not generate false positives.
3. For the timeline feed, set Response time alert to 5000 ms to catch slow database queries early.
4. For the repository browser, set a 2000 ms response time alert — slow VCS browsing typically means the underlying repository is on a slow or unmounted volume.
5. Use Maintenance windows in Vigilmon during Trac upgrades or database vacuums.

Summary

| Monitor | Target | What It Catches | |---|---|---| | WSGI server | /login | WSGI crash, DB failure, plugin error | | Wiki rendering | /wiki/WikiStart | Broken wiki renderer | | Ticket system | /query | Ticket DB unavailable | | Repository browser | /browser | VCS integration broken | | Database (DB probe) | Heartbeat URL | SQLite lock, PG/MySQL failure | | Search index | /search?q=test | Whoosh index errors | | Email notifications | Heartbeat URL | SMTP unreachable | | Plugin pages | Per-plugin URLs | Plugin-specific failures | | Timeline feed | /timeline?format=rss | Slow queries, feed generation error |

Trac may be older than many modern issue trackers, but the teams that rely on it often depend on it absolutely — it holds the entire history of a project's bugs, decisions, and documentation. With Vigilmon watching every layer from WSGI to database to SMTP, you get external uptime verification that catches failures at each layer independently, so a broken plugin does not look the same as a crashed WSGI server or a full disk on the SQLite volume.