Monitoring Paperless-AI with Vigilmon

Paperless-AI is an open-source extension for Paperless-ngx that adds automatic document classification, intelligent tagging, and content analysis using local LLMs (via Ollama) or cloud APIs (OpenAI-compatible endpoints). It runs as a Python/FastAPI service alongside your existing Paperless-ngx installation. When you self-host this AI pipeline, you own the reliability of every layer — the FastAPI process, the Paperless-ngx API it reads from, the LLM backend it sends documents to, the Celery workers that process the queue, and the Redis cache that ties it together. Vigilmon gives you uptime monitoring and alerting across every component.

What You'll Set Up

• HTTP uptime monitor for the Paperless-AI FastAPI service
• Paperless-ngx API connectivity check (the document source)
• LLM backend connectivity check (Ollama or OpenAI-compatible endpoint)
• Celery/asyncio worker heartbeat (document classification task queue)
• Redis cache and queue TCP connectivity check
• Document tagging pipeline health check via the /api/tag endpoint
• Document classification API response-time monitor (/api/classify)
• Cron heartbeat for scheduled document batch processing jobs
• New-document detection hook health check
• SSL/TLS certificate expiry alert

Prerequisites

• Paperless-AI deployed and running alongside Paperless-ngx
• An Ollama instance or OpenAI-compatible API configured as the LLM backend
• Redis running as the task queue broker
• A free Vigilmon account

Step 1: Monitor the Paperless-AI FastAPI Service

The FastAPI service is the heart of Paperless-AI. Monitor its liveness endpoint first.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your Paperless-AI URL: https://paperless-ai.yourdomain.com (or http://localhost:8080 if running locally).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

FastAPI automatically generates a /docs page at startup — if the service is running, the root or docs URL returns 200. If you have a dedicated /health endpoint, use that instead.

Step 2: Add a Structured Health Endpoint

If Paperless-AI doesn't expose a /health endpoint, add one that checks its most critical dependencies:

# main.py or router
from fastapi import APIRouter
import httpx
import redis as redis_lib

router = APIRouter()

@router.get("/health")
async def health():
    checks = {}

    # Check Paperless-ngx API
    try:
        async with httpx.AsyncClient() as client:
            r = await client.get(
                "http://paperless-ngx:8000/api/",
                headers={"Authorization": "Token YOUR_PAPERLESS_TOKEN"},
                timeout=5,
            )
        checks["paperless_ngx"] = "ok" if r.status_code == 200 else "error"
    except Exception:
        checks["paperless_ngx"] = "unreachable"

    # Check Redis
    try:
        r = redis_lib.Redis(host="redis", port=6379)
        r.ping()
        checks["redis"] = "ok"
    except Exception:
        checks["redis"] = "unreachable"

    status_code = 200 if all(v == "ok" for v in checks.values()) else 503
    return {"status": "ok" if status_code == 200 else "degraded", **checks}

Add a Vigilmon monitor for https://paperless-ai.yourdomain.com/health with expected status 200. A 503 response body tells you which dependency failed.

Step 3: Monitor Paperless-ngx API Connectivity

Paperless-AI reads documents from Paperless-ngx via its REST API. If the Paperless-ngx API goes down, Paperless-AI has nothing to classify — but its own health endpoint may still return 200.

Add a dedicated monitor for the Paperless-ngx API:

1. Click Add Monitor → HTTP / HTTPS.
2. Enter https://paperless.yourdomain.com/api/.
3. Add the header Authorization: Token YOUR_PAPERLESS_TOKEN.
4. Set Expected status to 200.
5. Label it Paperless-ngx API.

This confirms that the source document store is reachable independently of the AI service.

Step 4: Monitor the LLM Backend Connectivity

Paperless-AI sends documents to an LLM for classification. Whether you use Ollama locally or an OpenAI-compatible API, the inference endpoint must be reachable.

Ollama (local):

1. Add a TCP Port monitor to your Ollama host, port 11434.
2. Or add an HTTP monitor for http://your-ollama-host:11434/api/tags with expected status 200 — this lists loaded models and confirms the service is running.

OpenAI-compatible API:

1. Add an HTTP monitor for https://api.openai.com (or your proxy endpoint).
2. Set Expected status to 200 or 401 — a 401 from the real OpenAI endpoint means it's reachable, even if your key is not presented.
3. Label it LLM backend.

An LLM backend outage causes the classification pipeline to queue up silently — documents pile up untagged until the backend recovers.

Step 5: Monitor the Celery/Asyncio Worker

Paperless-AI processes documents asynchronously via Celery workers (or asyncio task queues, depending on the version). If workers die, documents are detected but never classified.

Add a heartbeat task to your Celery beat schedule:

# celery_config.py
from celery.schedules import crontab

CELERYBEAT_SCHEDULE = {
    "vigilmon-heartbeat": {
        "task": "paperless_ai.tasks.vigilmon_ping",
        "schedule": crontab(minute="*/5"),
    },
}

# tasks.py
import httpx
from celery import shared_task

@shared_task
def vigilmon_ping():
    httpx.get("https://vigilmon.online/heartbeat/YOUR_WORKER_ID", timeout=5)

In Vigilmon:

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected interval to 5 minutes.
3. Copy the heartbeat URL into YOUR_WORKER_ID above.

If all workers go down, the heartbeat task never executes and Vigilmon alerts after 5 minutes.

Step 6: Check Redis Cache and Queue Connectivity

Redis is Paperless-AI's task broker and cache. A Redis failure silently stops all worker task dispatch.

1. Click Add Monitor → TCP Port.
2. Enter your Redis host and port (default 6379).
3. Set Check interval to 1 minute.
4. Click Save.

Correlate Redis failures with worker heartbeat misses — both failing together confirms the broker is the root cause.

Step 7: Monitor the Document Tagging Pipeline

The /api/tag endpoint applies AI-generated tags to documents. Monitor it with a test request:

1. Click Add Monitor → HTTP / HTTPS.
2. Enter https://paperless-ai.yourdomain.com/api/tag.
3. Set Method to POST.
4. Add the header Content-Type: application/json.
5. Set Request body to:

{"document_id": 1, "dry_run": true}

6. Set Expected status to 200.
7. Enable Response time alerting at 10000 ms — LLM inference is slow; alert only if it takes more than 10 seconds for a simple document.

The dry_run: true flag (if supported by your Paperless-AI version) runs the classification without writing changes to Paperless-ngx. If your version doesn't support dry-run, use a dedicated test document ID.

Step 8: Track /api/classify Response Times

The classification endpoint is the most latency-sensitive API in the stack — users (or automated triggers) call it to get document categories.

1. Add an HTTP monitor for https://paperless-ai.yourdomain.com/api/classify.
2. Set Method to POST.
3. Set body:

{"document_id": 1}

4. Set Expected status to 200.
5. Enable Response time alerting at 15000 ms.

Classification time depends on document length and LLM model size. Establish a baseline in your first week and adjust the threshold accordingly.

Step 9: Heartbeat for Scheduled Batch Processing

Paperless-AI can run nightly batch jobs to classify documents that arrived during the day or re-classify documents after model updates. Add a heartbeat:

# crontab
0 1 * * * /opt/paperless-ai/scripts/batch-classify.sh && \
  curl -s https://vigilmon.online/heartbeat/YOUR_BATCH_ID

In Vigilmon, create a Cron Heartbeat with Expected interval 1440 minutes.

Step 10: Monitor the New-Document Detection Hook

Paperless-AI typically registers a post-consume script or webhook with Paperless-ngx that fires when a new document is added. If this hook is broken, new documents are never queued for classification even though Paperless-ngx ingests them normally.

Add an HTTP monitor for the hook endpoint that Paperless-AI exposes:

1. Add an HTTP monitor for https://paperless-ai.yourdomain.com/api/consume (or your hook URL).
2. Set Method to GET.
3. Set Expected status to 200 or 405 (Method Not Allowed for a GET on a POST-only endpoint is still a healthy process response).
4. Label it Document detection hook.

Step 11: SSL/TLS Certificate Expiry Alert

1. Open the Paperless-AI monitor from Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

Also add an SSL check to your Paperless-ngx monitor (Step 3) if it runs on a separate domain.

Step 12: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and connect Slack, email, or a webhook.
2. Set Consecutive failures before alert to 2 on the FastAPI and Paperless-ngx HTTP monitors.
3. Keep Consecutive failures at 1 for:
   • Celery worker heartbeat — a missed ping means tasks are silently not running
   • Batch processing heartbeat — a skipped nightly run means documents stay unclassified
4. Set Consecutive failures to 3 on the LLM backend monitor — LLM endpoints can be briefly slow without constituting an outage.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Paperless-AI service | https://paperless-ai.yourdomain.com | FastAPI process crash | | Health endpoint | GET /health | DB / Redis connectivity | | Paperless-ngx API | GET /api/ with token | Source document store down | | Ollama TCP | host:11434 | LLM backend unreachable | | Celery heartbeat | Vigilmon heartbeat URL | Worker crash, queue dead | | Redis TCP | host:6379 | Message broker down | | Tag pipeline | POST /api/tag | Tagging endpoint broken | | Classify API | POST /api/classify | Classification slow/broken | | Batch heartbeat | Vigilmon heartbeat URL | Nightly batch not running | | Detection hook | GET /api/consume | New-document hook broken | | SSL certificate | paperless-ai.yourdomain.com | TLS cert expiry |

Paperless-AI turns your document archive into an intelligently organized, auto-tagged library — but that intelligence depends on a stack of moving parts all working in concert. With Vigilmon monitoring every layer from the FastAPI service to the LLM backend and nightly batch jobs, you know immediately when any piece of the pipeline fails and documents start piling up unclassified.