tutorial

Monitoring PrivateGPT with Vigilmon

PrivateGPT lets you query sensitive documents with a local LLM — no data leaves your machine. But it depends on both a vector database and an LLM backend. Here's how to monitor the full RAG pipeline with Vigilmon.

PrivateGPT is a production-ready open-source project for private, offline document question answering. Ingest PDFs, DOCX files, spreadsheets, and Markdown into a local vector store (Qdrant or Chroma), then ask questions against them using a locally-running LLM — all without any data leaving your machine. It's the go-to solution for legal teams, medical practices, and financial organizations that need AI-powered document search without cloud dependencies.

But PrivateGPT's architecture has two independent failure points: the vector database that stores your document embeddings, and the LLM backend (Ollama, LlamaCpp, or LM Studio) that generates answers. If either goes down, the system answers questions incorrectly or not at all — often without any visible error to the end user. Vigilmon lets you monitor both layers plus the Gradio web UI and document ingestion API.

What You'll Set Up

  • HTTP uptime monitor for the PrivateGPT Gradio web UI (port 8001)
  • REST API health check for the /health endpoint
  • Document index availability check via the /v1/ingest/list endpoint
  • SSL certificate expiry alerts for HTTPS-proxied PrivateGPT deployments
  • Heartbeat monitor for the full RAG pipeline (vector retrieval + LLM inference)

Prerequisites

  • PrivateGPT running on Linux, macOS, or Windows (accessible at http://your-server:8001)
  • A free Vigilmon account

A minimal PrivateGPT startup (with Ollama as the LLM backend):

# Start Ollama in background
ollama serve &

# Start PrivateGPT
poetry run python -m private_gpt

PrivateGPT listens on port 8001 (Gradio UI) and 8001 (REST API) by default. Check your settings.yaml for the configured port.

Step 1: Monitor the PrivateGPT Gradio Web UI

PrivateGPT serves its Gradio chat interface on port 8001. A successful response here confirms the Python server process is alive and the web layer is serving:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: http://your-server-ip:8001 (or https://privategpt.yourdomain.com if behind a reverse proxy).
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

If the PrivateGPT Python process crashes or fails to bind, this monitor fires within 1 minute.

Step 2: Monitor the /health Endpoint

PrivateGPT exposes a /health endpoint that returns a structured health status. This is a more reliable signal than the Gradio UI root, as it bypasses the frontend layer and hits the API directly:

GET http://your-server-ip:8001/health

Response:

{ "status": "ok" }

Add a monitor:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: http://your-server-ip:8001/health
  3. Expected HTTP status: 200
  4. Response body contains: "ok"
  5. Check interval: 1 minute
  6. Click Save.

The body check on "ok" confirms you're getting a valid health response — not an nginx 200 from a misconfigured proxy.

Step 3: Monitor the Document Ingestion Index

PrivateGPT's value depends entirely on the documents in its vector store. The /v1/ingest/list endpoint returns the list of ingested documents and confirms the vector database is accessible and the document index is readable:

GET http://your-server-ip:8001/v1/ingest/list

Example response:

{
  "object": "list",
  "model": "private-gpt",
  "data": [
    {
      "object": "ingest.document",
      "doc_id": "a1b2c3d4...",
      "doc_metadata": {
        "file_name": "Q4-financial-report.pdf",
        "page_label": "1"
      }
    }
  ]
}

Add a monitor:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: http://your-server-ip:8001/v1/ingest/list
  3. Expected HTTP status: 200
  4. Response body contains: "data"
  5. Check interval: 2 minutes
  6. Click Save.

If the Qdrant or Chroma vector database becomes unreachable — because the process crashed, the data directory is unmounted, or the database port is blocked — this endpoint will fail before users notice their queries returning empty results.

Step 4: SSL Certificate Alerts for HTTPS-Proxied Deployments

If you've placed PrivateGPT behind an nginx or Caddy reverse proxy with TLS, add certificate expiry monitoring. For an organization handling sensitive documents, a lapsed certificate is a security and compliance concern.

  1. Open the HTTP monitor you created in Step 1 (your HTTPS URL).
  2. Enable Monitor SSL certificate in the SSL section.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

A typical nginx reverse proxy config for PrivateGPT:

server {
    listen 443 ssl;
    server_name privategpt.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/privategpt.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/privategpt.yourdomain.com/privkey.pem;

    # PrivateGPT document upload can be slow
    client_max_body_size 100M;
    proxy_read_timeout 300s;

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Step 5: Heartbeat Monitoring for the Full RAG Pipeline

PrivateGPT's architecture has three layers that must all be healthy simultaneously: the PrivateGPT Python server, the vector database (Qdrant or Chroma), and the LLM backend (Ollama). The /health endpoint only confirms the Python server is alive. A vector DB or LLM failure will make the server appear healthy while being functionally broken.

Use a Vigilmon heartbeat monitor with a live completion probe to validate the entire RAG pipeline:

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set the expected ping interval to 15 minutes.
  3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/YOUR_KEY

Create a script that sends a test completion request and pings Vigilmon only when the full pipeline responds:

#!/bin/bash
# privategpt-pipeline-check.sh
# Run via cron every 10 minutes

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_KEY"
PGPT_URL="http://localhost:8001/v1/completions"

RESPONSE=$(curl -s --max-time 120 -X POST "$PGPT_URL" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Hello",
    "use_context": false,
    "max_new_tokens": 1,
    "include_sources": false
  }' 2>/dev/null)

STATUS=$?

# Check curl succeeded and response contains a completion
if [ $STATUS -eq 0 ] && echo "$RESPONSE" | grep -q '"choices"'; then
    curl -s "$HEARTBEAT_URL"
fi

Setting use_context: false bypasses vector retrieval and tests only the LLM backend — set it to true if you also want to verify vector retrieval is working on each heartbeat:

# Test the full RAG pipeline including vector retrieval:
-d '{
    "prompt": "What documents have been ingested?",
    "use_context": true,
    "max_new_tokens": 5,
    "include_sources": false
}'

Add to crontab:

*/10 * * * * /home/user/privategpt-pipeline-check.sh

If the LLM backend (Ollama) crashes, the vector database becomes inaccessible, or the generation pipeline stalls, the heartbeat stops within 10 minutes and Vigilmon fires an alert.

Step 6: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
  2. Assign the alert channel to all monitors.
  3. For the web UI and /health monitors, set Consecutive failures before alert to 2 — brief restarts during model loading shouldn't immediately page you.
  4. For the /v1/ingest/list monitor, set consecutive failures to 1 — an inaccessible document index means users are querying against nothing.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Gradio web UI | http://your-server:8001 | Python server crash, port binding failure | | Health endpoint | /health | API layer failure | | Document index | /v1/ingest/list | Vector DB down, document store inaccessible | | SSL certificate | HTTPS domain | Let's Encrypt renewal failure | | Cron heartbeat | /v1/completions | LLM backend failure, full RAG pipeline broken |

PrivateGPT's promise — that sensitive documents never leave your machine — depends entirely on the system being operational. A downed vector database or crashed LLM backend can silently break the RAG pipeline, returning empty answers without alerting anyone. With Vigilmon watching every layer from the web UI to the end-to-end completion probe, you'll know the moment PrivateGPT stops being able to answer questions.

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles