tutorial

Monitoring KoboldCpp with Vigilmon

KoboldCpp is the go-to self-hosted LLM inference server — but large models can take 10+ minutes to load and fail silently. Here's how to monitor the web UI, REST API endpoints, and model loading status with Vigilmon.

KoboldCpp is the most popular all-in-one self-hosted LLM inference server — a single binary that wraps llama.cpp with GPU acceleration (CUDA, Metal, OpenCL, Vulkan), a built-in web UI, a KoboldAI-compatible API, and an OpenAI-compatible API. Running Llama, Mistral, Gemma, or any other GGUF model on consumer hardware is exactly what it's designed for.

But KoboldCpp has a monitoring blind spot: large models can take 2–10 minutes to load on startup. During that window, the web server is running but the model isn't ready — requests will return errors or empty responses. And if the server crashes or a GPU driver resets mid-session, nothing alerts you. Vigilmon fills that gap with endpoint health checks, model loading status heartbeats, and SSL certificate alerts for proxied deployments.

What You'll Set Up

  • HTTP uptime monitor for the KoboldCpp web UI (port 5001)
  • REST API availability checks for /api/extra/version and /api/v1/info
  • SSL certificate expiry alerts for HTTPS-proxied KoboldCpp deployments
  • Heartbeat monitor for KoboldCpp model loading status (confirms the model is ready for inference)

Prerequisites

  • KoboldCpp running on Linux, Windows, or macOS (accessible at http://your-server:5001 or via a reverse proxy)
  • A free Vigilmon account

Step 1: Monitor the KoboldCpp Web UI

KoboldCpp serves its built-in web interface on port 5001 by default. This is your primary availability signal — if port 5001 is unreachable, the entire server is down.

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: http://your-server-ip:5001 (or https://kobold.yourdomain.com if behind a reverse proxy).
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

This check probes the homepage of the KoboldCpp web UI. If KoboldCpp crashes, fails to bind to port 5001, or the host machine goes offline, this monitor fires immediately.

Step 2: Monitor the /api/extra/version Endpoint

KoboldCpp exposes an unauthenticated /api/extra/version endpoint that returns a JSON response confirming the server is running and the API is responsive. This is a lightweight health probe independent of model state:

GET http://your-server-ip:5001/api/extra/version

Example response:

{
  "result": "1.75.1",
  "version": 175
}

Add a monitor for this endpoint:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: http://your-server-ip:5001/api/extra/version
  3. Expected HTTP status: 200
  4. Response body contains: "result"
  5. Check interval: 1 minute
  6. Click Save.

The body check on "result" confirms you're getting a valid JSON API response — not an nginx 200 from a misconfigured proxy returning an error page.

Step 3: Monitor the /api/v1/info Endpoint

The /api/v1/info endpoint goes one level deeper — it returns model metadata confirming that a model is loaded and ready for inference:

GET http://your-server-ip:5001/api/v1/info

Example response when a model is loaded:

{
  "result": {
    "model": "mistral-7b-instruct-v0.2.Q4_K_M.gguf",
    "version": 175,
    "max_context_length": 4096,
    "queue": 0
  }
}

When no model is loaded (during startup or after a crash), this endpoint returns an error or an empty model name. Add a monitor:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: http://your-server-ip:5001/api/v1/info
  3. Expected HTTP status: 200
  4. Response body contains: "model"
  5. Check interval: 1 minute
  6. Click Save.

Step 4: SSL Certificate Alerts for HTTPS-Proxied Deployments

If you've placed KoboldCpp behind an nginx or Caddy reverse proxy with a TLS certificate, add certificate expiry monitoring:

  1. Open the HTTP monitor you created in Step 1 (your HTTPS URL).
  2. Enable Monitor SSL certificate in the SSL section.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

A typical nginx reverse proxy config for KoboldCpp:

server {
    listen 443 ssl;
    server_name kobold.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/kobold.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/kobold.yourdomain.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:5001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 300s;
    }
}

The 21-day alert window gives you time to diagnose and fix Let's Encrypt renewal issues before the certificate lapses.

Step 5: Heartbeat Monitoring for Model Loading Status

KoboldCpp's most dangerous failure mode is the extended startup window. When loading a 30B+ parameter model, the /api/v1/info endpoint may return a 200 response with an empty or null model name for several minutes — the server is up, but inference isn't available. A standard HTTP monitor will report the service as healthy when it isn't.

Use a Vigilmon heartbeat monitor to get a push signal after model loading completes. Set up a script that polls KoboldCpp's model status and pings Vigilmon only when the model is confirmed ready:

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set the expected ping interval to 15 minutes (model startup can take up to 10 minutes; this gives a generous buffer).
  3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/YOUR_KEY

Create a polling script that runs on your server at startup and pings Vigilmon once the model is ready:

#!/bin/bash
# kobold-ready-check.sh
# Run this in the background after starting KoboldCpp

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_KEY"
KOBOLD_URL="http://localhost:5001/api/v1/info"
MAX_WAIT=900  # 15 minutes max
ELAPSED=0
INTERVAL=30

while [ $ELAPSED -lt $MAX_WAIT ]; do
    RESPONSE=$(curl -s --max-time 10 "$KOBOLD_URL" 2>/dev/null)
    MODEL=$(echo "$RESPONSE" | grep -o '"model":"[^"]*"' | cut -d'"' -f4)

    if [ -n "$MODEL" ] && [ "$MODEL" != "null" ] && [ "$MODEL" != "" ]; then
        echo "Model loaded: $MODEL — pinging Vigilmon heartbeat"
        curl -s "$HEARTBEAT_URL"
        exit 0
    fi

    echo "Waiting for model... ($ELAPSED s elapsed)"
    sleep $INTERVAL
    ELAPSED=$((ELAPSED + INTERVAL))
done

echo "Model did not load within $MAX_WAIT seconds"
exit 1

Add this to your KoboldCpp startup sequence:

# Start KoboldCpp in background
./koboldcpp --model models/mistral-7b.gguf --port 5001 &

# Wait for model ready and ping heartbeat
bash kobold-ready-check.sh &

If KoboldCpp fails to load the model within 15 minutes — due to an OOM error, a corrupted model file, or a driver crash — the heartbeat never arrives and Vigilmon fires an alert.

Step 6: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add your preferred channel (Slack, Discord, email, or webhook).
  2. Assign the channel to all three monitors.
  3. Set Consecutive failures before alert to 2 on the web UI monitor — a single slow response during model loading shouldn't immediately page you.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Web UI | http://your-server:5001 | Server crash, port binding failure | | API version | /api/extra/version | API layer failure, proxy misconfiguration | | Model info | /api/v1/info | Model not loaded, inference unavailable | | SSL certificate | HTTPS domain | Let's Encrypt renewal failure | | Cron heartbeat | Heartbeat URL | Model load timeout, OOM crash during startup |

KoboldCpp puts powerful LLM inference on consumer hardware — but self-hosted means self-monitored. With Vigilmon watching the web UI, the API endpoints, and the model loading pipeline, you'll know the moment KoboldCpp stops serving inference — not when users start complaining.

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles