Monitoring text-generation-webui (Oobabooga) with Vigilmon

text-generation-webui (nicknamed "Oobabooga" after its creator) is the most widely used open-source web interface for running large language models locally. It supports every major backend — llama.cpp, ExLlamaV2, AutoGPTQ, transformers — and provides a Gradio chat interface, an OpenAI-compatible API extension, a rich model loader, and an extension ecosystem covering function calling, voice synthesis, and retrieval-augmented generation. It's the tool of choice for home lab users, researchers, and teams building private AI applications.

Running local LLM inference means no SLA from a cloud provider — uptime, performance, and crash recovery are entirely your responsibility. Vigilmon gives you continuous monitoring for text-generation-webui's Gradio UI, API endpoints, GPU resource headroom, and process health, so you know when the local AI stack goes down before your users do.

What You'll Set Up

• Gradio web UI availability monitor (port 7860)
• OpenAI-compatible API monitor (port 5000)
• Model loading status heartbeat
• GPU/CPU memory utilization alert
• Generation throughput heartbeat
• Extension health check
• Process liveness (crash detection) via heartbeat
• Disk space alert for model files

Prerequisites

• text-generation-webui running with at least one model loaded
• API extension enabled (--api flag) if you want API monitoring
• A free Vigilmon account

Why Monitoring a Local LLM UI Matters

text-generation-webui crashes more often than a typical web service. The Python process can be OOM-killed silently, a bad GGUF file can hang the model loader, an ExLlamaV2 context overflow can crash the backend, and a disk-full error can silently prevent new models from loading. Without monitoring, you discover the problem when you open the browser and see a blank page — or when an API client starts returning 500 errors.

Proactive monitoring means you get an alert before you sit down to use the tool, and before applications that depend on the API notice.

Step 1: Monitor the Gradio Web UI (Port 7860)

The Gradio interface is the primary way users interact with text-generation-webui. A simple HTTP check on the root URL tells you immediately if the UI is up:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter: http://your-server:7860/
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

Name it "Oobabooga Gradio UI". If text-generation-webui is configured with --listen to accept connections from other machines, use the server's LAN IP or hostname. For local-only setups monitoring from the same machine, use http://127.0.0.1:7860/.

If Gradio is behind an nginx reverse proxy with authentication, configure the monitor to probe a public health path or add the auth credentials to the monitor configuration.

Step 2: Monitor the OpenAI-Compatible API (Port 5000)

The --api extension enables an OpenAI-format API that many downstream applications use as a drop-in replacement for OpenAI's endpoints. Monitor it separately from the Gradio UI — the API and UI can fail independently.

1. Add a second monitor.
2. Set Type to HTTP / HTTPS.
3. Enter: http://your-server:5000/v1/models
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

The /v1/models endpoint returns the currently loaded model name in OpenAI-compatible format. A 200 response confirms the API extension is running and a model is loaded. A 404 means the API extension is not loaded; a connection refused means the API server is not running at all.

Name this monitor "Oobabooga API" to distinguish it from the UI monitor.

Step 3: Model Loading Status Heartbeat

text-generation-webui supports hot-swapping models from the UI — loading a 13B GGUF while a previous model is in memory can take 30–90 seconds and temporarily makes the API unavailable. More critically, a corrupt model file or insufficient VRAM can leave the loader stuck or crash the process entirely.

Set up a Vigilmon cron heartbeat that checks the API for a loaded model and sends a heartbeat only when a model is actively loaded:

#!/bin/bash
RESPONSE=$(curl -s --max-time 5 http://localhost:5000/v1/models)
MODEL_COUNT=$(echo "$RESPONSE" | python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d.get('data',[])))" 2>/dev/null || echo 0)

if [ "$MODEL_COUNT" -gt 0 ]; then
  curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID" > /dev/null
fi

Schedule this every 2 minutes. Set the Vigilmon heartbeat interval to 6 minutes so you need three consecutive failures before alerting — this prevents false alarms during model swaps.

Step 4: GPU and CPU Memory Monitoring

text-generation-webui loads model weights into VRAM (for GPU backends) or RAM (for CPU inference with llama.cpp). Exceeding available memory causes a crash — often an OOM kill with no warning to the user.

For GPU monitoring (NVIDIA):

#!/bin/bash
USED=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | head -1)
TOTAL=$(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits | head -1)
PCT=$(echo "scale=0; $USED * 100 / $TOTAL" | bc)

if [ "$PCT" -lt 90 ]; then
  curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID" > /dev/null
fi

For CPU/RAM monitoring (CPU inference or hybrid):

#!/bin/bash
AVAILABLE_KB=$(grep MemAvailable /proc/meminfo | awk '{print $2}')
TOTAL_KB=$(grep MemTotal /proc/meminfo | awk '{print $2}')
USED_PCT=$(echo "scale=0; ($TOTAL_KB - $AVAILABLE_KB) * 100 / $TOTAL_KB" | bc)

if [ "$USED_PCT" -lt 90 ]; then
  curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID" > /dev/null
fi

Schedule whichever is relevant every 2 minutes. Use a 6-minute heartbeat interval in Vigilmon to avoid alerting on momentary spikes during model loading.

Step 5: Generation Throughput Heartbeat

Generation speed (tokens/sec) varies significantly across backends and model configurations. A throughput regression after a backend update or config change can silently make the UI unusable — responses take 10× longer without any error.

Measure throughput by timing a test generation:

#!/bin/bash
START=$(date +%s%N)
RESPONSE=$(curl -s --max-time 30 -X POST http://localhost:5000/v1/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "any",
    "prompt": "The capital of France is",
    "max_tokens": 20
  }')
END=$(date +%s%N)

TOKENS=$(echo "$RESPONSE" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('usage',{}).get('completion_tokens',0))" 2>/dev/null || echo 0)
ELAPSED_SEC=$(echo "scale=3; ($END - $START) / 1000000000" | bc)

MIN_TOKENS_PER_SEC=2  # Adjust to your backend/model baseline

if [ "$TOKENS" -gt 0 ]; then
  TPS=$(echo "scale=2; $TOKENS / $ELAPSED_SEC" | bc)
  if (( $(echo "$TPS > $MIN_TOKENS_PER_SEC" | bc -l) )); then
    curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID" > /dev/null
  fi
fi

Schedule every 5 minutes. Adjust MIN_TOKENS_PER_SEC to 70% of your observed baseline for the current model and backend.

Step 6: Extension Health Check

text-generation-webui's extension system is powerful but fragile — extensions can fail to load without crashing the main process. Check that critical extensions (API, superbooga, silero-TTS) are active:

#!/bin/bash
# Check API extension is responding
API_STATUS=$(curl -s --max-time 3 -o /dev/null -w "%{http_code}" http://localhost:5000/v1/models)

if [ "$API_STATUS" = "200" ]; then
  curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID_API_EXT" > /dev/null
fi

Create a separate Vigilmon heartbeat for each critical extension. Label them clearly (e.g., "Oobabooga API Extension", "Oobabooga TTS Extension") so you know which extension failed when you receive an alert.

Step 7: Disk Space Alert for Model Files

LLM model files are large — a 7B GGUF is 4–8 GB, a 70B quantized model can be 40+ GB. Running out of disk space silently prevents new model downloads, can corrupt partial downloads, and occasionally causes llama.cpp to crash mid-generation when writing temporary files.

Set up a disk space heartbeat:

#!/bin/bash
MODEL_DIR="/path/to/text-generation-webui/models"
DISK_PCT=$(df "$MODEL_DIR" | awk 'NR==2 {print $5}' | tr -d '%')
THRESHOLD=90

if [ "$DISK_PCT" -lt "$THRESHOLD" ]; then
  curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID" > /dev/null
fi

Schedule every 10 minutes. Set the Vigilmon heartbeat interval to 20 minutes — disk space changes slowly and you want early warning, not an immediate alert.

Step 8: Process Liveness (Crash Detection)

text-generation-webui runs as a Python process. If it crashes (OOM kill, unhandled exception, CUDA error), the Gradio UI and API both go dark immediately. Your HTTP monitors in Steps 1 and 2 will catch this, but a dedicated process liveness heartbeat gives you a secondary signal and is useful for auto-restart integration.

#!/bin/bash
# Check if the server.py process is running
if pgrep -f "server.py" > /dev/null 2>&1; then
  curl -fsS --retry 3 "https://vigilmon.online/api/v1/heartbeats/YOUR_HEARTBEAT_ID" > /dev/null
fi

Pair this with a systemd service and Restart=on-failure to auto-restart text-generation-webui when it crashes:

[Unit]
Description=text-generation-webui
After=network.target

[Service]
Type=simple
User=youruser
WorkingDirectory=/path/to/text-generation-webui
ExecStart=/path/to/text-generation-webui/venv/bin/python server.py --api --listen
Restart=on-failure
RestartSec=10s

[Install]
WantedBy=multi-user.target

With this setup, the process restarts automatically and Vigilmon alerts you to the outage — you get notified without having to babysit the process.

Alerting Configuration

Configure alert destinations in Vigilmon under Settings → Notifications:

| Monitor | Alert condition | Recommended channel | |---------|----------------|---------------------| | Gradio UI (port 7860) | 1 failure | Slack / email | | OpenAI API (port 5000) | 1 failure | Slack | | Model loading heartbeat | Missed heartbeat | Slack | | GPU memory heartbeat | Missed heartbeat (2 intervals) | Slack | | Throughput heartbeat | Missed heartbeat | Slack | | Disk space heartbeat | Missed heartbeat | Email | | Process liveness heartbeat | Missed heartbeat | Slack |

For personal home lab setups, Slack or email is usually sufficient. For team or research deployments where others depend on the API, consider PagerDuty for the Gradio UI and API monitors.

Troubleshooting Common text-generation-webui Failures

Gradio UI down, API down: The Python process has crashed. Check journalctl -u text-generation-webui or the terminal where you started it. Common causes: OOM kill (CUDA or system RAM), uncaught exception in an extension, CUDA error from a corrupted context.

API down, Gradio UI up: The API extension failed to load or crashed independently. Restart the server with --api flag and check the startup logs for extension load errors.

Model loading stuck: The model file may be corrupt (partial download), or VRAM is insufficient for the selected quantization. Delete the cached model file and re-download; try a more aggressive quantization level (Q4_K_M instead of Q8_0).

Throughput regression: Check if the backend changed (llama.cpp version, ExLlamaV2 config), context length setting increased, or system RAM is being used as swap due to insufficient VRAM.

Disk full: Clean up unused model files. text-generation-webui downloads models to the models/ directory; models not actively used can be archived to external storage.

Conclusion

text-generation-webui gives you powerful local LLM inference, but it trades managed hosting reliability for full control. With Vigilmon monitoring the Gradio UI, OpenAI API, GPU memory, throughput, and process liveness, you get the visibility you need to run it reliably — whether for personal use, a home lab, or a team research environment.

Get started at vigilmon.online.