tutorial

Monitoring Tabby ML Code Assistant with Vigilmon

Tabby is a self-hosted GitHub Copilot alternative — but when the Rust server goes down or inference slows, every connected IDE loses code completion silently. Here's how to monitor Tabby's API, model loading, inference latency, and SSL with Vigilmon.

Tabby is an open-source, self-hosted AI coding assistant server. It serves code completion, chat, and code review to VS Code, JetBrains, and Vim extensions — using open models like CodeLlama, StarCoder, and DeepSeek Coder — without ever sending your code to a third-party server. The Rust binary runs on port 8080 and supports both CPU and GPU inference. Vigilmon monitors Tabby's critical endpoints so you know the moment a developer's inline completions go dark, a model fails to load, or the server crashes mid-session.

What You'll Set Up

  • Tabby server process availability monitor (port 8080)
  • Code completion API health check (POST /v1/completions)
  • Chat API health check (POST /v1/chat/completions)
  • Model registry endpoint health (GET /v1/models)
  • SSL certificate expiry alerts
  • Alerting tuned for developer experience impact

Prerequisites

  • Tabby server running: tabby serve --device cuda --model TabbyML/StarCoder-1B (or CPU equivalent)
  • Server accessible at http://your-host:8080 (or behind a reverse proxy)
  • A free Vigilmon account

Step 1: Monitor the Tabby Server Process

The Tabby Rust binary is the single process that all IDE extensions depend on. If it crashes — OOM-killed during model loading, CUDA error, or unexpected panic — every connected developer loses completions immediately with no IDE-side warning.

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the Tabby root URL: 
    http://your-host:8080/
    Or if behind a TLS reverse proxy: 
    https://tabby.yourdomain.com/
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

Tabby serves its web UI at the root path. A 200 response confirms the process is running and accepting connections.

Step 2: Monitor the Code Completion Endpoint

The /v1/completions endpoint is what IDE extensions call every time a developer pauses while typing. Latency here directly impacts developer experience — a response over 500ms feels broken even if the server is technically up. Monitor availability and catch endpoint-level failures that might not affect the root path.

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter: 
    http://your-host:8080/v1/completions
  3. Set Method to POST.
  4. Under Request body, enter a minimal valid completion request: 
    {
  "language": "python",
  "segments": {
    "prefix": "def hello():\n    ",
    "suffix": ""
  }
}
  5. Set Expected HTTP status to 200.
  6. Set Check interval to 2 minutes.
  7. Click Save.

This probe exercises the full completion path — request parsing, model inference, and response formatting — the same path IDE extensions use.

Step 3: Monitor the Chat Completions Endpoint

Tabby's chat API (/v1/chat/completions) powers the conversational code assistance feature in VS Code and JetBrains. It uses a separate model path from completions and can fail independently if the chat model fails to load or runs out of VRAM.

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter: 
    http://your-host:8080/v1/chat/completions
  3. Set Method to POST.
  4. Under Request body, enter: 
    {
  "messages": [
    {"role": "user", "content": "ping"}
  ],
  "model": "default"
}
  5. Set Expected HTTP status to 200.
  6. Set Check interval to 5 minutes.
  7. Click Save.

Step 4: Monitor the Model Registry Endpoint

GET /v1/models lists the models Tabby has loaded and available. An empty response or error here means Tabby failed to load any models at startup — a silent failure that causes completion requests to return errors or empty responses.

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter: 
    http://your-host:8080/v1/models
  3. Set Method to GET.
  4. Set Expected HTTP status to 200.
  5. Optionally, set Response must contain to a model name fragment like StarCoder or CodeLlama to verify the specific model you depend on is loaded.
  6. Set Check interval to 5 minutes.
  7. Click Save.

Step 5: Monitor Inference Latency with a Heartbeat Probe

Tabby's response time to a completion request is the clearest developer-experience metric. A server that responds in 3 seconds is technically up but practically broken for inline completion. Use a heartbeat probe that measures end-to-end latency and only pings Vigilmon when it's within your SLO.

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set Expected ping interval to 5 minutes.
  3. Copy the heartbeat URL.

Create check_tabby_latency.sh:

#!/bin/bash
TABBY_URL="${TABBY_URL:-http://localhost:8080}"
LATENCY_THRESHOLD_MS=1000  # alert if >1s

START=$(date +%s%3N)
RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" \
  -X POST "$TABBY_URL/v1/completions" \
  -H "Content-Type: application/json" \
  -d '{"language":"python","segments":{"prefix":"def ","suffix":""}}' \
  --max-time 10)
END=$(date +%s%3N)
ELAPSED=$((END - START))

if [ "$RESPONSE" = "200" ] && [ "$ELAPSED" -lt "$LATENCY_THRESHOLD_MS" ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID"
else
  echo "Tabby latency check failed: HTTP=$RESPONSE latency=${ELAPSED}ms"
fi

Schedule every 5 minutes:

*/5 * * * * /path/to/check_tabby_latency.sh

If latency spikes above 1 second (adjust to your team's tolerance) or the request fails entirely, the heartbeat stops and Vigilmon alerts.

Step 6: Monitor Model Loading on Startup

Tabby downloads GGUF/GGML model files on first startup and reloads them on subsequent starts. If the model file is corrupted, the directory is missing, or there's insufficient disk space, Tabby starts but returns errors for every completion request.

Add a disk space heartbeat for the Tabby model directory:

#!/bin/bash
MODEL_DIR="${TABBY_HOME:-$HOME/.tabby}/models"
# Check model directory exists and has files
[ -d "$MODEL_DIR" ] && [ "$(ls -A "$MODEL_DIR")" ] || exit 1
# Check at least 2GB free on the model volume
AVAIL_GB=$(df -BG "$MODEL_DIR" | awk 'NR==2{gsub("G",""); print $4}')
[ "$AVAIL_GB" -gt 2 ] || exit 1
curl -s "https://vigilmon.online/heartbeat/YOUR_MODEL_HEARTBEAT_ID"

Schedule hourly. If a model gets accidentally deleted or disk fills during a model download, this heartbeat stops within an hour.

Step 7: Monitor the Events Analytics Endpoint

Tabby tracks which completions developers accept vs. reject via POST /v1/events. This data powers the Tabby dashboard's productivity analytics. If this endpoint fails, analytics silently stop accumulating — you lose visibility into which models and completion strategies work best for your team.

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter: 
    http://your-host:8080/v1/events
  3. Set Method to POST.
  4. Under Request body, enter a minimal event: 
    {
  "type": "view",
  "completion_id": "test-monitor-id",
  "choice_index": 0
}
  5. Set Expected HTTP status to 200.
  6. Set Check interval to 10 minutes.
  7. Click Save.

Step 8: SSL Certificate Monitoring

If Tabby is behind a reverse proxy (nginx, Caddy, Traefik), monitor the TLS certificate to catch auto-renewal failures before they cause IDE extension connection errors.

  1. Open the Tabby server HTTP monitor from Step 1.
  2. Enable Monitor SSL certificate.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

Step 9: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon → Add → choose Slack, email, Discord, or webhook.
  2. Assign the alert channel to all Tabby monitors.
  3. Tune failure thresholds:
    • Server availability (Step 1): 2 consecutive failures — one missed probe may be a blip during garbage collection.
    • Completions endpoint (Step 2): 1 consecutive failure — completion failures are immediately user-visible.
    • Latency heartbeat (Step 5): default missed-interval alert is appropriate.
  4. Consider paging on-call during business hours for the completions endpoint — developer productivity is directly affected.

Summary

| Monitor | Type | URL / Target | Interval | |---------|------|-------------|----------| | Tabby server process | HTTP | http://host:8080/ | 1 min | | Code completions API | HTTP POST | /v1/completions | 2 min | | Chat completions API | HTTP POST | /v1/chat/completions | 5 min | | Model registry | HTTP GET | /v1/models | 5 min | | Events analytics | HTTP POST | /v1/events | 10 min | | Inference latency | Heartbeat | probe script → Vigilmon | 5 min | | Model directory / disk | Heartbeat | probe script → Vigilmon | 60 min | | SSL certificate | SSL (on HTTP monitor) | reverse proxy domain | — |

Every developer on your team depends on Tabby the moment they type. With Vigilmon watching every layer — server process, completion latency, model loading, and TLS — you catch problems before developers notice their suggestions have gone silent.

Get started for free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles