Monitoring Privy AI with Vigilmon

Privy is an open-source VS Code extension that provides AI code completion, inline suggestions, and a chat panel — all powered by a locally running LLM backend. Instead of sending code to GitHub or OpenAI, Privy talks to Ollama on port 11434 or LM Studio on port 1234, keeping every prompt and completion entirely on the developer's machine or company network. When the local backend goes down, all Privy features disappear silently inside VS Code — no error, just an unresponsive extension. Vigilmon gives you continuous visibility into the backend, the model, fill-in-the-middle completion health, latency, and the VS Code extension process itself.

What You'll Set Up

• Local LLM backend availability monitor (Ollama port 11434 or LM Studio port 1234)
• Code model load status heartbeat (deepseek-coder, codestral, or starcoder2)
• Fill-in-the-middle (FIM) completion health heartbeat
• Completion latency heartbeat (P50/P95 time from VS Code trigger to suggestion)
• Chat response latency heartbeat (Privy chat panel response time)
• VS Code Language Server Protocol health heartbeat
• Concurrent connection handling heartbeat (shared Ollama backend under load)
• Ollama model version currency heartbeat

Prerequisites

• Privy VS Code extension installed
• Ollama running on port 11434 or LM Studio on port 1234
• A code-specialized model loaded in Ollama (deepseek-coder, codestral, starcoder2, or equivalent)
• A free Vigilmon account

Step 1: Monitor the Local LLM Backend

Every Privy feature — autocomplete, inline suggestions, and the chat panel — depends on the local LLM backend being reachable. Ollama is robust but can crash on OOM events, power-cycle restarts, or after a system suspend resumes and the GPU is not yet initialized. LM Studio can unexpectedly unload a model when the user closes and reopens the application. A down backend is completely invisible inside VS Code: the extension shows no error and the developer assumes the feature is just slow.

Ollama (port 11434)

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter: http://localhost:11434/ (or your LAN IP if Ollama runs on a shared server, e.g. http://192.168.1.x:11434/).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

LM Studio (port 1234)

1. Click Add Monitor → HTTP / HTTPS.
2. Enter: http://localhost:1234/v1/models.
3. Set Expected HTTP status to 200.
4. Set Check interval to 2 minutes.
5. Click Save.

LM Studio's /v1/models endpoint returns the currently loaded model. An empty data array means no model is available.

Step 2: Heartbeat Monitor for Code Model Load Status

Ollama running is not enough — the specific code model that Privy requires must be loaded. Developers often pull a general-purpose model for chatting and then find that Privy's code completions fail because deepseek-coder:6.7b or codestral:7b was never pulled into Ollama. Similarly, a model evicted from VRAM after an idle timeout needs a warm-up request before fast completions resume.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 10 minutes.
3. Copy the heartbeat URL.

Create check_privy_model.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_MODEL_HEARTBEAT_ID"
# Set to the model name configured in Privy's VS Code settings
REQUIRED_MODEL="${PRIVY_MODEL:-deepseek-coder:6.7b}"

RESPONSE=$(curl -s http://localhost:11434/api/tags)
if echo "$RESPONSE" | grep -q "\"$REQUIRED_MODEL\""; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Required model '$REQUIRED_MODEL' not found in Ollama"
  echo "Available models: $RESPONSE"
fi

For LM Studio, query /v1/models instead:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_MODEL_HEARTBEAT_ID"
RESPONSE=$(curl -s http://localhost:1234/v1/models)
MODEL_COUNT=$(echo "$RESPONSE" | python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d.get('data',[])))")
[ "$MODEL_COUNT" -gt 0 ] && curl -s "$HEARTBEAT_URL" || echo "No model loaded in LM Studio"

Schedule every 10 minutes:

*/10 * * * * /path/to/check_privy_model.sh

Step 3: Heartbeat Monitor for Fill-in-the-Middle (FIM) Completion Health

Privy uses fill-in-the-middle (FIM) prompting for code completion — the model receives the code before the cursor and after the cursor, and must fill in the gap. This requires a model that supports FIM tokens (<fim_prefix>, <fim_suffix>, <fim_middle>). A non-FIM model will complete the prefix but ignore the suffix context, producing incorrect suggestions. Verify FIM capability on every heartbeat cycle.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 10 minutes.
3. Copy the heartbeat URL.

Create check_privy_fim.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_FIM_HEARTBEAT_ID"
MODEL="${PRIVY_MODEL:-deepseek-coder:6.7b}"

# Send a FIM-style prompt and verify a non-empty completion is returned
RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$MODEL\",
    \"prompt\": \"<fim_prefix>def greet(name):\\n    <fim_suffix>\\n    return greeting<fim_middle>\",
    \"stream\": false,
    \"options\": {\"num_predict\": 8}
  }")

if echo "$RESPONSE" | grep -q '"response"' && \
   ! echo "$RESPONSE" | python3 -c "import sys,json; r=json.load(sys.stdin); exit(0 if r.get('response','').strip() else 1)" 2>/dev/null; then
  curl -s "$HEARTBEAT_URL"
else
  echo "FIM completion returned empty or error: $RESPONSE"
fi

Schedule every 10 minutes:

*/10 * * * * /path/to/check_privy_fim.sh

Step 4: Heartbeat Monitor for Completion Latency

Privy's inline completion must appear quickly enough to feel interactive. Local models vary from 200 ms to 3 seconds depending on hardware, model size, and quantization level. If the P95 latency exceeds the VS Code extension's internal timeout — typically 2–3 seconds — the suggestion is discarded before it can be shown to the developer. Track latency as a leading indicator before developers start complaining that completions "don't work."

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 5 minutes.
3. Copy the heartbeat URL.

Create check_privy_completion_latency.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_COMPLETION_LATENCY_HEARTBEAT_ID"
MODEL="${PRIVY_MODEL:-deepseek-coder:6.7b}"
LATENCY_THRESHOLD_MS=2500

START_MS=$(date +%s%3N)

RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"prompt\":\"def add(a, b):\",\"stream\":false,\"options\":{\"num_predict\":8}}")

END_MS=$(date +%s%3N)
ELAPSED_MS=$(( END_MS - START_MS ))

if echo "$RESPONSE" | grep -q '"response"' && [ "$ELAPSED_MS" -lt "$LATENCY_THRESHOLD_MS" ]; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Completion latency ${ELAPSED_MS}ms (threshold: ${LATENCY_THRESHOLD_MS}ms)"
fi

Schedule every 5 minutes:

*/5 * * * * /path/to/check_privy_completion_latency.sh

Step 5: Heartbeat Monitor for Chat Response Latency

Privy's chat panel uses the same local LLM backend as code completion but sends longer prompts with more context. Chat latency is typically higher and more variable. If the chat panel takes more than 10–15 seconds to respond to a simple question, developers stop using it. Monitor chat-style latency separately from the short inline completion latency.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 10 minutes.
3. Copy the heartbeat URL.

Create check_privy_chat_latency.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_CHAT_LATENCY_HEARTBEAT_ID"
MODEL="${PRIVY_MODEL:-deepseek-coder:6.7b}"
LATENCY_THRESHOLD_MS=10000  # 10 seconds max for chat

START_MS=$(date +%s%3N)

RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"prompt\":\"What does this Python function do? def fib(n): return n if n<2 else fib(n-1)+fib(n-2)\",\"stream\":false,\"options\":{\"num_predict\":32}}")

END_MS=$(date +%s%3N)
ELAPSED_MS=$(( END_MS - START_MS ))

if echo "$RESPONSE" | grep -q '"response"' && [ "$ELAPSED_MS" -lt "$LATENCY_THRESHOLD_MS" ]; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Chat response latency ${ELAPSED_MS}ms — above threshold or request failed"
fi

Schedule every 10 minutes:

*/10 * * * * /path/to/check_privy_chat_latency.sh

Step 6: Heartbeat Monitor for VS Code LSP Extension Health

Privy runs as a VS Code Language Server Protocol extension. The extension host process manages the LSP server. If the extension host crashes — due to a Node.js exception, a memory leak, or a VS Code update — the Privy LSP server goes down with it. VS Code does not always surface extension host crashes visibly; the extension icon may remain in the sidebar while it is functionally dead.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 15 minutes.
3. Copy the heartbeat URL.

Create check_privy_lsp.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_LSP_HEARTBEAT_ID"

# Check if the Privy extension host process is running
# The extension runs inside VS Code's extension host (extensionHost.js)
# and can be identified by the 'privy' argument in the process list
if pgrep -f "extensionHost" > /dev/null && pgrep -f "privy" > /dev/null; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Privy LSP / VS Code extension host process not found"
fi

Note: On shared developer workstations, the VS Code extension host process name and argument vary. Adjust the pgrep pattern to match your environment (code --type=extensionHost on Linux).

Schedule every 15 minutes:

*/15 * * * * /path/to/check_privy_lsp.sh

Step 7: Heartbeat Monitor for Concurrent Connection Handling

On teams where multiple developers share a single Ollama backend server, concurrent completion requests queue inside Ollama's request handler. A spike of simultaneous IDE save events can cause all developers to experience multi-second latency at once. Monitor concurrent request handling by firing parallel requests and measuring wall-clock time.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 5 minutes.
3. Copy the heartbeat URL.

Create check_privy_concurrency.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_CONCURRENCY_HEARTBEAT_ID"
MODEL="${PRIVY_MODEL:-deepseek-coder:6.7b}"
MAX_WALL_CLOCK_MS=6000  # Allow up to 6s for 3 queued requests

START_MS=$(date +%s%3N)

# Fire 3 concurrent completion requests
for i in 1 2 3; do
  curl -s http://localhost:11434/api/generate \
    -H "Content-Type: application/json" \
    -d "{\"model\":\"$MODEL\",\"prompt\":\"// request $i\",\"stream\":false,\"options\":{\"num_predict\":4}}" \
    > /dev/null &
done
wait

END_MS=$(date +%s%3N)
TOTAL_MS=$(( END_MS - START_MS ))

if [ "$TOTAL_MS" -lt "$MAX_WALL_CLOCK_MS" ]; then
  curl -s "$HEARTBEAT_URL"
else
  echo "3 concurrent requests took ${TOTAL_MS}ms — Ollama queue may be backing up"
fi

Schedule every 5 minutes:

*/5 * * * * /path/to/check_privy_concurrency.sh

Step 8: Heartbeat Monitor for Ollama Model Version Currency

Code models are updated frequently — deepseek-coder, starcoder2, and codestral release new versions with improved completions. Running a months-old model version means missing bug fixes and quality improvements. Alert when the locally installed model version lags behind what the Ollama registry shows.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 1 day (1440 minutes).
3. Copy the heartbeat URL.

Create check_privy_model_version.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_VERSION_HEARTBEAT_ID"
MODEL="${PRIVY_MODEL:-deepseek-coder:6.7b}"

# Pull the latest version info without downloading (dry-run equivalent)
# Ollama's /api/show returns the current local model digest
LOCAL_DIGEST=$(curl -s -X POST http://localhost:11434/api/show \
  -H "Content-Type: application/json" \
  -d "{\"name\":\"$MODEL\"}" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('details',{}).get('digest',''))" 2>/dev/null)

if [ -n "$LOCAL_DIGEST" ]; then
  # Model is present locally — heartbeat is healthy (separate process handles version checks)
  curl -s "$HEARTBEAT_URL"
else
  echo "Model '$MODEL' not found locally — pull required"
fi

Schedule daily at 02:00:

0 2 * * * /path/to/check_privy_model_version.sh

Step 9: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, Discord, or a webhook.
2. On the Ollama / LM Studio backend monitor: set Consecutive failures before alert to 1 — inference server crashes are hard failures.
3. On the latency heartbeats: default single-missed-interval threshold is appropriate.
4. On the LSP health heartbeat: set Consecutive failures to 2 — VS Code extension restarts briefly interrupt the process.

Summary

| Monitor | Type | Target | Interval | |---------|------|--------|----------| | Ollama backend | HTTP | http://localhost:11434/ | 1 min | | LM Studio backend | HTTP | http://localhost:1234/v1/models | 2 min | | Code model load status | Heartbeat | probe script → Vigilmon | 10 min | | FIM completion health | Heartbeat | probe script → Vigilmon | 10 min | | Completion latency | Heartbeat | probe script → Vigilmon | 5 min | | Chat response latency | Heartbeat | probe script → Vigilmon | 10 min | | VS Code LSP extension health | Heartbeat | probe script → Vigilmon | 15 min | | Concurrent connection handling | Heartbeat | probe script → Vigilmon | 5 min | | Model version currency | Heartbeat | probe script → Vigilmon | daily |

Privy's promise is privacy-first AI code assistance where all data stays local. That privacy guarantee is only useful when the local backend is healthy. With Vigilmon watching the Ollama backend, the loaded model, FIM completion health, and latency, you know immediately when the invisible infrastructure that powers Privy goes wrong.

Get started for free at vigilmon.online.