Monitoring Wingman AI with Vigilmon

Wingman AI is an open-source VS Code extension that provides GitHub Copilot-like code completion, explanation, and refactoring — all powered by a locally running LLM via Ollama or any OpenAI-compatible backend. No code leaves your machine. The extension connects to Ollama on port 11434 and uses a user-configured model for every AI feature. When Ollama crashes or the configured model disappears from the registry, Wingman fails silently inside VS Code with no indication of what went wrong. Vigilmon gives you continuous visibility into the Ollama backend, the configured model's availability, completion and explanation latency, extension activation health, and GPU resource utilization.

What You'll Set Up

• Ollama backend availability monitor (port 11434)
• Configured model availability heartbeat (verify model is loaded in Ollama)
• Code completion trigger latency heartbeat (P50/P95 for inline completions)
• Code explanation response latency heartbeat (synchronous chat responses)
• Refactoring suggestion health heartbeat (file write success and syntax check)
• VS Code extension activation health heartbeat
• Multi-file context health heartbeat
• GPU/CPU resource utilization heartbeat

Prerequisites

• Wingman AI VS Code extension installed
• Ollama running on port 11434
• A code model loaded in Ollama (e.g., llama3.1:8b, deepseek-coder:6.7b, codestral:7b)
• A free Vigilmon account

Step 1: Monitor the Ollama Backend (Port 11434)

Wingman's primary LLM backend is Ollama. Every AI feature — completions, explanations, and refactors — is a request to Ollama. If Ollama crashes, restarts after an OOM event, or is not yet running when VS Code opens, Wingman shows no error and all AI features silently stop working. A 1-minute HTTP check catches crashes before any developer notices.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter: http://localhost:11434/ (or your LAN IP if Ollama runs on a shared server, e.g. http://192.168.1.x:11434/).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

Ollama returns Ollama is running as plain text on its root endpoint. A connection refused or non-200 means Ollama is down, starting up, or crashed.

Tip: If your team shares a single Ollama server, monitor it from a separate host using the LAN IP rather than localhost so that network connectivity is also tested.

Step 2: Heartbeat Monitor for Configured Model Availability

Wingman uses a user-specified model name from VS Code settings (e.g., wingman.modelName). If that exact model is not present in Ollama's local registry — perhaps because it was accidentally deleted, was never pulled on a new developer machine, or was replaced with a different tag — every Wingman request returns a 404-style error from Ollama that the extension surfaces as a generic failure.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 10 minutes.
3. Copy the heartbeat URL.

Create check_wingman_model.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_MODEL_HEARTBEAT_ID"
# Set this to the model name in your Wingman VS Code settings
WINGMAN_MODEL="${WINGMAN_MODEL:-llama3.1:8b}"

RESPONSE=$(curl -s http://localhost:11434/api/tags)

if echo "$RESPONSE" | grep -q "\"$WINGMAN_MODEL\""; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Wingman configured model '$WINGMAN_MODEL' not found in Ollama /api/tags"
  echo "Available: $RESPONSE"
fi

Schedule every 10 minutes:

*/10 * * * * /path/to/check_wingman_model.sh

If a developer accidentally removes the model or uses ollama rm during cleanup, the heartbeat stops and you receive an alert before the team discovers the silent failure.

Step 3: Heartbeat Monitor for Code Completion Trigger Latency

Wingman's inline code completion fires when the developer pauses typing. The suggestion must arrive within the VS Code inline completion timeout — typically 500 ms to 2 seconds — or it is discarded before the developer sees it. Local models at 7B parameters typically complete short prompts in 300–800 ms on a GPU; 13B+ models on CPU can take 2–5 seconds. Monitor P50/P95 latency and alert when it crosses the interaction threshold.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 5 minutes.
3. Copy the heartbeat URL.

Create check_wingman_completion_latency.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_COMPLETION_LATENCY_HEARTBEAT_ID"
MODEL="${WINGMAN_MODEL:-llama3.1:8b}"
LATENCY_THRESHOLD_MS=2000  # 2-second budget for inline completions

START_MS=$(date +%s%3N)

RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"prompt\":\"function calculateTotal(items) {\",\"stream\":false,\"options\":{\"num_predict\":16,\"temperature\":0}}")

END_MS=$(date +%s%3N)
ELAPSED_MS=$(( END_MS - START_MS ))

if echo "$RESPONSE" | grep -q '"response"' && [ "$ELAPSED_MS" -lt "$LATENCY_THRESHOLD_MS" ]; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Completion latency ${ELAPSED_MS}ms (threshold: ${LATENCY_THRESHOLD_MS}ms) or request failed"
fi

Schedule every 5 minutes:

*/5 * * * * /path/to/check_wingman_completion_latency.sh

Step 4: Heartbeat Monitor for Code Explanation Response Latency

Wingman can explain selected code on demand — the developer selects a block of code and asks Wingman to explain it. This is a synchronous chat-style request with a longer prompt than inline completion. Explanation latency is typically higher (1–8 seconds) and more visible to the developer since they are actively waiting for it. Alert when explanation latency is unacceptably high.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 10 minutes.
3. Copy the heartbeat URL.

Create check_wingman_explanation_latency.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_EXPLANATION_LATENCY_HEARTBEAT_ID"
MODEL="${WINGMAN_MODEL:-llama3.1:8b}"
LATENCY_THRESHOLD_MS=8000  # 8-second budget for code explanation

START_MS=$(date +%s%3N)

RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"prompt\":\"Explain what this code does in one sentence: const sum = arr.reduce((a,b) => a+b, 0);\",\"stream\":false,\"options\":{\"num_predict\":48}}")

END_MS=$(date +%s%3N)
ELAPSED_MS=$(( END_MS - START_MS ))

if echo "$RESPONSE" | grep -q '"response"' && [ "$ELAPSED_MS" -lt "$LATENCY_THRESHOLD_MS" ]; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Explanation latency ${ELAPSED_MS}ms (threshold: ${LATENCY_THRESHOLD_MS}ms)"
fi

Schedule every 10 minutes:

*/10 * * * * /path/to/check_wingman_explanation_latency.sh

Step 5: Heartbeat Monitor for Refactoring Suggestion Health

Wingman applies LLM-suggested code refactors by writing the modified code back to the file. A refactor that introduces a syntax error — from a model hallucinating an invalid code construct — or a file write that fails due to permissions creates a worse state than no refactor at all. Monitor file write success and run a basic syntax check on the output.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 15 minutes.
3. Copy the heartbeat URL.

Create check_wingman_refactor.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_REFACTOR_HEARTBEAT_ID"
MODEL="${WINGMAN_MODEL:-llama3.1:8b}"
TMPFILE="/tmp/wingman_refactor_test.js"

# Ask the model to refactor a simple function
RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"prompt\":\"Refactor this JavaScript function to use arrow syntax. Return only the code, no explanation: function add(a, b) { return a + b; }\",\"stream\":false,\"options\":{\"num_predict\":32}}")

CODE=$(echo "$RESPONSE" | python3 -c "import sys,json; r=json.load(sys.stdin); print(r.get('response',''))" 2>/dev/null)

if [ -z "$CODE" ]; then
  echo "Model returned empty response for refactor request"
  exit 1
fi

# Write the refactored code to a temp file and syntax-check with node
echo "$CODE" > "$TMPFILE"
if node --check "$TMPFILE" 2>/dev/null; then
  curl -s "$HEARTBEAT_URL"
  rm -f "$TMPFILE"
else
  echo "Refactored code failed syntax check: $CODE"
  rm -f "$TMPFILE"
fi

Schedule every 15 minutes:

*/15 * * * * /path/to/check_wingman_refactor.sh

Step 6: Heartbeat Monitor for VS Code Extension Activation Health

Wingman activates when VS Code opens a project containing code files. Activation failures — due to a crashed extension host, an incompatible VS Code version update, or a missing Node.js dependency — prevent all Wingman features from loading. The extension may appear installed in VS Code's extension panel while being non-functional.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 15 minutes.
3. Copy the heartbeat URL.

Create check_wingman_activation.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_ACTIVATION_HEARTBEAT_ID"

# Check that the VS Code extension host process is running
# On Linux/macOS: code --type=extensionHost or electron helper
if pgrep -f "extensionHost" > /dev/null; then
  curl -s "$HEARTBEAT_URL"
else
  echo "VS Code extension host process not found — Wingman may not be active"
fi

Note: Adjust pgrep for your OS. On macOS: pgrep -f "Code Helper". On Windows (Git Bash/WSL): use tasklist | grep -i code or query the VS Code output panel log file for the Wingman extension's activation message.

Schedule every 15 minutes:

*/15 * * * * /path/to/check_wingman_activation.sh

Step 7: Heartbeat Monitor for Multi-File Context Health

Wingman improves completion quality by including relevant project files in the context window passed to the LLM. This requires reading multiple files from disk. A project with thousands of files, large generated files, or slow network drives can cause context assembly to time out. Monitor that context assembly completes within a reasonable window.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 15 minutes.
3. Copy the heartbeat URL.

Create check_wingman_context.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_CONTEXT_HEARTBEAT_ID"
PROJECT_DIR="${WINGMAN_PROJECT_DIR:-$HOME/projects}"

# Build a large context by concatenating multiple source files
CONTEXT=$(find "$PROJECT_DIR" -name "*.js" -o -name "*.ts" -o -name "*.py" 2>/dev/null \
  | head -10 | xargs cat 2>/dev/null | head -c 4000)

if [ -n "$CONTEXT" ] && [ ${#CONTEXT} -gt 100 ]; then
  curl -s "$HEARTBEAT_URL"
else
  echo "Context assembly returned insufficient content (${#CONTEXT} chars) — check project path or file read access"
fi

Schedule every 15 minutes:

*/15 * * * * /path/to/check_wingman_context.sh

Step 8: Heartbeat Monitor for GPU/CPU Resource Utilization

Ollama running code models during peak developer hours consumes significant GPU VRAM and CPU cycles. A spike of simultaneous completion requests — triggered when the whole team starts their day and opens VS Code at the same time — can exhaust VRAM, causing Ollama to fall back to CPU inference, which is 5–10x slower. Alert on resource exhaustion before it becomes a latency problem.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 5 minutes.
3. Copy the heartbeat URL.

Create check_wingman_resources.sh:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_RESOURCE_HEARTBEAT_ID"
VRAM_THRESHOLD=88  # Alert if GPU VRAM exceeds 88%
CPU_THRESHOLD=90   # Alert if system CPU exceeds 90%

# GPU VRAM check (NVIDIA only; skip if nvidia-smi not available)
if command -v nvidia-smi &> /dev/null; then
  VRAM_USED=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits | head -1)
  VRAM_TOTAL=$(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits | head -1)
  VRAM_PCT=$(( VRAM_USED * 100 / VRAM_TOTAL ))
  if [ "$VRAM_PCT" -gt "$VRAM_THRESHOLD" ]; then
    echo "GPU VRAM at ${VRAM_PCT}% — above ${VRAM_THRESHOLD}% threshold"
    exit 1
  fi
fi

# CPU utilization check
CPU_IDLE=$(vmstat 1 2 | tail -1 | awk '{print $15}')
CPU_USED=$(( 100 - CPU_IDLE ))
if [ "$CPU_USED" -gt "$CPU_THRESHOLD" ]; then
  echo "CPU at ${CPU_USED}% — above ${CPU_THRESHOLD}% threshold"
  exit 1
fi

curl -s "$HEARTBEAT_URL"

Schedule every 5 minutes:

*/5 * * * * /path/to/check_wingman_resources.sh

Step 9: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, Discord, or a webhook.
2. On the Ollama backend monitor: set Consecutive failures before alert to 1 — Ollama crashes are hard and immediate.
3. On the model availability heartbeat: default single-missed-interval threshold is appropriate.
4. On the VS Code activation heartbeat: set Consecutive failures to 2 to tolerate brief VS Code restarts.
5. On GPU/CPU resource heartbeats: default single-missed-interval threshold works well since resource exhaustion is sudden.

Example: Slack Alert

1. Create a Slack incoming webhook in your workspace.
2. In Vigilmon → Alert Channels → Add → Slack.
3. Paste the webhook URL and assign it to all monitors in this tutorial.

Summary

| Monitor | Type | Target | Interval | |---------|------|--------|----------| | Ollama backend | HTTP | http://localhost:11434/ | 1 min | | Configured model availability | Heartbeat | probe script → Vigilmon | 10 min | | Completion trigger latency | Heartbeat | probe script → Vigilmon | 5 min | | Code explanation latency | Heartbeat | probe script → Vigilmon | 10 min | | Refactoring suggestion health | Heartbeat | probe script → Vigilmon | 15 min | | VS Code extension activation | Heartbeat | probe script → Vigilmon | 15 min | | Multi-file context health | Heartbeat | probe script → Vigilmon | 15 min | | GPU/CPU resource utilization | Heartbeat | probe script → Vigilmon | 5 min |

Wingman AI delivers Copilot-quality assistance without the privacy trade-off — but only when the local infrastructure stays healthy. A downed Ollama instance, a missing model, or a GPU VRAM crunch silently degrades every developer's experience. With Vigilmon watching each layer, your team can trust that AI-assisted development is always a tool call away.

Get started for free at vigilmon.online.