Monitoring FauxPilot with Vigilmon

FauxPilot is an open-source, self-hosted GitHub Copilot alternative. It wraps the Salesforce CodeGen models served by Triton Inference Server behind a Copilot-protocol-compatible API — meaning existing IDE Copilot plugins can be redirected to your own private server instead of GitHub's. FauxPilot's Python API runs on port 5000; Triton handles the heavy GPU inference behind it. Vigilmon monitors both layers — plus the GPU health and SSL certificate — so you know the moment a developer's completions go dark.

What You'll Set Up

• FauxPilot API server availability monitor (port 5000)
• Triton Inference Server health check (/v2/health/ready)
• Code completion endpoint health (POST /v1/engines/codegen/completions)
• Triton Prometheus metrics endpoint monitoring (port 8002)
• SSL certificate expiry alerts
• GPU health heartbeat for the Triton CUDA backend

Prerequisites

• FauxPilot stack running via Docker Compose (API on port 5000, Triton on port 8000/8002)
• Triton serving the CodeGen model successfully
• A free Vigilmon account

Step 1: Monitor the FauxPilot API Server

The FauxPilot Python API (port 5000) is the entry point for all IDE requests. It validates the Copilot protocol format, forwards to Triton, and streams responses back. If this process crashes or hangs, every developer's Copilot plugin silently stops receiving completions.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter the FauxPilot API URL:

   http://your-host:5000/
   

   Or if behind a TLS reverse proxy:

   https://copilot.yourdomain.com/
   

4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

Tip: FauxPilot does not expose a dedicated /health route by default. The root path or a GET to /v1/engines can be used to confirm the API process is responsive.

Step 2: Monitor the Triton Inference Server

Triton Inference Server is where the actual CodeGen model runs. FauxPilot is just a thin proxy layer — if Triton is down or unready, every completion request FauxPilot forwards will fail. Triton exposes a readiness endpoint at /v2/health/ready.

1. Click Add Monitor → HTTP / HTTPS.
2. Enter the Triton health endpoint:

   http://triton-host:8000/v2/health/ready
   

   (Replace triton-host with the Docker service name, container IP, or LAN IP depending on your deployment.)
3. Set Method to GET.
4. Set Expected HTTP status to 200.
5. Set Check interval to 1 minute.
6. Click Save.

Triton returns 200 OK only when the model repository is loaded and the server is ready to serve inference requests. A 503 means the model is still loading or failed to load; connection refused means Triton crashed.

Step 3: Monitor the Code Completion Endpoint

POST /v1/engines/codegen/completions is the primary endpoint IDE Copilot plugins call. Monitor it end-to-end — including the Triton inference path — to detect failures that may not surface as Triton health check errors (e.g. a model loaded but returning corrupted outputs).

1. Click Add Monitor → HTTP / HTTPS.
2. Enter:

   http://your-host:5000/v1/engines/codegen/completions
   

3. Set Method to POST.
4. Under Request headers, add:

   Content-Type: application/json
   

5. Under Request body, enter a minimal completion request:

   {
     "model": "fastertransformer",
     "prompt": "def hello_world():\n    ",
     "max_tokens": 16,
     "temperature": 0.1,
     "n": 1
   }
   

6. Set Expected HTTP status to 200.
7. Set Check interval to 2 minutes.
8. Click Save.

This probe exercises the full request path from the FauxPilot API through Triton and back. A 200 with a valid JSON body confirms end-to-end completion is working.

Step 4: Monitor Triton's Prometheus Metrics Endpoint

Triton exposes detailed performance metrics — requests per second, queue latency, inference latency — on its Prometheus endpoint at port 8002. Monitoring this endpoint confirms Triton's metrics pipeline is healthy and, more importantly, that Triton is running and reachable on its metrics port.

1. Click Add Monitor → HTTP / HTTPS.
2. Enter:

   http://triton-host:8002/metrics
   

3. Set Method to GET.
4. Set Expected HTTP status to 200.
5. Optionally, set Response must contain to nv_inference_request_success to verify Triton has processed at least one successful inference.
6. Set Check interval to 5 minutes.
7. Click Save.

Step 5: GPU Health Heartbeat

Triton runs CodeGen on GPU. A CUDA out-of-memory error, GPU fan failure, or driver crash can take down inference while the Triton process stays alive. Add a heartbeat driven by a GPU health probe.

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 5 minutes.
3. Copy the heartbeat URL.

Create check_gpu.sh on the host running Triton:

#!/bin/bash
# Check GPU availability and VRAM headroom
GPU_OK=$(nvidia-smi --query-gpu=gpu_name --format=csv,noheader 2>/dev/null | head -1)
if [ -z "$GPU_OK" ]; then
  echo "nvidia-smi failed — GPU unavailable"
  exit 1
fi

# Check GPU is not at full VRAM (leave at least 512MB free)
FREE_MEM=$(nvidia-smi --query-gpu=memory.free --format=csv,noheader,nounits | head -1 | tr -d ' ')
if [ "$FREE_MEM" -lt 512 ]; then
  echo "GPU VRAM critically low: ${FREE_MEM}MB free"
  exit 1
fi

curl -s "https://vigilmon.online/heartbeat/YOUR_GPU_HEARTBEAT_ID"

Schedule every 5 minutes:

*/5 * * * * /path/to/check_gpu.sh

If the GPU goes unavailable or VRAM fills, the heartbeat stops and Vigilmon alerts before Triton starts failing inference requests.

Step 6: Model Loading Heartbeat

The CodeGen model must be present in the Triton model repository at startup. If the model files are missing, corrupted, or the repository path is misconfigured, Triton starts but Triton's health endpoint may still return 200 while /v2/models/fastertransformer/ready returns 404.

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set Expected ping interval to 15 minutes.
3. Copy the heartbeat URL.

Extend the probe to check the model-specific readiness endpoint:

#!/bin/bash
MODEL_READY=$(curl -s -o /dev/null -w "%{http_code}" \
  http://localhost:8000/v2/models/fastertransformer/ready)

if [ "$MODEL_READY" = "200" ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_MODEL_HEARTBEAT_ID"
else
  echo "Triton model not ready: HTTP $MODEL_READY"
fi

Step 7: Monitor Inference Latency

FauxPilot's end-to-end latency is the developer experience metric. A latency spike above ~2 seconds makes Copilot feel broken even when technically functional. Use a heartbeat probe that measures the round-trip and only pings Vigilmon when within your SLO.

#!/bin/bash
THRESHOLD_MS=2000
START=$(date +%s%3N)
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -X POST http://localhost:5000/v1/engines/codegen/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"fastertransformer","prompt":"def ","max_tokens":8,"n":1}' \
  --max-time 15)
END=$(date +%s%3N)
ELAPSED=$((END - START))

if [ "$STATUS" = "200" ] && [ "$ELAPSED" -lt "$THRESHOLD_MS" ]; then
  curl -s "https://vigilmon.online/heartbeat/YOUR_LATENCY_HEARTBEAT_ID"
else
  echo "Latency check failed: HTTP=$STATUS elapsed=${ELAPSED}ms"
fi

Schedule every 5 minutes.

Step 8: SSL Certificate Monitoring

If FauxPilot is behind a reverse proxy (nginx, Caddy) with TLS, monitor the certificate to catch renewal failures before IDE plugins start reporting SSL errors.

1. Open the FauxPilot server HTTP monitor from Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

Step 9: Configure Alert Channels

1. Go to Alert Channels in Vigilmon → Add → Slack, email, Discord, or webhook.
2. Assign the channel to all FauxPilot monitors.
3. Tune thresholds:
   • FauxPilot API (Step 1): 2 consecutive failures — allows for transient restarts.
   • Triton health (Step 2): 1 consecutive failure — Triton going down is immediately user-visible.
   • Completion endpoint (Step 3): 1 consecutive failure — same reasoning.
   • GPU / latency heartbeats: default missed-interval alert.

Summary

| Monitor | Type | URL / Target | Interval | |---------|------|-------------|----------| | FauxPilot API server | HTTP | http://host:5000/ | 1 min | | Triton readiness | HTTP GET | :8000/v2/health/ready | 1 min | | Completion endpoint | HTTP POST | :5000/v1/engines/codegen/completions | 2 min | | Triton Prometheus metrics | HTTP GET | :8002/metrics | 5 min | | GPU health | Heartbeat | probe script → Vigilmon | 5 min | | Triton model readiness | Heartbeat | probe script → Vigilmon | 15 min | | Inference latency SLO | Heartbeat | probe script → Vigilmon | 5 min | | SSL certificate | SSL (on HTTP monitor) | reverse proxy domain | — |

FauxPilot's two-layer architecture — a Python API in front of Triton — means failures can occur at either layer. With Vigilmon covering both, plus the GPU and model state beneath them, you have full observability from the IDE plugin's perspective to the CUDA device doing the inference.

Get started for free at vigilmon.online.