Monitoring Jan AI with Vigilmon

Jan is an open-source local-first AI assistant that pairs a polished Electron desktop UI with a headless server mode — powered by Cortex, its C++/Rust inference engine that wraps llama.cpp and TensorRT-LLM. When running in server mode (port 1337), Jan exposes a full OpenAI-compatible REST API, making it easy to integrate with any tool that speaks the OpenAI API format.

But Jan's architecture has layered failure modes: the Jan API server can be healthy while Cortex isn't running, or Cortex can be running while the currently loaded model is unresponsive. Vigilmon lets you monitor every layer — from the /healthz liveness endpoint to model-level readiness and end-to-end inference availability.

What You'll Set Up

• HTTP uptime monitor for the Jan API server health endpoint (port 1337)
• REST API availability check for the /api/v1/models endpoint
• Model-level serving status check for a specific loaded model
• SSL certificate expiry alerts for HTTPS-proxied Jan server deployments
• Heartbeat monitor for the Cortex inference engine via a live completion probe

Prerequisites

• Jan running in server mode on Linux, macOS, or Windows (accessible at http://your-server:1337)
• A free Vigilmon account

To start Jan in server mode (headless):

jan server --port 1337

Or via the Jan Cortex CLI:

cortex server start

Step 1: Monitor the Jan API Server Health Endpoint

Jan exposes a /healthz endpoint that returns {"status": "ok"} when the server is running and accepting connections. This is your primary liveness signal:

GET http://your-server-ip:1337/healthz

Response:

{ "status": "ok" }

Add a monitor:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter the URL: http://your-server-ip:1337/healthz
4. Set Expected HTTP status to 200.
5. Set Response body contains to "ok".
6. Set Check interval to 1 minute.
7. Click Save.

If the Jan server process crashes or the port binding drops, this monitor fires within 1 minute.

Step 2: Monitor the /api/v1/models Endpoint

The /api/v1/models endpoint returns the list of models available in Jan's model hub. A successful response confirms the Jan API is serving model data and the Cortex backend is responsive:

GET http://your-server-ip:1337/api/v1/models

Example response:

{
  "object": "list",
  "data": [
    {
      "id": "mistral-ins-7b-q4",
      "object": "model",
      "created": 1715000000,
      "owned_by": "jan-hq"
    }
  ]
}

Add a monitor:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:1337/api/v1/models
3. Expected HTTP status: 200
4. Response body contains: "data"
5. Check interval: 1 minute
6. Click Save.

The body check on "data" confirms you're receiving a valid model list — not a proxy error page with a 200 status.

Step 3: Monitor a Specific Model's Serving Status

Jan's /api/v1/models/:model_id endpoint returns the status of a specific model. This lets you verify that your primary model is loaded and available for inference, not just that the API layer is responding:

GET http://your-server-ip:1337/api/v1/models/mistral-ins-7b-q4

Example response for a loaded model:

{
  "id": "mistral-ins-7b-q4",
  "object": "model",
  "engine": "cortex.llamacpp",
  "settings": {
    "ctx_len": 4096,
    "ngl": 32
  }
}

Add a monitor for your primary model:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:1337/api/v1/models/YOUR_MODEL_ID
3. Expected HTTP status: 200
4. Response body contains: "engine"
5. Check interval: 2 minutes
6. Click Save.

Replace YOUR_MODEL_ID with the model ID from your Jan installation (e.g. mistral-ins-7b-q4). You can find it via the /api/v1/models response or in the Jan UI under Models.

Step 4: SSL Certificate Alerts for HTTPS-Proxied Deployments

If you've placed Jan's API server behind an nginx or Caddy reverse proxy with TLS, add certificate expiry monitoring:

1. Create an additional HTTP / HTTPS monitor for your HTTPS URL: https://jan-api.yourdomain.com/healthz
2. Enable Monitor SSL certificate in the SSL section.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

A typical Caddy config for Jan API (Caddy auto-manages certificates via Let's Encrypt):

jan-api.yourdomain.com {
    reverse_proxy localhost:1337 {
        header_up Host {host}
    }
    # Increase timeout for long LLM generation requests
    request_timeout 300s
}

For nginx:

server {
    listen 443 ssl;
    server_name jan-api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/jan-api.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/jan-api.yourdomain.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:1337;
        proxy_read_timeout 300s;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Step 5: Heartbeat Monitoring for the Cortex Inference Engine

Jan runs the Cortex inference engine as a subprocess. The Jan API server can appear healthy — /healthz returns 200, /api/v1/models lists your models — while Cortex is deadlocked, OOM-killed, or stuck. The only reliable way to detect this is to actually attempt an inference request.

Use a Vigilmon cron heartbeat with a lightweight completion probe:

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set the expected ping interval to 10 minutes.
3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/YOUR_KEY

Create a script that sends a minimal test prompt to Jan's chat completions endpoint and pings Vigilmon only on success:

#!/bin/bash
# jan-cortex-check.sh
# Run via cron every 5 minutes

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_KEY"
JAN_URL="http://localhost:1337/api/v1/chat/completions"
MODEL_ID="mistral-ins-7b-q4"  # Replace with your model ID

RESPONSE=$(curl -s --max-time 60 -X POST "$JAN_URL" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$MODEL_ID\",
    \"messages\": [{\"role\": \"user\", \"content\": \"Hi\"}],
    \"max_tokens\": 1,
    \"stream\": false
  }" 2>/dev/null)

STATUS=$?

# Check curl succeeded and response contains a completion choice
if [ $STATUS -eq 0 ] && echo "$RESPONSE" | grep -q '"choices"'; then
    curl -s "$HEARTBEAT_URL"
fi

Add to crontab:

*/5 * * * * /home/user/jan-cortex-check.sh

This end-to-end probe validates the full inference path: Jan API → Cortex subprocess → model → response. If Cortex deadlocks or the model fails to generate a response within 60 seconds, the heartbeat stops and Vigilmon alerts after 10 minutes.

Step 6: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, Discord, email, or a webhook.
2. Assign the alert channel to all monitors.
3. Set Consecutive failures before alert to 2 on the /healthz monitor — a brief restart during model loading won't immediately page you.
4. For the model-specific monitor, set consecutive failures to 1 — if your primary model is unavailable, you want to know immediately.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Health endpoint | /healthz | Jan server crash, port binding failure | | Models list | /api/v1/models | API layer failure, Cortex not running | | Model status | /api/v1/models/:model_id | Specific model unavailable | | SSL certificate | HTTPS domain | Let's Encrypt renewal failure | | Cron heartbeat | Chat completions endpoint | Cortex deadlock, model inference failure |

Jan gives you a polished local AI experience with a production-grade API — self-hosted means you own the uptime. With Vigilmon watching every layer from the health endpoint to actual Cortex inference, you'll know the moment Jan stops being useful, not just when it stops responding.

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.