Monitoring Ollama with Vigilmon: LLM Server Health, Model Loading Checks & Inference Heartbeats

Ollama makes running large language models locally as simple as ollama run llama3. But "simple to start" doesn't mean it stays running: Ollama can exhaust VRAM, stall during a long inference job, or lose its connection to a GPU driver after a kernel update. Applications that depend on Ollama — Open WebUI, LiteLLM, custom scripts — fail silently when the server goes down. Vigilmon monitors Ollama's health endpoint, tracks which models are loaded, and uses heartbeats to catch long-running inference jobs that stall without any error.

What You'll Build

• An HTTP monitor on Ollama's /api/health endpoint
• A model availability check via /api/tags
• SSL certificate monitoring for reverse-proxied Ollama deployments
• A heartbeat monitor for long-running inference jobs
• Alerting that distinguishes server crashes from model-loading failures

Prerequisites

• Ollama running (default port 11434)
• Optionally: Ollama exposed via reverse proxy (Caddy/nginx) with HTTPS
• A free account at vigilmon.online

Step 1: Check Ollama's Health Endpoint

Ollama exposes a /api/health endpoint (available since Ollama 0.1.23) that returns {"status":"ok"} when the server is running and ready to accept requests:

curl http://localhost:11434/api/health
# {"status":"ok"}

If Ollama is behind a reverse proxy with HTTPS:

curl https://ollama.yourdomain.com/api/health

The root path also works as a lightweight check and returns a plain-text "Ollama is running":

curl http://localhost:11434/
# Ollama is running

Use /api/health for monitoring since it's the designated health endpoint and returns structured JSON.

Step 2: Create the Health Monitor in Vigilmon

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: https://ollama.yourdomain.com/api/health (or http://YOUR-SERVER-IP:11434/api/health if not reverse-proxied).
3. Check interval: 60 seconds.
4. Response timeout: 10 seconds.
5. Expected status: 200.
6. Keyword: ok (matches the {"status":"ok"} response).
7. Click Save.

Security note: Ollama's default configuration binds to localhost only and has no authentication. If you're monitoring directly via IP, ensure your Vigilmon agent or monitor source IP is trusted. For public deployments, always put Ollama behind a reverse proxy with authentication.

Step 3: Monitor Model Availability with /api/tags

The health endpoint confirms Ollama is running, but it doesn't verify that models are actually loaded and available. A freshly restarted Ollama might be healthy but still pulling model weights from the internet. The /api/tags endpoint lists all locally available models:

curl http://localhost:11434/api/tags
# {"models":[{"name":"llama3:latest","size":4661224676,...}]}

Add a monitor for this endpoint:

1. Add Monitor → HTTP.
2. URL: https://ollama.yourdomain.com/api/tags
3. Expected status: 200.
4. Keyword: models (present in the JSON response when the endpoint is functioning).
5. Check interval: 120 seconds.

If you depend on a specific model, you can also keyword-match its name (e.g., llama3) to catch cases where the model was accidentally deleted or its symlink broke.

Step 4: Monitor SSL Certificate for Reverse-Proxied Deployments

If Ollama is exposed to the network via a reverse proxy (required for multi-user or remote access deployments), monitor the SSL certificate. An expired certificate blocks all API clients — Open WebUI, custom scripts, and remote tools all fail simultaneously.

1. Add Monitor → SSL Certificate.
2. Domain: ollama.yourdomain.com
3. Alert when expiry is within: 30 days.
4. Alert again: 14 days, 7 days, 3 days.

This is especially important for Ollama deployments because clients often don't show clear SSL error messages — they just fail with a generic "connection error."

Step 5: Set Up Heartbeat Monitoring for Long-Running Inference

Ollama can handle inference jobs that take minutes for large models or long prompts. During these jobs, Ollama is technically running but fully occupied. If an inference job stalls — common when a model partially exhausts VRAM and starts thrashing — the server becomes unresponsive without crashing.

Use a cron job on the Ollama host to run a quick inference probe:

#!/bin/bash
# /etc/cron.d/ollama-heartbeat
# Runs every 5 minutes

# Quick inference probe with a tiny model and short prompt
# Use your smallest available model (tinyllama, phi3:mini, etc.)
RESULT=$(curl -s -X POST http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{"model":"tinyllama","prompt":"1+1=","stream":false}' \
  --max-time 60 2>&1)

if echo "$RESULT" | grep -q '"response"'; then
  curl -fsS -X POST https://vigilmon.online/api/heartbeat/YOUR-HEARTBEAT-ID
fi

Create a Heartbeat monitor in Vigilmon:

1. Add Monitor → Heartbeat.
2. Expected interval: 5 minutes.
3. Grace period: 15 minutes.
4. Alert: when heartbeat is missed.

The health endpoint stays green even when Ollama is stalled on a hung inference — only the heartbeat catches this. Use tinyllama or another small model for the probe so it completes in seconds and doesn't interfere with production inference load.

Step 6: Monitor GPU Health via Heartbeat

GPU-related failures (VRAM exhaustion, driver issues, CUDA errors) often manifest as Ollama hanging rather than crashing. Add a GPU check to your heartbeat script:

#!/bin/bash
# Enhanced heartbeat with GPU check

# Check GPU availability (NVIDIA)
GPU_OK=$(nvidia-smi --query-gpu=gpu_name --format=csv,noheader 2>/dev/null | head -1)

# Check Ollama inference
INFERENCE_OK=$(curl -s -X POST http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{"model":"tinyllama","prompt":"ping","stream":false}' \
  --max-time 45 2>/dev/null | grep -c '"response"')

if [ -n "$GPU_OK" ] && [ "$INFERENCE_OK" -gt "0" ]; then
  curl -fsS -X POST https://vigilmon.online/api/heartbeat/YOUR-HEARTBEAT-ID
fi

This heartbeat only fires when both the GPU is accessible and inference is actually completing — a complete health signal for the entire Ollama stack.

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, wire up your alert channels:

| Monitor | Trigger | First action | |---|---|---| | /api/health | Non-200 or keyword missing | systemctl restart ollama; check journalctl -u ollama | | /api/tags | Non-200 | Ollama is running but model storage may be corrupted; ollama list | | SSL certificate | < 30 days | certbot renew or check Caddy ACME logs | | Inference heartbeat | Missed > 15 min | Check ollama ps; look for stalled requests; consider ollama stop <model> |

Alert after 1 consecutive failure for the inference heartbeat — a missed heartbeat is a meaningful signal. Use 2 consecutive failures for the HTTP monitors.

Common Ollama Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Ollama process crashes | Health endpoint fails within 60 s | | VRAM exhausted → inference stalls | Heartbeat stops; health endpoint stays green | | Model deleted or symlink broken | /api/tags check may still pass; keyword check for specific model catches it | | GPU driver update breaks CUDA | Heartbeat stops; health endpoint may stay green | | SSL certificate expires | SSL monitor alerts at 30 days | | Reverse proxy drops connection | Health monitor fires; direct port check may pass | | ollama serve not auto-started after reboot | Health endpoint fails immediately after restart |

Ollama makes local LLMs accessible to everyone, but self-hosting a GPU-accelerated AI server introduces failure modes that typical web app monitoring doesn't cover: VRAM exhaustion, inference stalls, and CUDA driver issues. Vigilmon's combination of endpoint monitoring and heartbeat checks gives you full coverage — catching both the crashes that happen immediately and the silent stalls that happen gradually.

Start monitoring Ollama in under 5 minutes — register free at vigilmon.online.