Monitoring EasyDiffusion with Vigilmon: Web UI Availability, GPU Health & Image Generation Queue Checks

EasyDiffusion brings Stable Diffusion image generation to anyone with a consumer GPU — no Python environment setup, no command-line expertise, just a one-click installer and a browser. The FastAPI backend on port 9000 handles text-to-image, image-to-image, inpainting, and upscaling tasks through a clean web UI. But self-hosted image generation servers have hardware-specific failure modes: GPU VRAM exhaustion stalls the generation queue without crashing the server, Stable Diffusion checkpoint files can silently fail to load after an update, and generated images fill up disk space faster than expected. Vigilmon monitors EasyDiffusion's availability, detects queue stalls, and uses heartbeats to confirm end-to-end GPU inference is healthy.

What You'll Build

• An HTTP monitor on EasyDiffusion's web server (port 9000)
• An API health check for the backend task endpoint
• A GPU/VRAM and generation heartbeat
• Disk space monitoring via a server-side script
• SSL certificate monitoring for LAN/reverse-proxy deployments
• Alerting rules for queue stalls and model-loading failures

Prerequisites

• EasyDiffusion installed and running (default port 9000)
• At least one Stable Diffusion checkpoint model loaded
• A free account at vigilmon.online

Step 1: Verify EasyDiffusion Is Running

EasyDiffusion's FastAPI backend serves both the web UI and the generation API on port 9000. Confirm it is accessible:

# Check if the web UI is reachable
curl -s -o /dev/null -w "%{http_code}" http://localhost:9000/
# 200

# Check the API ping endpoint
curl -s http://localhost:9000/ping
# {"status":"OK"}

EasyDiffusion also exposes image generation via:

curl -s http://localhost:9000/image \
  -H "Content-Type: application/json" \
  -d '{"prompt":"a blue circle","seed":42,"num_outputs":1}'

Use the /ping endpoint for health monitoring — it is fast, stateless, and doesn't trigger model loading.

Step 2: Create the Web Server Availability Monitor

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: http://YOUR-SERVER-IP:9000/ping (or your reverse-proxy HTTPS URL).
3. Check interval: 60 seconds.
4. Response timeout: 10 seconds.
5. Expected status: 200.
6. Keyword: OK (matches the {"status":"OK"} ping response).
7. Click Save.

If EasyDiffusion crashes or its Python/uvicorn process dies, this monitor fires within one check cycle. It also catches port binding failures caused by process restarts that leave the port briefly unavailable.

Step 3: Monitor the Generation Task API

The /ping endpoint confirms the server is up but doesn't verify that the model is loaded or that the generation pipeline is functional. Add a more complete check against EasyDiffusion's task API:

# Check the render task endpoint (GET returns available options, not a generation)
curl -s http://localhost:9000/render

In Vigilmon:

1. Add Monitor → HTTP.
2. URL: http://YOUR-SERVER-IP:9000/render
3. Method: GET.
4. Expected status: 200.
5. Keyword: prompt (present in the render options response).
6. Check interval: 120 seconds.

This catches cases where EasyDiffusion is running but the generation backend is misconfigured or the model hasn't loaded successfully after a restart.

Step 4: Set Up Heartbeat Monitoring for GPU Inference

VRAM exhaustion is the most common EasyDiffusion failure. The server stays responsive (ping returns 200) while the generation queue is backed up with stuck tasks waiting for VRAM to free up. Use a cron heartbeat that submits a real generation task and waits for it to complete:

#!/bin/bash
# /etc/cron.d/easydiffusion-heartbeat — runs every 10 minutes

# Submit a tiny image generation (64x64 is fast even on CPU)
TASK_ID=$(curl -s -X POST http://localhost:9000/render \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "a blue circle",
    "negative_prompt": "",
    "seed": 42,
    "width": 64,
    "height": 64,
    "num_outputs": 1,
    "num_inference_steps": 5,
    "guidance_scale": 7.5,
    "sampler_name": "euler_a",
    "use_stable_diffusion_model": "sd-v1-5"
  }' | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('task_id',''))" 2>/dev/null)

if [ -z "$TASK_ID" ]; then
  exit 1
fi

# Poll for task completion (max 90 seconds)
for i in $(seq 1 18); do
  sleep 5
  STATUS=$(curl -s "http://localhost:9000/task/$TASK_ID" | \
    python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('status',''))" 2>/dev/null)
  if [ "$STATUS" = "succeeded" ]; then
    curl -fsS -X POST https://vigilmon.online/api/heartbeat/YOUR-HEARTBEAT-ID
    exit 0
  fi
done
# Task didn't complete in 90 seconds — no heartbeat ping

Create a Heartbeat monitor in Vigilmon:

1. Add Monitor → Heartbeat.
2. Expected interval: 10 minutes.
3. Grace period: 5 minutes.
4. Alert: when heartbeat is missed.

Use the fewest inference steps that still exercise the full pipeline (5 steps is sufficient for health monitoring). Keep the resolution small (64×64) so the check completes in seconds on GPU and under a minute on CPU.

Step 5: Monitor VRAM and GPU Availability

Add a VRAM check to detect GPU issues before they stall the queue:

#!/bin/bash
# Enhanced heartbeat with GPU/VRAM check

# Check GPU free VRAM — SD 1.5 needs ~2 GB; SDXL needs ~6+ GB
FREE_VRAM=$(nvidia-smi --query-gpu=memory.free --format=csv,noheader,nounits 2>/dev/null | head -1)

if [ -z "$FREE_VRAM" ] || [ "$FREE_VRAM" -lt "1500" ]; then
  # GPU unavailable or VRAM critically low
  exit 1
fi

# Generation probe
TASK_ID=$(curl -s -X POST http://localhost:9000/render \
  -H "Content-Type: application/json" \
  -d '{"prompt":"a circle","seed":1,"width":64,"height":64,"num_inference_steps":5}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin).get('task_id',''))" 2>/dev/null)

[ -z "$TASK_ID" ] && exit 1

for i in $(seq 1 12); do
  sleep 5
  STATUS=$(curl -s "http://localhost:9000/task/$TASK_ID" | \
    python3 -c "import sys,json; print(json.load(sys.stdin).get('status',''))" 2>/dev/null)
  [ "$STATUS" = "succeeded" ] && {
    curl -fsS -X POST https://vigilmon.online/api/heartbeat/YOUR-HEARTBEAT-ID
    exit 0
  }
done

Step 6: Monitor Disk Space for Generated Images

EasyDiffusion saves all generated images to disk. High-resolution outputs at 512×512 to 1024×1024 at batch sizes of 4–16 fill storage quickly, especially if automatic cleanup isn't configured. Add a disk space check to the heartbeat:

#!/bin/bash
# Disk space check for EasyDiffusion output directory
OUTPUT_DIR="${HOME}/EasyDiffusion/output"  # adjust to your install path

USED_PERCENT=$(df "$OUTPUT_DIR" | awk 'NR==2 {print $5}' | tr -d '%')

if [ "$USED_PERCENT" -gt "85" ]; then
  # Disk 85%+ full — skip heartbeat to trigger alert
  exit 1
fi

# ... rest of generation probe

Alternatively, create a dedicated Vigilmon Heartbeat for disk health that runs a separate cron job and only pings when disk usage is under the threshold.

Step 7: Monitor SSL Certificate for LAN/Reverse-Proxy Deployments

If EasyDiffusion is accessible across your LAN or via a reverse proxy with HTTPS, monitor the SSL certificate — an expired cert blocks all browser access:

1. Add Monitor → SSL Certificate.
2. Domain: easydiffusion.yourdomain.com (or your reverse proxy hostname).
3. Alert when expiry is within: 30 days.
4. Alert escalation: 14 days, 7 days.

Step 8: Configure Alerting

Wire up alert channels in Vigilmon under Settings → Notifications:

| Monitor | Trigger | First action | |---|---|---| | /ping | Non-200 or OK missing | Restart EasyDiffusion (./start.sh); check terminal output | | /render GET | Non-200 | Backend model pipeline issue; check EasyDiffusion logs | | Generation heartbeat | Missed > 15 min | Check VRAM usage (nvidia-smi); look for stuck tasks in UI; consider restarting | | Disk space | Heartbeat missed | Clear old outputs from EasyDiffusion's output folder | | SSL certificate | < 30 days | Renew via Caddy ACME or certbot renew |

Alert after 2 consecutive failures for HTTP monitors. Alert after 1 missed heartbeat for the generation probe.

Common EasyDiffusion Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | EasyDiffusion Python process crashes | /ping fails within 60 s | | VRAM exhausted → queue stalls | Generation heartbeat stops; /ping stays green | | SD model fails to load after update | /render GET check fails or generation heartbeat stalls | | Output disk full → saves fail silently | Disk space heartbeat stops | | GPU driver broken after kernel update | GPU heartbeat fails; HTTP endpoints may still respond | | SSL certificate expires | SSL monitor alerts at 30 days | | EasyDiffusion not auto-started after reboot | /ping fails immediately |

EasyDiffusion makes Stable Diffusion accessible without technical setup, but self-hosting a GPU-accelerated image generation server introduces failure modes that standard web monitoring misses: VRAM exhaustion, model-loading failures, and silent disk full errors. Vigilmon's combination of endpoint checks and heartbeat monitoring covers the complete health signal — from server availability to end-to-end generation success.

Start monitoring EasyDiffusion in under 5 minutes — register free at vigilmon.online.