tutorial

Monitoring ComfyUI with Vigilmon

ComfyUI is a powerful node-based Stable Diffusion workflow engine — but when the web UI goes down, the GPU memory fills up, or the prompt queue silently stalls, you lose your entire image generation pipeline. Here's how to monitor ComfyUI's web interface, REST API, and queue state with Vigilmon.

ComfyUI is a powerful open-source node-based GUI for Stable Diffusion and AI image generation. Built in Python with a modular workflow graph interface, it supports SD 1.5, SDXL, SD3, Flux, ControlNet, LoRA, and a rich ecosystem of custom nodes — making it the platform of choice for AI artists, researchers, and developers building complex image generation pipelines. But ComfyUI runs on your own hardware, and self-hosting it means owning every failure mode: the Python server can crash, VRAM can fill up and freeze inference, or the prompt queue can stall silently with pending jobs that never process. Vigilmon gives you HTTP uptime monitors for the web interface and REST API endpoints, SSL certificate alerts for proxied deployments, and a heartbeat check on the prompt queue to catch frozen generation without any manual intervention.

What You'll Set Up

  • HTTP uptime monitor for the ComfyUI web interface (port 8188)
  • /system_stats REST API monitor for GPU/CPU memory state
  • /history API availability check for generation history
  • SSL certificate expiry alerts for HTTPS-proxied deployments
  • Cron heartbeat that monitors the /queue endpoint for stuck pending jobs

Prerequisites

  • ComfyUI running on a server or workstation (port 8188 by default)
  • ComfyUI started with --listen 0.0.0.0 if monitoring from a remote system
  • A free Vigilmon account

Step 1: Monitor the ComfyUI Web Interface

The ComfyUI frontend serves the workflow graph editor on port 8188. Add a Vigilmon HTTP monitor to confirm the server is reachable:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: http://your-server-ip:8188 (or https://comfyui.yourdomain.com if reverse-proxied).
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

ComfyUI serves its full node editor at the root path; a 200 response confirms the Python HTTP server is alive and the static assets are being served.

Step 2: Monitor /system_stats for GPU and CPU Memory

ComfyUI exposes a /system_stats REST endpoint that returns GPU VRAM usage, system RAM, and other runtime metrics. Monitoring it confirms that ComfyUI's API layer is functional and gives you a signal before VRAM saturation causes inference to freeze:

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter the URL: http://your-server-ip:8188/system_stats
  3. Set Method to GET.
  4. Set Expected HTTP status to 200.
  5. Under Response body check, enter "devices" to confirm the GPU device list is returned.
  6. Set Check interval to 2 minutes.
  7. Click Save.

The response looks like:

{
  "system": { "os": "posix", "ram_total": 32768, "ram_free": 8192 },
  "devices": [{ "name": "NVIDIA RTX 4090", "vram_total": 24576, "vram_free": 18000 }]
}

A 500 error or missing devices key here indicates that ComfyUI's backend is unhealthy even if the frontend loads.

Step 3: Monitor the /history API Endpoint

The /history endpoint returns recent generation history. It exercises the ComfyUI database and confirms that completed jobs are being persisted:

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter the URL: http://your-server-ip:8188/history
  3. Set Method to GET.
  4. Set Expected HTTP status to 200.
  5. Set Check interval to 5 minutes.
  6. Click Save.

No response body check is needed here — any valid JSON 200 response confirms the history API is operational.

Step 4: SSL Certificate Alerts for HTTPS Deployments

ComfyUI does not handle TLS itself; HTTPS deployments use nginx or Caddy as a reverse proxy. An expired certificate will reject all browser connections and API clients without any alert unless you monitor it explicitly.

  1. Open the HTTP monitor for your proxied domain (e.g. https://comfyui.yourdomain.com) created in Step 1.
  2. Enable Monitor SSL certificate under the SSL section.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

For Caddy users, verify the certificate is managed:

caddy list-certificates

For Let's Encrypt via certbot:

certbot certificates
systemctl status certbot.timer

Step 5: Heartbeat Monitoring for the Prompt Queue

The ComfyUI prompt queue can enter a stuck state where jobs are listed as pending but never process — this happens when a model crashes mid-inference, a custom node raises an unhandled exception, or VRAM is fully saturated. The /queue endpoint exposes pending and running job counts; you can use it in a heartbeat probe that alerts when pending jobs have been stuck for too long.

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Name it ComfyUI queue health probe.
  3. Set the expected ping interval to 10 minutes.
  4. Copy the heartbeat URL (e.g. https://vigilmon.online/heartbeat/abc123).

Create a probe script that checks the queue state and pings Vigilmon only when the queue is healthy (either empty or actively processing):

#!/bin/bash
QUEUE=$(curl -s http://localhost:8188/queue)

# Extract pending and running counts
PENDING=$(echo "$QUEUE" | python3 -c "import sys,json; q=json.load(sys.stdin); print(len(q.get('queue_pending', [])))" 2>/dev/null)
RUNNING=$(echo "$QUEUE" | python3 -c "import sys,json; q=json.load(sys.stdin); print(len(q.get('queue_running', [])))" 2>/dev/null)

# Check if the state file exists (tracks how long we've had pending jobs)
STATE_FILE="/tmp/comfyui_queue_state"
NOW=$(date +%s)

if [ "$PENDING" -gt 0 ] && [ "$RUNNING" -eq 0 ]; then
  # Jobs pending but nothing running — potentially stuck
  if [ -f "$STATE_FILE" ]; then
    SINCE=$(cat "$STATE_FILE")
    ELAPSED=$((NOW - SINCE))
    if [ "$ELAPSED" -gt 300 ]; then
      echo "Queue stuck: $PENDING pending, 0 running for ${ELAPSED}s" >&2
      exit 1
    fi
  else
    echo "$NOW" > "$STATE_FILE"
  fi
else
  # Queue is healthy (empty or actively running)
  rm -f "$STATE_FILE"
  curl -s https://vigilmon.online/heartbeat/abc123
fi

Schedule it:

# /etc/cron.d/comfyui-heartbeat
*/10 * * * * root /usr/local/bin/comfyui-queue-probe.sh >> /var/log/comfyui-hb.log 2>&1

When the queue is stuck for more than 5 minutes, the script stops pinging Vigilmon and the heartbeat alert fires after your configured interval.

Step 6: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
  2. Set Consecutive failures before alert to 2 on the web interface monitor — ComfyUI can take up to 60 seconds to restart when loading large SDXL or Flux models.
  3. Set the /system_stats and /history monitors to alert after 1 failure — those API endpoints should always respond instantly.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Web interface | http://your-server:8188 | Python server crash, server down | | System stats API | /system_stats | Backend unhealthy, VRAM API failure | | History API | /history | Database layer failure | | SSL certificate | HTTPS reverse proxy domain | Let's Encrypt / Caddy renewal failure | | Queue heartbeat | /queue probe | Frozen queue, stuck generation, model crash |

ComfyUI puts sophisticated AI image generation pipelines within reach of anyone with capable hardware — but those pipelines run on your own infrastructure. With Vigilmon watching the web interface, API endpoints, SSL certificate, and a live queue health probe, you have complete visibility into your ComfyUI deployment and can catch frozen generation jobs before they block your entire workflow.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles