tutorial

Monitoring Obico with Vigilmon

Obico uses AI to detect 3D print failures — but if its ML inference worker or printer agent connection goes down, spaghetti failures go undetected and prints are ruined. Here's how to monitor every Obico component with Vigilmon.

Obico (formerly The Spaghetti Detective) is an AI-powered 3D print monitoring platform that uses machine learning to detect print failures — the dreaded "spaghetti" of extruded filament going nowhere — and automatically pauses the print before an entire spool is wasted. Its stack is substantial: a Python/Django web server, Celery async workers running AI inference, Redis for queuing, PostgreSQL for storage, and OctoPrint/Moonraker plugin agents running on your printers. When any component fails, your printer keeps printing into the void while Obico silently watches nothing. Vigilmon monitors every layer of the Obico stack so failure detection stays online.

What You'll Set Up

  • Web server and dashboard uptime monitor (port 3334)
  • Celery worker health via admin API heartbeat
  • Redis TCP connectivity check
  • ML inference endpoint health probe
  • Printer agent connection status monitor
  • Webcam stream ingestion check
  • Time-lapse job and notification delivery heartbeats

Prerequisites

  • Obico installed via Docker Compose (standard self-hosted deployment)
  • Web UI accessible at http://yourhost:3334 or https://obico.yourdomain.com
  • Django admin accessible at /admin/
  • A free Vigilmon account

Step 1: Monitor the Obico Web Server

The Django web application serves the dashboard, API, and WebSocket connections from printer agents. If it goes down, you lose visibility into every connected printer.

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter your Obico URL: https://obico.yourdomain.com (or http://yourhost:3334).
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Enable Monitor SSL certificate if using HTTPS, with expiry alert at 21 days.
  7. Click Save.

Step 2: Monitor PostgreSQL Database Connectivity

Obico stores print history, user accounts, printer configurations, and AI detection events in PostgreSQL.

  1. Click Add Monitor → TCP Port.
  2. Host and port: yourhost:5432 (or the internal port if using Docker networking).
  3. Check interval: 1 minute.
  4. Click Save.

For a richer check, monitor the Obico API endpoint that reads from the database:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: https://obico.yourdomain.com/api/v1/printers/.
  3. Add your Obico API token: Authorization: Token YOUR_OBICO_API_TOKEN.
  4. Expected status: 200.
  5. Check interval: 2 minutes.
  6. Click Save.

An API 500 while the web UI serves its HTML shell points directly to a PostgreSQL connectivity or migration issue.

Step 3: Check Redis Connectivity

Redis serves as Obico's Celery task queue and caching layer. If Redis goes down, AI inference jobs queue up but never run — prints continue unmonitored.

  1. Click Add Monitor → TCP Port.
  2. Host and port: yourhost:6379.
  3. Check interval: 1 minute.
  4. Click Save.

For a functional check via Obico's health endpoint:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: https://obico.yourdomain.com/health_check/.
  3. Expected status: 200.
  4. Under Response validation, check that the body contains "redis" with a passing status.
  5. Check interval: 2 minutes.
  6. Click Save.

Step 4: Heartbeat for Celery AI Inference Workers

The Celery workers are the core of Obico — they receive webcam frames from printer agents, run them through the ML failure detection model, and trigger pause commands when spaghetti is detected. A dead Celery worker means zero AI monitoring regardless of whether the web server is up.

  1. Click Add Monitor → Cron Heartbeat.
  2. Set expected interval to 3 minutes.
  3. Copy the heartbeat URL.
  4. Add to a worker-probe script:
#!/bin/bash
# /opt/obico/celery-probe.sh
# Check if at least one Celery worker is responding

WORKERS=$(docker exec obico_ml_api_1 celery -A config inspect ping 2>/dev/null | grep -c "OK")

if [ "$WORKERS" -gt 0 ]; then
  curl -s https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID
fi

For Docker Compose deployments, adjust the container name to match your setup (docker compose ps to check names).

Schedule every 3 minutes:

*/3 * * * * /opt/obico/celery-probe.sh

Step 5: Monitor ML Model Inference Service Health

The AI failure detection model runs inside Obico's ml_api container. Monitor its health endpoint directly:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: http://yourhost:3333/ (Obico's internal ML API port — adjust if remapped).
  3. Expected status: 200.
  4. Check interval: 2 minutes.
  5. Click Save.

If the ML API container is not exposed externally, use a heartbeat via a wrapper script:

#!/bin/bash
# Probe the ML API from within Docker network
RESPONSE=$(docker exec obico_ml_api_1 curl -s -o /dev/null -w "%{http_code}" http://localhost:3333/)

if [ "$RESPONSE" = "200" ]; then
  curl -s https://vigilmon.online/heartbeat/YOUR_ML_HEARTBEAT_ID
fi

Step 6: Monitor Printer Agent Connectivity

Printer agents (OctoPrint plugin or Moonraker companion) maintain WebSocket connections back to Obico. If a printer loses its connection, Obico stops receiving webcam frames — failure detection silently stops.

Use the Obico API to probe printer tunnel status:

  1. Click Add Monitor → HTTP / HTTPS.
  2. URL: https://obico.yourdomain.com/api/v1/printers/?limit=1.
  3. Add the API token header.
  4. Under Response validation, check that the body contains "is_online": true (confirms at least one printer is connected).
  5. Check interval: 5 minutes.
  6. Click Save.

For more granular per-printer monitoring, create a heartbeat probe script:

#!/bin/bash
# /opt/obico/printer-status.sh
TOKEN="YOUR_OBICO_API_TOKEN"
HOST="https://obico.yourdomain.com"

ONLINE=$(curl -s -H "Authorization: Token $TOKEN" \
  "$HOST/api/v1/printers/" | python3 -c \
  "import sys,json; data=json.load(sys.stdin); print(sum(1 for p in data['results'] if p.get('is_online')))")

if [ "$ONLINE" -gt 0 ]; then
  curl -s https://vigilmon.online/heartbeat/YOUR_PRINTER_HEARTBEAT_ID
fi

Step 7: Monitor Webcam Stream Ingestion

Obico continuously ingests webcam frames for AI analysis. If frame ingestion stops (network issues, OctoPrint misconfiguration), the model has nothing to analyze.

Add a heartbeat that confirms recent frame ingestion via the Obico API:

#!/bin/bash
TOKEN="YOUR_OBICO_API_TOKEN"
HOST="https://obico.yourdomain.com"

# Get last seen time for first printer
LAST_SEEN=$(curl -s -H "Authorization: Token $TOKEN" \
  "$HOST/api/v1/printers/?limit=1" | python3 -c \
  "import sys,json; data=json.load(sys.stdin); print(data['results'][0].get('last_seen_at','') if data['results'] else '')")

# If last_seen_at was within the last 10 minutes, the stream is healthy
if [ -n "$LAST_SEEN" ]; then
  curl -s https://vigilmon.online/heartbeat/YOUR_STREAM_HEARTBEAT_ID
fi

Step 8: Heartbeat for Notification Delivery and Time-Lapse Jobs

Obico sends push notifications and emails when failures are detected, and generates time-lapse videos of completed prints. These run as Celery tasks.

Notification delivery heartbeat:

  1. Add a Cron Heartbeat with expected interval 30 minutes.
  2. After Obico sends a scheduled digest or test notification, confirm delivery and ping the heartbeat from a test script or monitoring harness.

Time-lapse generation heartbeat:

#!/bin/bash
# Check if time-lapse Celery queue is not stuck
QUEUE_DEPTH=$(docker exec obico_ml_api_1 celery -A config inspect active 2>/dev/null | grep -c "timelapse" || echo 0)

# A stuck queue (>50 pending time-lapses) means the worker is overloaded
if [ "$QUEUE_DEPTH" -lt 50 ]; then
  curl -s https://vigilmon.online/heartbeat/YOUR_TIMELAPSE_HEARTBEAT_ID
fi

Step 9: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
  2. For Redis and Celery worker monitors, set Consecutive failures before alert to 1 — AI monitoring stops the moment these go down.
  3. For the web server, set threshold to 2 to absorb brief Django restart cycles.
  4. For printer connectivity, consider alerting only during active print hours to avoid overnight false positives.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Web server (HTTP) | https://obico.yourdomain.com | Django crash, proxy error | | PostgreSQL (TCP) | yourhost:5432 | Database server down | | API probe | /api/v1/printers/ | DB connectivity, Django errors | | Redis (TCP) | yourhost:6379 | Cache and queue down | | Celery workers (heartbeat) | Heartbeat URL | AI inference not running | | ML API | http://yourhost:3333/ | Model service crash | | Printer agents (heartbeat) | Heartbeat URL | All printers disconnected | | Webcam ingestion (heartbeat) | Heartbeat URL | Frame stream stopped | | Time-lapse jobs (heartbeat) | Heartbeat URL | Post-processing queue stuck | | SSL certificate | Domain | TLS expiry |

Obico's value is its continuous AI watchfulness — the moment it stops watching, failed prints go undetected and filament is wasted. With Vigilmon monitoring the Celery workers, ML API, Redis, printer agent connections, and webcam ingestion independently, you know immediately when Obico's eyes go dark — and can restore monitoring before your next print starts.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles