Monitoring Label Studio with Vigilmon

Label Studio is a self-hosted multi-type data annotation platform that supports text, images, audio, video, time-series, HTML, and PDF annotation tasks — all in one interface. ML teams use it to build training datasets for computer vision, NLP, audio processing, and time-series models, with active learning workflows powered by ML backend integrations. When Celery workers stall, an ML backend becomes unreachable, or the task import endpoint fails, annotation pipelines block silently: annotators see loading spinners, ML predictions stop appearing, and downstream training runs wait indefinitely for exported datasets. Vigilmon gives you continuous coverage across the Django app server, PostgreSQL, Celery, Redis, the projects and task APIs, ML backend connectivity, file storage, webhook delivery, and TLS certificate expiry — so a broken active learning loop pages you the moment it stalls.

What You'll Set Up

• HTTP uptime monitor for the Label Studio web UI (port 8080)
• PostgreSQL database connectivity heartbeat (production mode)
• Celery worker availability heartbeat
• Redis connectivity monitor (Celery broker and cache)
• Projects API health check
• Task import API monitor
• Annotation export API health check
• ML backend integration health check
• File storage health check
• Webhook delivery health monitor
• TLS certificate expiry alert

Prerequisites

• Label Studio running in production mode (PostgreSQL, not SQLite), accessible on port 8080
• Redis configured as the Celery broker and cache
• PostgreSQL configured as the Label Studio database
• SSH access to the Label Studio host for scripted health checks
• A free Vigilmon account

Step 1: Monitor the Label Studio Web UI

Label Studio's Django application serves the annotation interface on port 8080. A crash here blocks all annotators, dataset managers, and ML engineers from accessing their projects.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your Label Studio URL: http://your-server-ip:8080.
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

If Label Studio is behind a reverse proxy:

https://labelstudio.yourdomain.com

For a richer health signal, use Label Studio's built-in health endpoint:

1. In Vigilmon, click Add Monitor → HTTP / HTTPS.
2. Enter: http://your-server-ip:8080/health.
3. Set Expected HTTP status to 200.
4. Enable Response body must contain and enter "status": "UP".
5. Set Check interval to 1 minute.
6. Click Save.

The /health endpoint checks Django, database connectivity, and Redis simultaneously — a single monitor that covers multiple failure modes.

Step 2: Monitor PostgreSQL Database Connectivity

In production mode, Label Studio stores projects, tasks, annotations, predictions, and user accounts in PostgreSQL. SQLite is development-only. A database outage in production stops all annotation operations cold.

#!/bin/bash
# /usr/local/bin/labelstudio-db-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_DB_HEARTBEAT_ID"
DB_HOST="${DJANGO_DB_HOST:-localhost}"
DB_PORT="${DJANGO_DB_PORT:-5432}"
DB_NAME="${DJANGO_DB_NAME:-labelstudio}"
DB_USER="${DJANGO_DB_USER:-labelstudio}"

if pg_isready -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -U "$DB_USER" -q; then
    curl -s "$HEARTBEAT_URL"
else
    echo "PostgreSQL not ready for Label Studio"
fi

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set expected interval to 2 minutes.

Schedule:

crontab -e
*/2 * * * * /usr/local/bin/labelstudio-db-check.sh

Step 3: Monitor Celery Worker Availability

Label Studio uses Celery workers for ML backend prediction requests, import/export jobs, and webhook deliveries. If workers go offline, annotation import from S3 or local storage stalls, ML predictions stop updating, and webhooks to downstream ML pipelines queue indefinitely.

#!/bin/bash
# /usr/local/bin/labelstudio-worker-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_WORKER_HEARTBEAT_ID"

# Ping Celery workers
WORKER_STATUS=$(celery -A label_studio.core inspect ping --timeout 10 2>/dev/null)

if echo "$WORKER_STATUS" | grep -q "pong"; then
    curl -s "$HEARTBEAT_URL"
else
    echo "No Label Studio Celery workers responding"
fi

If running with Docker:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_WORKER_HEARTBEAT_ID"

if docker exec labelstudio-worker celery -A label_studio.core inspect ping --timeout 10 2>/dev/null | grep -q "pong"; then
    curl -s "$HEARTBEAT_URL"
fi

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set expected interval to 5 minutes.

Schedule:

*/5 * * * * /usr/local/bin/labelstudio-worker-check.sh

Step 4: Monitor Redis Connectivity

Redis serves as both Celery broker and Django cache for Label Studio. Redis going offline stops all background task processing and can degrade UI performance as cache misses hit the database directly.

Add a direct TCP monitor:

1. In Vigilmon, click Add Monitor → TCP Port.
2. Enter your server hostname and port 6379.
3. Set Check interval to 1 minute.
4. Click Save.

Add a heartbeat script for a deeper check:

#!/bin/bash
# /usr/local/bin/labelstudio-redis-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_REDIS_HEARTBEAT_ID"
REDIS_HOST="${REDIS_HOST:-localhost}"
REDIS_PORT="${REDIS_PORT:-6379}"

RESPONSE=$(redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" PING 2>/dev/null)

if [ "$RESPONSE" = "PONG" ]; then
    curl -s "$HEARTBEAT_URL"
else
    echo "Redis PING failed: $RESPONSE"
fi

Schedule every 2 minutes:

*/2 * * * * /usr/local/bin/labelstudio-redis-check.sh

Step 5: Monitor the Projects API

The projects listing API at GET /api/projects is the starting point for all Label Studio automation — scripts that create annotation tasks, CI/CD pipelines that trigger exports, and ML backend orchestrators that manage prediction requests all begin by listing projects.

1. In Vigilmon, click Add Monitor → HTTP / HTTPS.
2. Enter: http://your-server-ip:8080/api/projects.
3. Set Expected HTTP status to 401 (unauthenticated probe — confirms the API is responding without exposing credentials in the monitor).
4. Set Check interval to 2 minutes.
5. Click Save.

For an authenticated check:

#!/bin/bash
# /usr/local/bin/labelstudio-projects-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_PROJECTS_HEARTBEAT_ID"
LS_URL="http://localhost:8080"
LS_TOKEN="${LABEL_STUDIO_API_TOKEN}"

HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -m 10 \
  -H "Authorization: Token $LS_TOKEN" \
  "$LS_URL/api/projects")

if [ "$HTTP_CODE" = "200" ]; then
    curl -s "$HEARTBEAT_URL"
else
    echo "Projects API returned HTTP $HTTP_CODE"
fi

Get your API token from Label Studio under Account & Settings → Access Token.

Schedule every 5 minutes:

*/5 * * * * /usr/local/bin/labelstudio-projects-check.sh

Step 6: Monitor the Task Import API

The task import endpoint (POST /api/projects/:id/import) is the pipeline entry point for bringing annotation tasks into Label Studio from data sources — S3 buckets, databases, CI/CD pipelines, and data preparation scripts all write here. Import endpoint failures block all dataset growth.

#!/bin/bash
# /usr/local/bin/labelstudio-import-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_IMPORT_HEARTBEAT_ID"
LS_URL="http://localhost:8080"
LS_TOKEN="${LABEL_STUDIO_API_TOKEN}"
PROJECT_ID="${LS_TEST_PROJECT_ID:-1}"

# Check import endpoint availability (GET on tasks list — avoids creating test data)
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -m 10 \
  -H "Authorization: Token $LS_TOKEN" \
  "$LS_URL/api/projects/$PROJECT_ID/tasks?page_size=1")

if [ "$HTTP_CODE" = "200" ]; then
    curl -s "$HEARTBEAT_URL"
else
    echo "Task import endpoint returned HTTP $HTTP_CODE"
fi

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set expected interval to 10 minutes.

Schedule:

*/10 * * * * /usr/local/bin/labelstudio-import-check.sh

Step 7: Monitor the Annotation Export API

The export endpoint (GET /api/projects/:id/export) produces annotation datasets in JSON, CSV, or COCO format for ML training pipeline consumption. A stalled export blocks all downstream training runs waiting for labeled data.

#!/bin/bash
# /usr/local/bin/labelstudio-export-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_EXPORT_HEARTBEAT_ID"
LS_URL="http://localhost:8080"
LS_TOKEN="${LABEL_STUDIO_API_TOKEN}"
PROJECT_ID="${LS_TEST_PROJECT_ID:-1}"

# Trigger a minimal export to verify the endpoint and Celery worker can handle it
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -m 60 \
  -H "Authorization: Token $LS_TOKEN" \
  "$LS_URL/api/projects/$PROJECT_ID/export?exportType=JSON")

if [ "$HTTP_CODE" = "200" ]; then
    curl -s "$HEARTBEAT_URL"
else
    echo "Export API returned HTTP $HTTP_CODE"
fi

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set expected interval to 30 minutes.

Schedule:

*/30 * * * * /usr/local/bin/labelstudio-export-check.sh

Step 8: Monitor ML Backend Integration Health

Label Studio can connect to a custom ML backend for active learning — sending prediction requests as annotators label tasks. If the ML backend becomes unreachable, predictions stop appearing in the annotation interface and the active learning loop breaks silently.

Add an HTTP monitor for your ML backend's prediction endpoint:

1. In Vigilmon, click Add Monitor → HTTP / HTTPS.
2. Enter your ML backend health URL: http://your-ml-backend:9090/health.
3. Set Expected HTTP status to 200.
4. Set Check interval to 2 minutes.
5. Click Save.

If your ML backend exposes a custom health endpoint:

#!/bin/bash
# /usr/local/bin/labelstudio-mlbackend-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_MLBACKEND_HEARTBEAT_ID"
ML_BACKEND_URL="${ML_BACKEND_URL:-http://localhost:9090}"

# Check the health endpoint that Label Studio uses
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -m 10 "$ML_BACKEND_URL/health")

if [ "$HTTP_CODE" = "200" ]; then
    curl -s "$HEARTBEAT_URL"
else
    echo "ML backend health check returned HTTP $HTTP_CODE"
fi

To verify Label Studio can actually reach the ML backend (tests the configured connection, not just the raw URL):

HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -m 10 \
  -H "Authorization: Token $LS_TOKEN" \
  "http://localhost:8080/api/ml/$ML_BACKEND_ID")

if [ "$HTTP_CODE" = "200" ]; then
    curl -s "$HEARTBEAT_URL"
fi

Schedule every 5 minutes:

*/5 * * * * /usr/local/bin/labelstudio-mlbackend-check.sh

Step 9: Monitor File Storage Health

Label Studio annotation tasks reference images, audio, video, and other assets — stored either locally on disk or in cloud object storage (S3, GCS). If the storage volume becomes inaccessible or permissions change, annotation tasks load with broken asset URLs and annotators cannot complete their work.

Local file storage

#!/bin/bash
# /usr/local/bin/labelstudio-storage-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_STORAGE_HEARTBEAT_ID"
MEDIA_ROOT="${LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT:-/home/user/labelstudio/media}"
THRESHOLD_PCT=90

if [ ! -d "$MEDIA_ROOT" ] || [ ! -r "$MEDIA_ROOT" ]; then
    echo "Media directory inaccessible: $MEDIA_ROOT"
    exit 1
fi

USAGE=$(df "$MEDIA_ROOT" | awk 'NR==2 {gsub("%",""); print $5}')
if [ "$USAGE" -ge "$THRESHOLD_PCT" ]; then
    echo "Storage at ${USAGE}% — above ${THRESHOLD_PCT}% threshold"
    exit 1
fi

curl -s "$HEARTBEAT_URL"

S3 or GCS object storage

Verify presigned URL generation is working — if IAM permissions change, Label Studio cannot generate URLs for assets even though the objects still exist:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_STORAGE_HEARTBEAT_ID"

# Verify S3 bucket is accessible
if aws s3 ls "s3://${LS_S3_BUCKET}" --max-items 1 &>/dev/null; then
    curl -s "$HEARTBEAT_URL"
else
    echo "S3 bucket access failed"
fi

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set expected interval to 5 minutes.

Schedule:

*/5 * * * * /usr/local/bin/labelstudio-storage-check.sh

Step 10: Monitor Webhook Delivery Health

Label Studio can trigger webhooks when annotations are submitted or updated — delivering labeled data directly to downstream ML pipelines, databases, or CI/CD triggers. Webhook delivery failures mean training pipelines never receive notification of new annotations.

Monitor your webhook receiver endpoint:

1. In Vigilmon, click Add Monitor → HTTP / HTTPS.
2. Enter your webhook receiver URL (the same endpoint configured in Label Studio's webhook settings).
3. Set Expected HTTP status to 200 or 405 (GET on a POST-only receiver — confirms the endpoint is reachable).
4. Set Check interval to 5 minutes.
5. Click Save.

To verify webhook delivery success rate from Label Studio's perspective:

#!/bin/bash
# /usr/local/bin/labelstudio-webhook-check.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_WEBHOOK_HEARTBEAT_ID"
LS_TOKEN="${LABEL_STUDIO_API_TOKEN}"
WEBHOOK_ID="${LS_WEBHOOK_ID:-1}"

# Check webhook delivery status (Label Studio 1.x API)
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -m 10 \
  -H "Authorization: Token $LS_TOKEN" \
  "http://localhost:8080/api/webhooks/$WEBHOOK_ID/tasks")

if [ "$HTTP_CODE" = "200" ]; then
    curl -s "$HEARTBEAT_URL"
else
    echo "Webhook API returned HTTP $HTTP_CODE"
fi

Schedule every 10 minutes:

*/10 * * * * /usr/local/bin/labelstudio-webhook-check.sh

Step 11: Monitor TLS Certificate Expiry

If Label Studio is exposed over HTTPS, an expired certificate immediately blocks annotators, ML backend connections, and API clients.

1. In Vigilmon, open your web UI monitor from Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

For a dedicated certificate monitor:

1. In Vigilmon, click Add Monitor → HTTP / HTTPS.
2. Enter: https://labelstudio.yourdomain.com.
3. Enable Monitor SSL certificate with a 21 day alert.
4. Set Check interval to 1 hour.
5. Click Save.

Step 12: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
2. Set Consecutive failures before alert to 2 on the web UI monitor — Django restarts take a few seconds.
3. Set Consecutive failures before alert to 1 on Redis, Celery, and ML backend monitors — a broken active learning loop stalls the entire annotation workflow immediately.

Route failures by urgency:

• Redis down, Celery workers silent, ML backend unreachable → Slack #labelstudio-critical (immediate)
• Web UI down, database unreachable, export stalled → Slack #labelstudio-alerts (urgent)
• Storage approaching capacity, TLS expiry, webhook failures → email (investigate within hours)

Summary

| Monitor | Target | What It Catches | |---|---|---| | Web UI / health | GET /health | Django crash, database or Redis failure | | PostgreSQL | pg_isready heartbeat | Production database unavailable | | Redis TCP | Port 6379 | Celery broker and cache offline | | Celery workers | celery inspect ping | Import/export/webhook pipeline dead | | Projects API | GET /api/projects | REST layer failure | | Task import | GET /api/projects/:id/tasks | Import endpoint failure | | Annotation export | GET /api/projects/:id/export | Export pipeline stall | | ML backend | GET /health on backend | Active learning predictions stopped | | File storage | Disk/S3 heartbeat | Asset URLs broken for annotators | | Webhook receiver | Receiver endpoint HTTP check | Downstream pipeline not notified | | TLS certificate | HTTPS endpoint | Certificate expiry |

Label Studio's value is its active learning loop — annotations flow in, ML predictions flow out, and model quality improves continuously. That loop depends on Celery workers, Redis, PostgreSQL, ML backend connectivity, and storage all running together. With Vigilmon watching every link, a stalled Celery worker or unreachable ML backend pages you immediately instead of silently degrading annotation quality while annotators wonder why predictions disappeared.