tutorial

Uptime monitoring for Celery distributed task queue infrastructure

Celery powers the asynchronous backbone of Python applications — email delivery, image processing, data pipeline runs, third-party API calls, and scheduled j...

Celery powers the asynchronous backbone of Python applications — email delivery, image processing, data pipeline runs, third-party API calls, and scheduled jobs all route through it. When a Celery worker pool goes down or a broker connection drops, tasks pile up silently in the queue. No errors appear in your web application, but background work stops: password reset emails are never sent, invoices are never generated, and data imports hang indefinitely. By the time a user reports the issue, the queue can be hours deep.

This tutorial covers production-grade uptime monitoring for Celery using Vigilmon. We will walk through:

  • Exposing a health check endpoint from your Django or Flask application
  • Monitoring Celery worker availability and broker connectivity
  • SSL certificate monitoring for task broker endpoints
  • Webhook alerts for DOWN/UP events

Prerequisites

  • Celery running with Redis or RabbitMQ as broker
  • Django, Flask, or FastAPI application using Celery
  • A free account at vigilmon.online

Part 1: Add a Celery health check endpoint

Celery does not expose an HTTP health endpoint by default. Add one to your web application that actively pings the broker and checks worker availability using Celery's inspect API.

Django implementation

# myapp/views.py
import json
from celery import current_app
from django.http import JsonResponse
from django.views import View


class CeleryHealthView(View):
    def get(self, request):
        try:
            inspector = current_app.control.inspect(timeout=3.0)
            ping_result = inspector.ping()

            if not ping_result:
                return JsonResponse(
                    {"status": "error", "detail": "No workers responded to ping"},
                    status=503,
                )

            worker_count = len(ping_result)
            return JsonResponse(
                {"status": "ok", "workers": worker_count},
                status=200,
            )

        except Exception as exc:
            return JsonResponse(
                {"status": "error", "detail": str(exc)},
                status=503,
            )
# myapp/urls.py
from django.urls import path
from .views import CeleryHealthView

urlpatterns = [
    path("health/celery/", CeleryHealthView.as_view(), name="celery-health"),
]

Flask implementation

# app/health.py
from celery import current_app as celery_app
from flask import Blueprint, jsonify

health_bp = Blueprint("health", __name__)


@health_bp.get("/health/celery")
def celery_health():
    try:
        inspector = celery_app.control.inspect(timeout=3.0)
        ping_result = inspector.ping()

        if not ping_result:
            return jsonify({"status": "error", "detail": "No workers responded"}), 503

        return jsonify({"status": "ok", "workers": len(ping_result)}), 200

    except Exception as exc:
        return jsonify({"status": "error", "detail": str(exc)}), 503

Verify the health endpoint

# Start your Celery worker first
celery -A myproject worker --loglevel=info &

# Then test the endpoint
curl -s http://localhost:8000/health/celery | python3 -m json.tool

Expected response with workers running:

{
  "status": "ok",
  "workers": 2
}

Response with no workers available:

{
  "status": "error",
  "detail": "No workers responded to ping"
}

Part 2: Expose the health endpoint for external monitoring

Run your Django or Flask app behind a reverse proxy with TLS. Expose /health/celery publicly so Vigilmon can poll it.

nginx configuration

# /etc/nginx/sites-available/myapp
server {
    listen 443 ssl;
    server_name api.example.com;

    ssl_certificate     /etc/letsencrypt/live/api.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;

    # Celery health check — no authentication required
    location = /health/celery {
        proxy_pass http://127.0.0.1:8000/health/celery;
        proxy_set_header Host $host;
        proxy_read_timeout 10s;
        access_log off;
    }

    location / {
        proxy_pass       http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Verify external accessibility:

curl https://api.example.com/health/celery
# {"status": "ok", "workers": 2}

Part 3: Set up HTTP monitoring in Vigilmon

Monitor Celery worker availability

  1. Log in to vigilmon.online and click Add Monitor.
  2. Choose HTTP(S) monitor.
  3. Enter: https://api.example.com/health/celery
  4. Set interval to 1 minute.
  5. Add a keyword check: must contain "status":"ok".
  6. Add your alert channel.
  7. Click Save.

Monitor additional Celery layers

| Component | Monitor URL | Keyword | |-----------|-------------|---------| | Worker ping | https://api.example.com/health/celery | "status":"ok" | | Broker (Redis) | https://api.example.com/health/redis | ok | | Beat scheduler | https://api.example.com/health/beat | ok |

The keyword check on "status":"ok" ensures Vigilmon marks the monitor DOWN when the endpoint returns HTTP 200 but with an error JSON body — a case that happens when the health view itself is reachable but all workers are gone.


Part 4: Monitor the Redis or RabbitMQ broker

The broker is the critical dependency. If it goes down, workers stop consuming tasks even if the worker processes are healthy.

Redis broker health endpoint

# app/health.py
import redis
from django.conf import settings
from django.http import JsonResponse
from django.views import View


class RedisHealthView(View):
    def get(self, request):
        try:
            client = redis.from_url(settings.CELERY_BROKER_URL)
            client.ping()
            return JsonResponse({"status": "ok"}, status=200)
        except Exception as exc:
            return JsonResponse({"status": "error", "detail": str(exc)}, status=503)

RabbitMQ management API

RabbitMQ exposes a management HTTP API on port 15672. Add a Vigilmon monitor for it:

curl -s -u guest:guest http://rabbitmq.example.com:15672/api/healthchecks/node
# {"status": "ok"}

Add a Vigilmon monitor:

  1. Add MonitorHTTP(S)
  2. URL: https://rabbitmq.example.com/api/healthchecks/node
  3. Basic auth: add your RabbitMQ credentials
  4. Keyword: "status":"ok"
  5. Interval: 1 minute

Part 5: SSL certificate monitoring

If your application API or broker management endpoint uses TLS, an expired certificate causes Celery workers to fail broker reconnection silently — tasks stop being processed with no obvious error in the application layer.

  1. In Vigilmon, click Add Monitor.
  2. Choose SSL monitor.
  3. Enter: api.example.com
  4. Set alert threshold to 14 days before expiry.
  5. Add your alert channel.
  6. Click Save.

Add a separate SSL monitor for rabbitmq.example.com if your broker management UI uses TLS.


Part 6: Webhook alerts for DOWN events

Route Vigilmon alerts into your incident management system so worker outages trigger immediate escalation:

// webhook-receiver.ts
import express from 'express';

const app = express();
app.use(express.json());

app.post('/webhook/vigilmon', (req, res) => {
  const { monitor_name, status, url, response_code, checked_at } = req.body;

  if (status === 'down') {
    console.error('[VIGILMON] Celery monitor DOWN', {
      monitor: monitor_name,
      url,
      code: response_code,
      at: checked_at,
    });

    postSlackAlert({
      text: `⚠️ Celery workers DOWN\n${url}\nHTTP ${response_code}\nBackground jobs are not processing.`,
      channel: '#infrastructure-alerts',
    });

    createIncidentTicket({
      title: `Celery workers unavailable — ${monitor_name}`,
      severity: 'high',
      runbook: 'https://wiki.example.com/runbooks/celery-down',
    });
  } else if (status === 'up') {
    console.info('[VIGILMON] Celery monitor recovered', { monitor: monitor_name });
    postSlackAlert({
      text: `✅ Celery workers recovered\n${url}`,
      channel: '#infrastructure-alerts',
    });
  }

  res.sendStatus(204);
});

app.listen(3000);

Vigilmon sends this payload on DOWN and UP transitions:

{
  "monitor_id": "mon_abc123",
  "monitor_name": "Celery Worker Health",
  "status": "down",
  "url": "https://api.example.com/health/celery",
  "checked_at": "2026-06-30T09:00:00Z",
  "response_code": 503,
  "response_time_ms": 3050
}

Part 7: Kubernetes deployments

For Celery workers deployed as Kubernetes Deployments, add liveness probes that verify the worker is consuming from the broker:

# celery-worker-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: celery-worker
  namespace: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: celery-worker
  template:
    metadata:
      labels:
        app: celery-worker
    spec:
      containers:
        - name: celery-worker
          image: myapp:latest
          command:
            - celery
            - -A
            - myproject
            - worker
            - --loglevel=info
            - --concurrency=4
          livenessProbe:
            exec:
              command:
                - celery
                - -A
                - myproject
                - inspect
                - ping
                - --timeout=5
                - --destination=celery@$HOSTNAME
            initialDelaySeconds: 30
            periodSeconds: 60
            timeoutSeconds: 10
            failureThreshold: 3
          env:
            - name: CELERY_BROKER_URL
              valueFrom:
                secretKeyRef:
                  name: celery-secrets
                  key: broker-url
            - name: CELERY_RESULT_BACKEND
              valueFrom:
                secretKeyRef:
                  name: celery-secrets
                  key: result-backend

Expose the application health endpoint via Ingress for Vigilmon:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp
  namespace: myapp
spec:
  tls:
    - hosts:
        - api.example.com
      secretName: myapp-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /health
            pathType: Prefix
            backend:
              service:
                name: myapp-web
                port:
                  number: 8000

Summary

Your Celery deployment now has four layers of monitoring:

  1. Worker ping endpoint — actively checks that at least one worker responds to a broker ping, polled every 60 seconds by Vigilmon.
  2. Broker monitor — a separate check on Redis or RabbitMQ so you can distinguish between "workers are dead" and "broker is unreachable."
  3. SSL monitor — alerts you 14 days before certificate expiry, before worker-to-broker TLS connections start failing.
  4. Webhook/Slack alerts — DOWN events with a runbook link reach your on-call channel within one check interval.

Vigilmon handles check scheduling, multi-region polling, alert routing, and uptime history. When Celery workers go down, your team knows within 60 seconds — before users notice their password reset emails never arrived.


Monitor your Celery infrastructure free at vigilmon.online

#celery #python #asynctasks #backgroundjobs #monitoring #uptime

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →