Celery powers the asynchronous backbone of Python applications — email delivery, image processing, data pipeline runs, third-party API calls, and scheduled jobs all route through it. When a Celery worker pool goes down or a broker connection drops, tasks pile up silently in the queue. No errors appear in your web application, but background work stops: password reset emails are never sent, invoices are never generated, and data imports hang indefinitely. By the time a user reports the issue, the queue can be hours deep.
This tutorial covers production-grade uptime monitoring for Celery using Vigilmon. We will walk through:
- Exposing a health check endpoint from your Django or Flask application
- Monitoring Celery worker availability and broker connectivity
- SSL certificate monitoring for task broker endpoints
- Webhook alerts for DOWN/UP events
Prerequisites
- Celery running with Redis or RabbitMQ as broker
- Django, Flask, or FastAPI application using Celery
- A free account at vigilmon.online
Part 1: Add a Celery health check endpoint
Celery does not expose an HTTP health endpoint by default. Add one to your web application that actively pings the broker and checks worker availability using Celery's inspect API.
Django implementation
# myapp/views.py
import json
from celery import current_app
from django.http import JsonResponse
from django.views import View
class CeleryHealthView(View):
def get(self, request):
try:
inspector = current_app.control.inspect(timeout=3.0)
ping_result = inspector.ping()
if not ping_result:
return JsonResponse(
{"status": "error", "detail": "No workers responded to ping"},
status=503,
)
worker_count = len(ping_result)
return JsonResponse(
{"status": "ok", "workers": worker_count},
status=200,
)
except Exception as exc:
return JsonResponse(
{"status": "error", "detail": str(exc)},
status=503,
)
# myapp/urls.py
from django.urls import path
from .views import CeleryHealthView
urlpatterns = [
path("health/celery/", CeleryHealthView.as_view(), name="celery-health"),
]
Flask implementation
# app/health.py
from celery import current_app as celery_app
from flask import Blueprint, jsonify
health_bp = Blueprint("health", __name__)
@health_bp.get("/health/celery")
def celery_health():
try:
inspector = celery_app.control.inspect(timeout=3.0)
ping_result = inspector.ping()
if not ping_result:
return jsonify({"status": "error", "detail": "No workers responded"}), 503
return jsonify({"status": "ok", "workers": len(ping_result)}), 200
except Exception as exc:
return jsonify({"status": "error", "detail": str(exc)}), 503
Verify the health endpoint
# Start your Celery worker first
celery -A myproject worker --loglevel=info &
# Then test the endpoint
curl -s http://localhost:8000/health/celery | python3 -m json.tool
Expected response with workers running:
{
"status": "ok",
"workers": 2
}
Response with no workers available:
{
"status": "error",
"detail": "No workers responded to ping"
}
Part 2: Expose the health endpoint for external monitoring
Run your Django or Flask app behind a reverse proxy with TLS. Expose /health/celery publicly so Vigilmon can poll it.
nginx configuration
# /etc/nginx/sites-available/myapp
server {
listen 443 ssl;
server_name api.example.com;
ssl_certificate /etc/letsencrypt/live/api.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;
# Celery health check — no authentication required
location = /health/celery {
proxy_pass http://127.0.0.1:8000/health/celery;
proxy_set_header Host $host;
proxy_read_timeout 10s;
access_log off;
}
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Verify external accessibility:
curl https://api.example.com/health/celery
# {"status": "ok", "workers": 2}
Part 3: Set up HTTP monitoring in Vigilmon
Monitor Celery worker availability
- Log in to vigilmon.online and click Add Monitor.
- Choose HTTP(S) monitor.
- Enter:
https://api.example.com/health/celery - Set interval to 1 minute.
- Add a keyword check: must contain
"status":"ok". - Add your alert channel.
- Click Save.
Monitor additional Celery layers
| Component | Monitor URL | Keyword |
|-----------|-------------|---------|
| Worker ping | https://api.example.com/health/celery | "status":"ok" |
| Broker (Redis) | https://api.example.com/health/redis | ok |
| Beat scheduler | https://api.example.com/health/beat | ok |
The keyword check on "status":"ok" ensures Vigilmon marks the monitor DOWN when the endpoint returns HTTP 200 but with an error JSON body — a case that happens when the health view itself is reachable but all workers are gone.
Part 4: Monitor the Redis or RabbitMQ broker
The broker is the critical dependency. If it goes down, workers stop consuming tasks even if the worker processes are healthy.
Redis broker health endpoint
# app/health.py
import redis
from django.conf import settings
from django.http import JsonResponse
from django.views import View
class RedisHealthView(View):
def get(self, request):
try:
client = redis.from_url(settings.CELERY_BROKER_URL)
client.ping()
return JsonResponse({"status": "ok"}, status=200)
except Exception as exc:
return JsonResponse({"status": "error", "detail": str(exc)}, status=503)
RabbitMQ management API
RabbitMQ exposes a management HTTP API on port 15672. Add a Vigilmon monitor for it:
curl -s -u guest:guest http://rabbitmq.example.com:15672/api/healthchecks/node
# {"status": "ok"}
Add a Vigilmon monitor:
- Add Monitor → HTTP(S)
- URL:
https://rabbitmq.example.com/api/healthchecks/node - Basic auth: add your RabbitMQ credentials
- Keyword:
"status":"ok" - Interval: 1 minute
Part 5: SSL certificate monitoring
If your application API or broker management endpoint uses TLS, an expired certificate causes Celery workers to fail broker reconnection silently — tasks stop being processed with no obvious error in the application layer.
- In Vigilmon, click Add Monitor.
- Choose SSL monitor.
- Enter:
api.example.com - Set alert threshold to 14 days before expiry.
- Add your alert channel.
- Click Save.
Add a separate SSL monitor for rabbitmq.example.com if your broker management UI uses TLS.
Part 6: Webhook alerts for DOWN events
Route Vigilmon alerts into your incident management system so worker outages trigger immediate escalation:
// webhook-receiver.ts
import express from 'express';
const app = express();
app.use(express.json());
app.post('/webhook/vigilmon', (req, res) => {
const { monitor_name, status, url, response_code, checked_at } = req.body;
if (status === 'down') {
console.error('[VIGILMON] Celery monitor DOWN', {
monitor: monitor_name,
url,
code: response_code,
at: checked_at,
});
postSlackAlert({
text: `⚠️ Celery workers DOWN\n${url}\nHTTP ${response_code}\nBackground jobs are not processing.`,
channel: '#infrastructure-alerts',
});
createIncidentTicket({
title: `Celery workers unavailable — ${monitor_name}`,
severity: 'high',
runbook: 'https://wiki.example.com/runbooks/celery-down',
});
} else if (status === 'up') {
console.info('[VIGILMON] Celery monitor recovered', { monitor: monitor_name });
postSlackAlert({
text: `✅ Celery workers recovered\n${url}`,
channel: '#infrastructure-alerts',
});
}
res.sendStatus(204);
});
app.listen(3000);
Vigilmon sends this payload on DOWN and UP transitions:
{
"monitor_id": "mon_abc123",
"monitor_name": "Celery Worker Health",
"status": "down",
"url": "https://api.example.com/health/celery",
"checked_at": "2026-06-30T09:00:00Z",
"response_code": 503,
"response_time_ms": 3050
}
Part 7: Kubernetes deployments
For Celery workers deployed as Kubernetes Deployments, add liveness probes that verify the worker is consuming from the broker:
# celery-worker-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: celery-worker
namespace: myapp
spec:
replicas: 3
selector:
matchLabels:
app: celery-worker
template:
metadata:
labels:
app: celery-worker
spec:
containers:
- name: celery-worker
image: myapp:latest
command:
- celery
- -A
- myproject
- worker
- --loglevel=info
- --concurrency=4
livenessProbe:
exec:
command:
- celery
- -A
- myproject
- inspect
- ping
- --timeout=5
- --destination=celery@$HOSTNAME
initialDelaySeconds: 30
periodSeconds: 60
timeoutSeconds: 10
failureThreshold: 3
env:
- name: CELERY_BROKER_URL
valueFrom:
secretKeyRef:
name: celery-secrets
key: broker-url
- name: CELERY_RESULT_BACKEND
valueFrom:
secretKeyRef:
name: celery-secrets
key: result-backend
Expose the application health endpoint via Ingress for Vigilmon:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp
namespace: myapp
spec:
tls:
- hosts:
- api.example.com
secretName: myapp-tls
rules:
- host: api.example.com
http:
paths:
- path: /health
pathType: Prefix
backend:
service:
name: myapp-web
port:
number: 8000
Summary
Your Celery deployment now has four layers of monitoring:
- Worker ping endpoint — actively checks that at least one worker responds to a broker ping, polled every 60 seconds by Vigilmon.
- Broker monitor — a separate check on Redis or RabbitMQ so you can distinguish between "workers are dead" and "broker is unreachable."
- SSL monitor — alerts you 14 days before certificate expiry, before worker-to-broker TLS connections start failing.
- Webhook/Slack alerts — DOWN events with a runbook link reach your on-call channel within one check interval.
Vigilmon handles check scheduling, multi-region polling, alert routing, and uptime history. When Celery workers go down, your team knows within 60 seconds — before users notice their password reset emails never arrived.
Monitor your Celery infrastructure free at vigilmon.online
#celery #python #asynctasks #backgroundjobs #monitoring #uptime