Uptime monitoring for Faktory background job server infrastructure

Faktory is a language-agnostic background job server that decouples job producers from job consumers across any stack. Go services enqueue jobs that Ruby workers process, Python scripts schedule tasks that Node.js handlers execute — Faktory handles the coordination. When the Faktory server goes down, every producer immediately starts failing to enqueue jobs and every worker stops picking up new work, regardless of language or framework. Because Faktory is infrastructure rather than application code, its failure surface is often undermonitored until a queue-backed feature visibly stops working.

This tutorial covers production-grade uptime monitoring for Faktory using Vigilmon. We will walk through:

• Monitoring the Faktory server health endpoint
• Monitoring the Faktory Web UI
• SSL certificate monitoring for your job server endpoints
• Webhook alerts for DOWN/UP events

Prerequisites

• Faktory server running (Docker or binary install)
• A free account at vigilmon.online

Part 1: Enable and verify the Faktory health endpoint

Faktory exposes a /healthz HTTP endpoint on its Web UI port (7420 by default). This endpoint returns HTTP 200 when the server is accepting connections and its embedded database is healthy.

Docker deployment

docker run -d \
  --name faktory \
  -p 7419:7419 \
  -p 7420:7420 \
  -e FAKTORY_PASSWORD=your-secret-password \
  -v faktory-data:/var/lib/faktory \
  contribsys/faktory:latest

Docker Compose

# docker-compose.yml
version: "3.8"

services:
  faktory:
    image: contribsys/faktory:latest
    ports:
      - "7419:7419"   # Worker protocol port
      - "7420:7420"   # Web UI / HTTP API port
    environment:
      FAKTORY_PASSWORD: ${FAKTORY_PASSWORD}
    volumes:
      - faktory-data:/var/lib/faktory
    restart: unless-stopped
    command: /faktory -b :7419 -w :7420 -e production

volumes:
  faktory-data:

Verify the health endpoint

curl -s http://localhost:7420/healthz

Expected response when healthy:

OK

Or verify with status code:

curl -o /dev/null -w "%{http_code}" http://localhost:7420/healthz
# 200

The /healthz endpoint returns HTTP 200 with body OK when the server is ready. It returns a non-200 status when the embedded RocksDB database cannot be accessed or the server is still initializing.

Part 2: Expose the health endpoint for external monitoring

Faktory's Web UI runs on port 7420. Place it behind a reverse proxy with TLS to give Vigilmon a public HTTPS endpoint to check.

nginx configuration

# /etc/nginx/sites-available/faktory
server {
    listen 443 ssl;
    server_name faktory.example.com;

    ssl_certificate     /etc/letsencrypt/live/faktory.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/faktory.example.com/privkey.pem;

    # Health endpoint — no authentication required
    location = /healthz {
        proxy_pass http://127.0.0.1:7420/healthz;
        proxy_set_header Host $host;
        access_log off;
    }

    # Web UI — require basic auth
    location / {
        auth_basic           "Faktory";
        auth_basic_user_file /etc/nginx/.htpasswd;
        proxy_pass           http://127.0.0.1:7420;
        proxy_set_header     Host $host;
        proxy_set_header     X-Real-IP $remote_addr;
    }
}

Caddy configuration

faktory.example.com {
    handle /healthz {
        reverse_proxy localhost:7420
    }

    handle {
        basicauth {
            admin $2a$14$...
        }
        reverse_proxy localhost:7420
    }
}

Verify from outside:

curl https://faktory.example.com/healthz
# OK

Part 3: Set up HTTP monitoring in Vigilmon

Monitor the Faktory server

1. Log in to vigilmon.online and click Add Monitor.
2. Choose HTTP(S) monitor.
3. Enter: https://faktory.example.com/healthz
4. Set interval to 1 minute.
5. Add a keyword check: must contain OK.
6. Add your alert channel.
7. Click Save.

Monitor additional Faktory endpoints

| Component | Monitor URL | Keyword | |-----------|-------------|---------| | Health check | https://faktory.example.com/healthz | OK | | Web UI | https://faktory.example.com/ | — (status 200/302) | | Stats API | https://faktory.example.com/stats | faktory |

The keyword check on OK is important — a misconfigured reverse proxy can return HTTP 200 with a custom error page while Faktory itself is down. Checking the body confirms the Faktory process responded.

Part 4: Monitor the Faktory Web UI

The Web UI provides the primary operational view into queue depth, job retries, and dead jobs. Add a separate monitor for it so you know whether a 503 represents a UI failure or a full server failure:

1. In Vigilmon, click Add Monitor.
2. Choose HTTP(S) monitor.
3. Enter: https://faktory.example.com/
4. Set interval to 1 minute.
5. If the Web UI requires auth, Vigilmon will see a 302 redirect to login — set expected status to 302 or add a keyword present on the login page.
6. Click Save.

Use the Faktory stats API for a richer health signal:

curl -s https://faktory.example.com/stats | python3 -m json.tool

Sample output:

{
  "server": {
    "description": "Faktory",
    "faktory_version": "1.9.0",
    "uptime": 3600,
    "connections": 4,
    "used_memory_mb": 128
  },
  "queues": {
    "default": 0,
    "critical": 0,
    "bulk": 12
  }
}

Add a monitor against /stats with a keyword check for faktory_version to confirm the API endpoint is responding correctly.

Part 5: SSL certificate monitoring

The worker protocol port (7419) uses a separate TLS certificate from the Web UI port (7420) in some deployments. Workers that cannot verify the Faktory TLS certificate immediately stop processing — they fail to authenticate and disconnect.

1. In Vigilmon, click Add Monitor.
2. Choose SSL monitor.
3. Enter: faktory.example.com
4. Set alert threshold to 14 days before expiry.
5. Add your alert channel.
6. Click Save.

If you expose Faktory on a custom subdomain for workers (e.g., faktory-workers.example.com), add a separate SSL monitor for it.

Part 6: Webhook alerts for DOWN events

Wire Vigilmon's DOWN/UP notifications into your incident workflow:

// webhook-receiver.ts
import express from 'express';

const app = express();
app.use(express.json());

app.post('/webhook/vigilmon', (req, res) => {
  const { monitor_name, status, url, response_code, checked_at } = req.body;

  if (status === 'down') {
    console.error('[VIGILMON] Faktory monitor DOWN', {
      monitor: monitor_name,
      url,
      code: response_code,
      at: checked_at,
    });

    postSlackAlert({
      text: `⚠️ Faktory server DOWN\n${url}\nHTTP ${response_code}\nAll background jobs have stopped processing across all services.`,
      channel: '#infrastructure-alerts',
    });

    createIncidentTicket({
      title: `Faktory server unavailable — ${monitor_name}`,
      severity: 'critical',
      runbook: 'https://wiki.example.com/runbooks/faktory-down',
    });
  } else if (status === 'up') {
    console.info('[VIGILMON] Faktory recovered', { monitor: monitor_name });
    postSlackAlert({
      text: `✅ Faktory server recovered\n${url}`,
      channel: '#infrastructure-alerts',
    });
  }

  res.sendStatus(204);
});

app.listen(3000);

Vigilmon sends this payload on DOWN and UP transitions:

{
  "monitor_id": "mon_abc123",
  "monitor_name": "Faktory /healthz",
  "status": "down",
  "url": "https://faktory.example.com/healthz",
  "checked_at": "2026-06-30T09:00:00Z",
  "response_code": 503,
  "response_time_ms": 4100
}

Part 7: Kubernetes deployments

For Faktory deployed as a StatefulSet in Kubernetes (required because Faktory is stateful — it embeds RocksDB):

# faktory-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: faktory
  namespace: jobs
spec:
  serviceName: faktory
  replicas: 1
  selector:
    matchLabels:
      app: faktory
  template:
    metadata:
      labels:
        app: faktory
    spec:
      containers:
        - name: faktory
          image: contribsys/faktory:latest
          args: ["/faktory", "-b", ":7419", "-w", ":7420", "-e", "production"]
          ports:
            - containerPort: 7419
              name: worker
            - containerPort: 7420
              name: web
          livenessProbe:
            httpGet:
              path: /healthz
              port: 7420
            initialDelaySeconds: 15
            periodSeconds: 20
            timeoutSeconds: 5
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /healthz
              port: 7420
            initialDelaySeconds: 5
            periodSeconds: 10
            timeoutSeconds: 3
          env:
            - name: FAKTORY_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: faktory-secrets
                  key: password
          volumeMounts:
            - name: faktory-data
              mountPath: /var/lib/faktory
  volumeClaimTemplates:
    - metadata:
        name: faktory-data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 20Gi
---
apiVersion: v1
kind: Service
metadata:
  name: faktory
  namespace: jobs
spec:
  selector:
    app: faktory
  ports:
    - name: worker
      port: 7419
      targetPort: 7419
    - name: web
      port: 7420
      targetPort: 7420
  type: ClusterIP

Expose the health endpoint and Web UI via Ingress for Vigilmon:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: faktory
  namespace: jobs
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: faktory-basic-auth
spec:
  tls:
    - hosts:
        - faktory.example.com
      secretName: faktory-tls
  rules:
    - host: faktory.example.com
      http:
        paths:
          - path: /healthz
            pathType: Exact
            backend:
              service:
                name: faktory
                port:
                  number: 7420
          - path: /
            pathType: Prefix
            backend:
              service:
                name: faktory
                port:
                  number: 7420

Note: override the basic auth annotation for the /healthz path so Vigilmon can reach it without credentials:

# Add a separate Ingress rule for /healthz without auth
metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      location = /healthz {
        auth_basic off;
        proxy_pass http://faktory.jobs.svc.cluster.local:7420/healthz;
      }

Summary

Your Faktory deployment now has four layers of monitoring:

1. /healthz endpoint — confirms the Faktory process and embedded RocksDB are healthy, polled every 60 seconds by Vigilmon.
2. Web UI monitor — a separate HTTP check on the management interface so you can distinguish between a UI failure and a full server failure.
3. SSL monitor — alerts you 14 days before certificate expiry, before workers fail TLS handshakes on the job protocol port.
4. Webhook/incident alerts — DOWN events tagged as critical reach your on-call rotation within one check interval.

Vigilmon handles check scheduling, multi-region polling, alert routing, and uptime history. When Faktory goes down, your team knows within 60 seconds — before background job failures cascade across every service in your stack.

Monitor your Faktory infrastructure free at vigilmon.online

#faktory #backgroundjobs #jobqueue #devops #monitoring #uptime