tutorial

How to Monitor Docker Distribution Registry with Vigilmon

Docker Distribution (formerly docker/distribution) is the open source OCI-compatible registry that powers Docker Hub and countless private registries. When y...

Docker Distribution (formerly docker/distribution) is the open source OCI-compatible registry that powers Docker Hub and countless private registries. When you self-host a distribution registry — whether as a pull-through cache, an air-gapped Kubernetes registry, or a private company registry — its availability directly controls whether your CI/CD pipelines can push images and whether your clusters can pull them.

In this tutorial you'll set up end-to-end monitoring for a self-hosted distribution registry using Vigilmon.


Why registry monitoring matters

A broken registry doesn't just slow down deployments — it can stop them entirely:

  • All docker pull operations fail — Kubernetes pod scheduling stalls because image pulls return 503
  • CI/CD pipelines break — every build that pushes an image fails at the registry push step
  • Storage backend degradation — the registry process is up but S3 or the local filesystem returns errors; pulls fail non-deterministically
  • GC job leaves orphaned blobs — storage grows unboundedly when garbage collection fails silently
  • TLS certificate expiry — the registry goes HTTPS-only overnight when the cert lapses

External monitoring catches all of these before your first failed deployment.


What you'll need

  • A running Docker Distribution registry (port 5000 by default)
  • A free Vigilmon account

Step 1: Verify the OCI Distribution API health endpoint

The OCI Distribution Specification defines GET /v2/ as the health probe endpoint. An anonymous request returns HTTP 200 if no auth is configured, or HTTP 401 with an auth challenge header if auth is enabled. Either response means the registry is alive.

Test it:

# No auth
curl -I http://your-registry:5000/v2/

# With basic auth
curl -I -u username:password https://your-registry:5000/v2/

# With a self-signed cert
curl -I -k https://your-registry:5000/v2/

A healthy response:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Docker-Distribution-Api-Version: registry/2.0

Or with auth enabled:

HTTP/1.1 401 Unauthorized
Www-Authenticate: Bearer realm="https://your-registry:5000/service/token"
Docker-Distribution-Api-Version: registry/2.0

Both are valid "registry is alive" responses. Configure your config.yml to expose the API:

version: 0.1
log:
  level: info
storage:
  filesystem:
    rootdirectory: /var/lib/registry
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

The health.storagedriver block makes the registry return 503 instead of 200 when the storage backend has consecutive failures — Vigilmon will catch this automatically.


Step 2: Set up HTTP monitoring for the registry API

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS as the type
  3. Set the URL to https://your-registry:5000/v2/
  4. If auth is required, set Basic Auth credentials
  5. Under Expected response, set status code to 200 (no auth) or 401 (with auth — 401 here means the registry is alive and challenging correctly)
    • To accept both 200 and 401, use the "status code in range" option or use a webhook validator
  6. Set the check interval to 1 minute
  7. Save the monitor

This monitor catches: registry process crash, port binding failure, storage backend outage (returns 503 via health driver), and network routing failures.


Step 3: Monitor the registry catalog endpoint

The catalog endpoint GET /v2/_catalog lists all repositories in the registry. Slow responses here indicate backend storage degradation — especially with large registries where the catalog scan becomes expensive.

curl -u username:password https://your-registry:5000/v2/_catalog

Add a second HTTP monitor:

  1. New Monitor → HTTP / HTTPS
  2. URL: https://your-registry:5000/v2/_catalog
  3. Authentication as needed
  4. Expected status: 200
  5. Add a response time alert — set a threshold of 5 seconds; a catalog scan taking longer than this indicates storage performance degradation

Step 4: Set up a TCP port check

TCP monitoring catches the case where the registry process is running but the port binding has failed or the load balancer in front of the registry is unreachable:

  1. New Monitor → TCP Port
  2. Hostname: your-registry
  3. Port: 5000 (or 443 if you're terminating TLS at nginx/Caddy in front of the registry)
  4. Save

Step 5: Monitor storage backend health

The distribution registry's health.storagedriver setting makes GET /v2/ return 503 after 3 consecutive storage failures. But you can also probe the storage backend directly.

For an S3/MinIO backend, add a separate HTTP monitor for the storage endpoint:

# MinIO health probe
curl http://your-minio:9000/minio/health/live

# AWS S3 — check bucket accessibility via the registry's IAM role
aws s3 ls s3://your-registry-bucket/ --region us-east-1

For a local filesystem backend, monitor disk utilization with a heartbeat:

#!/bin/bash
# /usr/local/bin/check-registry-disk.sh

REGISTRY_DIR="/var/lib/registry"
THRESHOLD=80
PING_URL="https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID"

usage=$(df -h "$REGISTRY_DIR" | awk 'NR==2 {gsub(/%/,""); print $5}')

if [ "$usage" -gt "$THRESHOLD" ]; then
  echo "ALERT: Registry disk at ${usage}% — above ${THRESHOLD}% threshold"
  exit 1
fi

# Report to Vigilmon heartbeat
curl -fsS "$PING_URL" > /dev/null 2>&1
echo "OK: Registry disk at ${usage}%"

Add this to cron:

*/5 * * * * /usr/local/bin/check-registry-disk.sh >> /var/log/registry-disk-check.log 2>&1

Step 6: Monitor TLS certificate expiry

If your registry serves HTTPS (required for most Docker clients), a lapsed TLS certificate will break all docker pull and docker push operations immediately.

Vigilmon's SSL monitoring checks your certificate automatically:

  1. Go to Monitors → New Monitor → SSL Certificate
  2. Enter your registry hostname
  3. Set an alert for 30 days before expiry (gives you time to renew before impact)
  4. Save

You'll receive an alert at 30 days, 14 days, and 7 days before the certificate expires — enough time to rotate it without any downtime.


Step 7: Monitor garbage collection job health

The distribution registry doesn't run GC automatically — you must run it manually or via a cron job. If GC isn't running, unreferenced blobs accumulate and storage grows without bound.

Set up a Vigilmon heartbeat monitor to track GC job health:

  1. Go to Monitors → New Monitor → Heartbeat
  2. Name it "Registry GC Job"
  3. Set the expected interval to 24 hours (or whatever your GC schedule is)
  4. Copy the heartbeat ping URL

Add the heartbeat ping to your GC script:

#!/bin/bash
# /usr/local/bin/registry-gc.sh

REGISTRY_CONTAINER="registry"
PING_URL="https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_ID"

# Stop the registry to run GC safely (or use online GC if your version supports it)
docker stop "$REGISTRY_CONTAINER"

# Run garbage collection
docker run --rm \
  -v /var/lib/registry:/var/lib/registry \
  registry:2 \
  garbage-collect /etc/docker/registry/config.yml

# Restart the registry
docker start "$REGISTRY_CONTAINER"

# Ping Vigilmon only on success
if [ $? -eq 0 ]; then
  curl -fsS "$PING_URL" > /dev/null 2>&1
  echo "GC completed successfully at $(date)"
else
  echo "GC FAILED at $(date)" >&2
fi

If the GC job fails or doesn't run on schedule, Vigilmon will alert you before storage exhaustion occurs.


Step 8: Configure alerting

  1. Go to Alert Channels → Add Channel → Webhook
  2. Set up Slack, PagerDuty, or any webhook receiver
  3. Assign the channel to all your registry monitors

Recommended alert policy:

  • Registry API down (HTTP 5xx or timeout) → page immediately (P1 — all pull/push operations broken)
  • TCP port unreachable → page immediately (P1)
  • TLS certificate <30 days → warn (P2 — renew before impact)
  • Disk usage >80% → warn (P2 — add storage or run GC)
  • GC heartbeat missed → warn (P2 — orphaned blobs accumulating)
  • Catalog response time >5s → warn (P2 — storage performance degradation)

Putting it all together

| Monitor | Type | Alert level | What it catches | |---------|------|-------------|-----------------| | https://registry:5000/v2/ | HTTP | P1 | Registry process / storage backend down | | https://registry:5000/v2/_catalog | HTTP | P2 | Catalog scan degradation | | registry:5000 | TCP | P1 | Port binding / LB failure | | registry:5000 TLS | SSL | P2 | Certificate expiry | | Registry GC cron | Heartbeat | P2 | GC job missed / failed | | Disk usage script | Heartbeat | P2 | Storage >80% full |


What's next

  • Pull-through cache monitoring — if your registry is configured as a Docker Hub mirror, add a heartbeat that periodically pulls a known public image through the cache and alerts if the pull fails or exceeds a latency threshold
  • Push/pull throughput metrics — enable registry access logging and ship to your log aggregator to track push/pull rates and identify slow builds
  • Auth service monitoring — if you're using a token auth service (e.g., Docker Registry Auth, Portus, or Harbor's auth service), add it as a separate HTTP monitor

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →