Monitoring Velero Kubernetes Backups with Vigilmon

Velero is the de-facto tool for Kubernetes backup and disaster recovery. It snapshots cluster resources and persistent volume data to object storage (S3, GCS, Azure Blob) on a schedule you define. The problem: Velero failures are silent. A misconfigured backup schedule, a failed object storage credential, or a crashing Velero server all stop backups from running — and you won't discover the gap until you attempt a restore after an incident. Vigilmon gives you proactive monitoring so backup failures surface immediately rather than at the worst possible moment.

What You'll Set Up

• Metrics endpoint monitor for the Velero server
• Heartbeat monitor to confirm backups run on schedule
• Object storage connectivity check
• SSL certificate alerts for custom certificates
• Backup failure alerts via Velero's metrics

Prerequisites

• Velero installed in your Kubernetes cluster
• Object storage backend configured (S3, GCS, or Azure)
• A free Vigilmon account

Step 1: Monitor the Velero Metrics Endpoint

Velero exposes a Prometheus metrics endpoint at port :8085/metrics that includes backup counts, last backup timestamps, and failure counts. Use Vigilmon to monitor its availability:

1. First, expose the Velero metrics endpoint via kubectl port-forward or a Kubernetes Service:

# Port-forward for quick testing
kubectl port-forward deployment/velero 8085:8085 -n velero

# Or create a ClusterIP service for persistent access
kubectl expose deployment velero --port=8085 --target-port=8085 \
  --name=velero-metrics -n velero

2. Log in to vigilmon.online and click Add Monitor.
3. Set Type to HTTP / HTTPS.
4. Enter the metrics URL: http://velero-metrics.velero.svc.cluster.local:8085/metrics (internal) or the exposed external URL.
5. Set Expected HTTP status to 200.
6. Set interval to 2 minutes.
7. Click Save.

The metrics endpoint being reachable confirms the Velero server pod is running and healthy.

Step 2: Heartbeat Monitoring for Backup Schedule Compliance

The most critical thing to monitor isn't just whether Velero is running — it's whether backups are actually completing on schedule. Use Vigilmon's cron heartbeat for this.

Create a Kubernetes CronJob that runs after each expected backup window, checks the last backup status, and pings Vigilmon:

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set the interval to match your backup schedule (e.g., 1440 minutes for daily backups).
3. Copy the heartbeat URL.
4. Create a Kubernetes CronJob that validates backup completion:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: velero-backup-heartbeat
  namespace: velero
spec:
  schedule: "0 3 * * *"  # Run 1 hour after daily backup at 2am
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: velero
          containers:
          - name: check
            image: bitnami/kubectl:latest
            command:
            - /bin/sh
            - -c
            - |
              LAST=$(kubectl get backup -n velero \
                --sort-by=.status.completionTimestamp \
                -o jsonpath='{.items[-1].status.phase}')
              if [ "$LAST" = "Completed" ]; then
                wget -q https://vigilmon.online/heartbeat/YOUR_ID -O /dev/null
              fi
          restartPolicy: OnFailure

If a backup fails or the schedule slips, the heartbeat stops and Vigilmon alerts you within one interval.

Step 3: Verify Object Storage Connectivity

Velero writes backups to object storage. If S3 credentials rotate, bucket permissions change, or the storage endpoint becomes unreachable, backups fail silently at upload time.

Check storage connectivity directly:

# Verify the backup storage location is available
velero backup-location get

# Expected output shows "Available" in the Phase column
# NAME      PROVIDER   BUCKET/PREFIX   PHASE       LAST VALIDATED
# default   aws        my-bucket       Available   2026-07-01 ...

For continuous monitoring, add the storage location check to the heartbeat CronJob:

STORAGE=$(kubectl get backupstoragelocation default -n velero \
  -o jsonpath='{.status.phase}')
if [ "$STORAGE" = "Available" ]; then
  # proceed with backup check
fi

You can also add a Vigilmon TCP monitor for the object storage endpoint if you're using self-hosted MinIO:

1. Click Add Monitor → TCP Port.
2. Enter your MinIO host and port: minio.yourdomain.com:9000.
3. Set interval to 5 minutes.
4. Click Save.

Step 4: Monitor Backup Failure Metrics

Velero's Prometheus metrics include a velero_backup_failure_total counter. If this counter increments, a backup has failed. Create an alert by scraping this metric:

# Check failure count from metrics endpoint
curl -s http://localhost:8085/metrics | grep velero_backup_failure_total
# velero_backup_failure_total{schedule="daily-backup"} 0

If you have Grafana, create an alert rule on this counter. If you don't, add a metric-scraping check to the heartbeat CronJob:

FAILURES=$(curl -s http://velero:8085/metrics | \
  grep 'velero_backup_failure_total' | \
  awk '{print $2}')
if [ "$FAILURES" = "0" ]; then
  curl -s https://vigilmon.online/heartbeat/YOUR_ID
fi

Step 5: SSL Certificate Alerts

If Velero communicates with object storage over HTTPS with custom certificates (common with self-hosted MinIO or private S3-compatible endpoints), certificate expiry breaks backup uploads.

Add an SSL monitor for your object storage endpoint:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: https://minio.yourdomain.com or your S3-compatible endpoint.
3. Enable Monitor SSL certificate.
4. Set Alert when certificate expires in less than 21 days.
5. Click Save.

For the Velero API server itself, if exposed externally via an Ingress with TLS:

kubectl get ingress -n velero

Add an SSL monitor for any externally-exposed Velero endpoints found.

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP / Metrics | :8085/metrics | Velero server pod down | | Cron heartbeat | Heartbeat URL | Backup missed schedule, backup failed | | TCP / MinIO | :9000 | Object storage unreachable | | SSL certificate | Storage endpoint | Credential/cert expiry breaks uploads |

Kubernetes gives you the tools to recover from disasters — but only if your backups actually ran. Velero's silent failure modes (missed schedules, storage auth failures, pod crashes) are exactly what Vigilmon's heartbeat monitoring catches. With continuous monitoring across the Velero metrics endpoint, backup schedule compliance, and object storage connectivity, you'll know about backup problems in minutes rather than discovering them mid-incident.