tutorial

Monitoring Vertical Pod Autoscaler (VPA): Keep K8s Cost Optimization Healthy

How to monitor Vertical Pod Autoscaler recommendations, admission webhook health, and OOMKill rates so your Kubernetes cost optimization never silently breaks production.

Vertical Pod Autoscaler automatically right-sizes container CPU and memory requests based on actual usage. When it works, your cluster runs leaner and cheaper. When it breaks — admission webhook down, recommender stuck, OOMKills spiking — workloads crash or waste money and nobody notices until the next billing cycle or a 3 AM page.

This tutorial shows you how to monitor VPA itself so your cost optimization stays reliable.

What Is Vertical Pod Autoscaler?

VPA has three components that each need to stay healthy:

  • Recommender — watches historical resource usage and computes optimal request and limit values
  • Updater — evicts pods whose current requests deviate too far from recommendations so they restart with better values
  • Admission Controller — mutating webhook that patches resource requests on new pods at creation time

A failure in any component silently degrades the others. The Recommender going down means stale recommendations. The Admission Controller going down means pods launch with the old (wasteful or too-small) requests defined in the Deployment spec.

Why VPA Monitoring Matters

VPA failures are silent by default:

Admission webhook downtime — if the admission controller pod crashes, Kubernetes falls back to the resource requests in your Deployment manifests. Pods may OOMKill (requests too low) or waste memory for weeks (requests too high) while Kubernetes reports everything healthy.

Recommender data staleness — the Recommender needs a continuous stream of metrics from the Metrics Server or Prometheus. If that pipeline breaks, recommendations freeze. VPA keeps applying the last known values, which may no longer reflect current load patterns.

Updater over-eviction — under high churn, the Updater can evict healthy pods too aggressively, creating rolling restarts that look like a deployment rollout but are actually VPA adjusting to a load spike.

Mode misconfiguration — VPA updateMode: Auto and Off behave very differently. A misconfigured VPA object silently stops adjusting resources without any error visible to engineers.

What You'll Set Up

  • External health monitoring for the VPA admission webhook endpoint
  • Heartbeat monitoring to verify the Recommender is updating recommendations
  • Alert routing for OOMKill spikes and webhook latency

You'll need a free Vigilmon account — no credit card required.

Step 1: Expose a Health Endpoint for the VPA Admission Controller

The VPA Admission Controller runs as a webhook server. Expose its /healthz endpoint through an Ingress or NodePort so Vigilmon can probe it externally:

# vpa-webhook-monitor-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: vpa-webhook-monitor
  namespace: kube-system
spec:
  selector:
    app: vpa-admission-controller
  ports:
    - name: https
      port: 443
      targetPort: 8000
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: vpa-webhook-health
  namespace: kube-system
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  rules:
    - host: vpa-health.internal.yourdomain.com
      http:
        paths:
          - path: /healthz
            pathType: Prefix
            backend:
              service:
                name: vpa-webhook-monitor
                port:
                  number: 443

Verify the endpoint returns 200 before adding it to Vigilmon:

curl -sk https://vpa-health.internal.yourdomain.com/healthz
# Expected: ok

Step 2: Add an HTTP Monitor for the VPA Admission Webhook

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL: https://vpa-health.internal.yourdomain.com/healthz
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: ok
    • Response time threshold: 500ms
  6. Save the monitor

If the VPA admission controller crashes or becomes unreachable, you'll receive an alert within 2 minutes — long before misconfigured pod resources cause production impact.

Step 3: Heartbeat Monitoring for the VPA Recommender

The Recommender updates recommendations on a continuous loop. A CronJob that verifies recommendations are fresh and pings Vigilmon on success gives you a dead-man's switch:

# save as vpa-recommender-check.sh
#!/bin/bash
set -e

NAMESPACE="${VPA_NAMESPACE:-kube-system}"
MAX_AGE_SECONDS=600  # alert if no recommendation updated in 10 minutes
HEARTBEAT_URL="${VIGILMON_HEARTBEAT_URL}"

# Find the most recently updated VPA recommendation
LATEST=$(kubectl get vpa -A -o json | \
  jq -r '[.items[].status.recommendation.containerRecommendations[0].target] | 
         map(select(. != null)) | length')

if [ "$LATEST" -eq "0" ]; then
  echo "ERROR: No VPA recommendations found — Recommender may be stalled"
  exit 1
fi

echo "VPA Recommender active: $LATEST container recommendations present"
curl -fsS "$HEARTBEAT_URL" > /dev/null
echo "Heartbeat sent."

Deploy as a Kubernetes CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: vpa-recommender-heartbeat
  namespace: kube-system
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          serviceAccountName: vpa-monitor-sa
          containers:
            - name: check
              image: bitnami/kubectl:latest
              env:
                - name: VIGILMON_HEARTBEAT_URL
                  valueFrom:
                    secretKeyRef:
                      name: vigilmon-secrets
                      key: vpa-recommender-heartbeat-url
              command: ["/bin/sh", "-c"]
              args: ["/scripts/vpa-recommender-check.sh"]
              volumeMounts:
                - name: scripts
                  mountPath: /scripts
          volumes:
            - name: scripts
              configMap:
                name: vpa-check-scripts
                defaultMode: 0755

Set up the heartbeat in Vigilmon:

  1. Go to Monitors → New Monitor → Heartbeat
  2. Name it VPA Recommender Active
  3. Set the expected interval: 5 minutes
  4. Set grace period: 3 minutes
  5. Copy the ping URL into your Kubernetes Secret

Step 4: OOMKill Rate Monitoring

OOMKill spikes indicate VPA is under-provisioning memory — either because the Recommender has stale data or because the updateMode is Off. Add a script that counts OOMKill events and reports them:

#!/bin/bash
# Check for recent OOMKills — run as a CronJob every 5 minutes
THRESHOLD=3
WINDOW_MINUTES=5

OOMKILLS=$(kubectl get events -A --field-selector reason=OOMKilling \
  --sort-by='.lastTimestamp' -o json | \
  jq --arg cutoff "$(date -u -d "$WINDOW_MINUTES minutes ago" +%Y-%m-%dT%H:%M:%SZ)" \
  '[.items[] | select(.lastTimestamp > $cutoff)] | length')

if [ "$OOMKILLS" -ge "$THRESHOLD" ]; then
  echo "WARNING: $OOMKILLS OOMKills in last ${WINDOW_MINUTES}m — VPA may be under-provisioning"
  # Do NOT ping heartbeat — allow the monitor to fire
  exit 1
fi

echo "OOMKill rate normal: $OOMKILLS events in last ${WINDOW_MINUTES}m"
curl -fsS "$VIGILMON_OOMKILL_HEARTBEAT_URL" > /dev/null

Set the heartbeat interval to 5 minutes with a 6-minute grace period. If OOMKills exceed the threshold, the heartbeat goes silent and Vigilmon fires an alert.

Step 5: Alert Routing

Slack Alerts for VPA Issues

  1. In Vigilmon go to Alert Channels → New Channel → Webhook
  2. Add your Slack Incoming Webhook URL
  3. Assign the channel to all three VPA monitors

A typical alert payload when the admission controller goes down:

{
  "monitor_name": "VPA Admission Webhook /healthz",
  "status": "down",
  "url": "https://vpa-health.internal.yourdomain.com/healthz",
  "started_at": "2026-07-12T08:15:00Z",
  "duration_seconds": 0,
  "regions_failing": ["eu-west", "us-east"]
}

Recommended Alert Thresholds

| Monitor | Interval | Grace Period | Action | |---|---|---|---| | VPA Admission Webhook | 1 min | — | Page on-call immediately | | VPA Recommender heartbeat | 5 min | 3 min | Page on-call | | OOMKill rate | 5 min | 6 min | Alert SRE channel |

Key Metrics to Watch

| Metric | What it signals | Threshold to alert | |---|---|---| | Admission webhook response time | Controller load / degradation | > 500ms | | Recommender heartbeat age | Recommender stall | > 10 minutes | | OOMKill event count | Under-provisioning | > 3 in 5 minutes | | Pods evicted by Updater | Over-aggressive tuning | > 10% of pod count per hour | | VPA objects with Off mode | Cost opt disabled | > 0 unexpected |

Summary

VPA silently breaks in ways that cost you money or cause pod crashes. External monitoring closes the gap:

| Component | What breaks silently | Vigilmon monitor type | |---|---|---| | Admission Controller | Pods launch with wrong resources | HTTP /healthz monitor | | Recommender | Stale recommendations | Heartbeat monitor | | OOMKill rate | Under-provisioning | Heartbeat with threshold script |

Start free at vigilmon.online — your first VPA monitor is running in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles