How to Monitor Kubewarden Admission Controllers with Vigilmon

Kubewarden is a Kubernetes dynamic admission controller that enforces policies written in any language that compiles to WebAssembly. Policies run as WASM modules inside the cluster, validating and mutating resources at admission time. Because Kubewarden sits directly in the path of every kubectl apply and automated deployment, its availability is critical — if the admission webhook is down, resource creation can be blocked entirely depending on your failure mode configuration.

In this tutorial you'll set up comprehensive monitoring for Kubewarden using Vigilmon — free tier, no credit card required.

Why Kubewarden needs external monitoring

Kubewarden consists of several components, each a potential failure point:

• PolicyServer pod down — the WebAssembly policy evaluation server is unavailable; depending on your failurePolicy, all admission requests either fail closed (blocking all deployments) or fail open (skipping all policy checks)
• ValidatingWebhookConfiguration becomes unreachable — the Kubernetes webhook points to a service that isn't responding; API server times out on admission calls
• Policy download fails — Kubewarden pulls WASM policies from an OCI registry at startup; if the registry is unreachable during a pod restart, the PolicyServer may refuse to start
• Controller manager down — the Kubewarden controller that reconciles PolicyServer and ClusterAdmissionPolicy CRDs stops working; policy changes don't take effect
• Certificate rotation breaks — Kubewarden manages TLS certs for webhook communication; if cert rotation fails, the API server stops trusting the webhook

External monitoring from Vigilmon catches availability problems before they manifest as mysterious deployment failures.

What you'll need

• A Kubernetes cluster with Kubewarden installed (kubewarden-controller and at least one PolicyServer)
• A free Vigilmon account

Step 1: Expose Kubewarden health endpoints

Kubewarden's PolicyServer exposes a metrics and health port. By default it runs on port 8080 for metrics and has health endpoints. Expose them externally for monitoring:

# Check what's running in the kubewarden-system namespace
kubectl get pods -n kubewarden-system
# NAME                                        READY   STATUS    RESTARTS   AGE
# kubewarden-controller-manager-xxx-yyy       2/2     Running   0          5d
# policy-server-default-xxx-yyy              1/1     Running   0          5d

# Check existing services
kubectl get svc -n kubewarden-system

Expose the PolicyServer's metrics port as a LoadBalancer service so Vigilmon can probe it:

# kubewarden-monitor-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: kubewarden-health-monitor
  namespace: kubewarden-system
  labels:
    app: kubewarden-monitoring
spec:
  type: LoadBalancer
  selector:
    # This selector targets the default PolicyServer pods
    app: kubewarden-policy-server-default
  ports:
    - name: metrics
      port: 8080
      targetPort: 8080
      protocol: TCP

kubectl apply -f kubewarden-monitor-svc.yaml
kubectl get svc kubewarden-health-monitor -n kubewarden-system
# NAME                        TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)          AGE
# kubewarden-health-monitor   LoadBalancer   10.96.20.50    203.0.113.60    8080:31600/TCP   2m

Verify the metrics/health endpoint responds:

curl http://203.0.113.60:8080/metrics
# # HELP kubewarden_policy_evaluations_total ...
# # TYPE kubewarden_policy_evaluations_total counter

curl http://203.0.113.60:8080/readyz
# OK

Step 2: Add a custom health proxy if needed

If the PolicyServer metrics endpoint isn't accessible directly, deploy a thin health proxy in the same namespace:

# kubewarden-health-proxy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kubewarden-health-proxy
  namespace: kubewarden-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kubewarden-health-proxy
  template:
    metadata:
      labels:
        app: kubewarden-health-proxy
    spec:
      serviceAccountName: kubewarden-health-reader
      containers:
        - name: proxy
          image: curlimages/curl:latest
          command: ["/bin/sh", "-c"]
          args:
            - |
              while true; do
                STATUS=$(curl -s -o /dev/null -w '%{http_code}' \
                  http://kubewarden-controller-manager-metrics-service.kubewarden-system.svc:8443/healthz \
                  --cacert /tmp/ca.crt 2>/dev/null)
                echo "Controller health: $STATUS"
                sleep 30
              done
          resources:
            requests:
              cpu: 10m
              memory: 16Mi
---
apiVersion: v1
kind: Service
metadata:
  name: kubewarden-health-proxy
  namespace: kubewarden-system
spec:
  type: LoadBalancer
  selector:
    app: kubewarden-health-proxy
  ports:
    - port: 8090
      targetPort: 8090

Step 3: Set up HTTP monitoring in Vigilmon

Add your Kubewarden endpoints to Vigilmon:

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS
3. Add monitors:

| Monitor name | URL | Expected status | |---|---|---| | Kubewarden PolicyServer Metrics | http://203.0.113.60:8080/metrics | 200 | | Kubewarden PolicyServer Ready | http://203.0.113.60:8080/readyz | 200 | | Kubewarden Controller Health | https://controller.yourdomain.com/healthz | 200 |

4. Set check interval to 1 minute
5. Under Expected response, confirm status code 200; optionally check body for OK on the readyz endpoint
6. Save each monitor

Vigilmon's multi-region consensus means you won't get paged for a single transient probe failure. True downtime — the PolicyServer pod is crashed, the webhook service is unreachable — is confirmed quickly across regions.

Step 4: Monitor the webhook TCP port

The Kubernetes API server communicates with Kubewarden's admission webhook over HTTPS. The webhook service defaults to port 443. Monitor it at the TCP level:

1. Go to Monitors → New Monitor
2. Choose TCP Port
3. Enter the Kubewarden webhook service host and port:
   • Host: kubewarden-webhook.yourdomain.com
   • Port: 443
4. Save the monitor

If the TCP check fails while pods are running, you likely have a certificate or LoadBalancer issue preventing the API server from reaching the webhook.

Step 5: Configure alert channels

A Kubewarden outage can have two very different impacts depending on your failurePolicy:

• failurePolicy: Fail — every kubectl apply returns an error until the webhook recovers
• failurePolicy: Ignore — all policy checks are silently bypassed

Either outcome is critical. Configure aggressive alerting:

Email alerts

1. Go to Alert Channels → Add Channel → Email
2. Add your platform team's on-call address
3. Assign to all Kubewarden monitors

Webhook alerts

1. Go to Alert Channels → Add Channel → Webhook
2. Route to PagerDuty or Slack
3. Vigilmon payload:

{
  "monitor_name": "Kubewarden PolicyServer Ready",
  "status": "down",
  "url": "http://203.0.113.60:8080/readyz",
  "started_at": "2024-01-15T14:00:00Z",
  "duration_seconds": 60
}

For failurePolicy: Ignore deployments, include a note in your runbook that alerts mean policy enforcement has gone dark — deployments are succeeding but nothing is being validated.

Step 6: Diagnose Kubewarden failures

When Vigilmon fires an alert, use this sequence to identify the failure:

# 1. Check PolicyServer pod status
kubectl get pods -n kubewarden-system -l app=kubewarden-policy-server-default

# 2. Check controller status
kubectl get pods -n kubewarden-system -l control-plane=controller-manager

# 3. Check policy evaluation metrics — look for sudden drop to zero
kubectl port-forward svc/kubewarden-health-monitor 8080:8080 -n kubewarden-system
curl http://localhost:8080/metrics | grep kubewarden_policy_evaluations_total

# 4. Check ClusterAdmissionPolicy status
kubectl get clusteradmissionpolicies
kubectl get admissionpolicies --all-namespaces

# 5. Check PolicyServer CRD status
kubectl get policyservers
kubectl describe policyserver default

# 6. Check webhook configuration is healthy
kubectl get validatingwebhookconfigurations | grep kubewarden
kubectl describe validatingwebhookconfiguration kubewarden-validating-webhook-configuration

# 7. Look for OCI registry pull failures (common after pod restarts)
kubectl logs -n kubewarden-system -l app=kubewarden-policy-server-default --tail=50 | \
  grep -E 'error|failed|pull|wasm'

The most common failure pattern after a pod restart is an OCI registry pull failure — the WASM policy module can't be fetched. If this is the cause, the PolicyServer will enter a CrashLoopBackOff while the registry is temporarily unreachable, and Vigilmon will show the health endpoint as down.

Step 7: Monitor policy evaluation rates

A healthy Kubewarden installation evaluates admission requests continuously. A drop in evaluation rate — even if the health endpoint returns 200 — can indicate a misconfigured webhook that's no longer receiving traffic.

Add a response-body check to your metrics monitor:

1. Edit the Kubewarden PolicyServer Metrics monitor in Vigilmon
2. Under Expected response, add a body contains check for kubewarden_policy_evaluations_total

This confirms the metrics endpoint is returning real Kubewarden data, not just an empty 200 from a reverse proxy.

Step 8: Create a status page for admission control

1. Go to Status Pages → New Status Page
2. Name it: "Kubewarden Admission Control"
3. Add monitors: PolicyServer Metrics, PolicyServer Ready, Controller Health, Webhook TCP
4. Share the URL with your development teams

When a developer can't apply a resource and gets a webhook timeout error, the first thing they should check is this status page — not file a ticket or ping the platform team.

Summary

| What you set up | What it catches | |---|---| | HTTP monitor on /readyz | PolicyServer unavailability (blocks or bypasses admission) | | HTTP monitor on /metrics | PolicyServer running but not serving real data | | HTTP monitor on controller /healthz | Controller manager failures (policy changes stall) | | TCP monitor on webhook port 443 | Certificate failures, LoadBalancer routing issues | | Multi-region consensus | Fewer false positives from transient network blips | | Alert channels | Immediate notification before deployment pipelines break |

Kubewarden makes your cluster safer by enforcing policy at admission time. Vigilmon makes your cluster more observable by ensuring Kubewarden itself is always running — because a silent admission controller failure is one of the hardest problems to diagnose under pressure.

Get started at vigilmon.online — free tier, no credit card required.