tutorial

Monitoring KubeVirt with Vigilmon: API Server Availability, virt-api Health, Pod Health & Prometheus Metrics

How to monitor KubeVirt with Vigilmon — API server availability, virt-api health endpoint, virt-controller and virt-handler pod health via Kubernetes API, Prometheus metrics, and SSL certificate alerts.

KubeVirt extends Kubernetes to manage virtual machines alongside containers — making it the foundation for platforms like OpenShift Virtualization and any organization running VMs on Kubernetes. When KubeVirt's control plane components go down, virtual machines can't be created, migrated, or managed, and running VMs may lose their lifecycle guarantees. Vigilmon gives you external visibility into KubeVirt's health so you know about control plane failures before they affect VM workloads: the KubeVirt API server, virt-api health endpoint, pod health for virt-controller and virt-handler, Prometheus metrics, and SSL certificate alerts for the webhook.

What You'll Build

  • A monitor on the KubeVirt API server via kubectl proxy
  • A check on the virt-api health endpoint
  • Pod health monitoring for virt-controller and virt-handler via the Kubernetes API
  • Prometheus metrics endpoint monitoring
  • SSL certificate alerts for the virt-api webhook

Prerequisites

  • A running Kubernetes cluster with KubeVirt installed
  • kubectl configured with access to the cluster
  • A free account at vigilmon.online

Step 1: Verify the KubeVirt API Server

KubeVirt registers its API under the Kubernetes API aggregation layer. You can reach it via kubectl proxy or directly through the Kubernetes API server:

# Start kubectl proxy in the background
kubectl proxy --port=8001 &

# Verify the KubeVirt API is registered
curl http://localhost:8001/apis/kubevirt.io/

A healthy KubeVirt installation returns a JSON API group response listing available API versions and resources:

{
  "kind": "APIGroup",
  "apiVersion": "v1",
  "name": "kubevirt.io",
  "versions": [{"groupVersion": "kubevirt.io/v1", "version": "v1"}],
  ...
}

Step 2: Create a Vigilmon Monitor for the KubeVirt API

Expose the Kubernetes API server externally (or use an internal monitoring endpoint) so Vigilmon can reach it. For external monitoring, configure your cluster's API server or use a dedicated monitoring ingress.

If your Kubernetes API is accessible at https://k8s-api.example.com:

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://k8s-api.example.com/apis/kubevirt.io/.
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: kubevirt.io (present in every valid KubeVirt API group response).
  7. Headers: Add Authorization: Bearer <your-monitoring-serviceaccount-token>.
  8. Click Save.

Create a dedicated read-only service account for monitoring:

kubectl create serviceaccount vigilmon-monitor -n kubevirt
kubectl create clusterrolebinding vigilmon-monitor \
  --clusterrole=view \
  --serviceaccount=kubevirt:vigilmon-monitor
kubectl create token vigilmon-monitor -n kubevirt --duration=8760h

Step 3: Monitor the virt-api Health Endpoint

virt-api is the KubeVirt API handler that processes VM creation, migration, and lifecycle requests. It exposes a health endpoint at port 8443:

curl -k https://<virt-api-pod-ip>:8443/healthz

A healthy virt-api returns ok. To monitor this externally, expose the health endpoint through a Kubernetes Service and Ingress:

apiVersion: v1
kind: Service
metadata:
  name: virt-api-monitor
  namespace: kubevirt
spec:
  selector:
    kubevirt.io: virt-api
  ports:
  - port: 8443
    targetPort: 8443

Then add a Vigilmon monitor:

  1. Add Monitor → HTTP.
  2. URL: https://virt-api.example.com/healthz (your exposed endpoint).
  3. Check interval: 60 seconds.
  4. Expected status: 200.
  5. Keyword: ok.
  6. Label: KubeVirt virt-api health.
  7. Click Save.

When virt-api is unhealthy, VM API calls fail immediately — new VMs cannot be created and existing VM management operations return errors.

Step 4: Monitor virt-controller and virt-handler Pod Health

virt-controller manages VM lifecycle state machines; virt-handler runs as a DaemonSet on every node and handles VM process management. You can check pod health via the Kubernetes API:

# Check virt-controller pods
curl -s https://k8s-api.example.com/api/v1/namespaces/kubevirt/pods?labelSelector=kubevirt.io=virt-controller \
  -H "Authorization: Bearer <token>" | jq '.items[].status.phase'

# Check virt-handler pods (DaemonSet)
curl -s https://k8s-api.example.com/api/v1/namespaces/kubevirt/pods?labelSelector=kubevirt.io=virt-handler \
  -H "Authorization: Bearer <token>" | jq '.items[].status.phase'

For Vigilmon monitoring, create an endpoint that summarizes pod health and expose it via a monitoring sidecar or script. Alternatively, monitor the virt-controller and virt-handler readiness through a dedicated endpoint:

  1. Add Monitor → HTTP.
  2. URL: https://k8s-api.example.com/apis/apps/v1/namespaces/kubevirt/deployments/virt-controller.
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: availableReplicas (present when at least one replica is available).
  6. Label: KubeVirt virt-controller.
  7. Click Save.

Repeat for the virt-handler DaemonSet by targeting its DaemonSet resource.

Step 5: Monitor the Prometheus Metrics Endpoint

KubeVirt exposes Prometheus metrics at :8443/metrics on virt-api and other components. Monitoring the metrics endpoint confirms that observability is functioning and that the components are running:

curl -sk https://<virt-api-ip>:8443/metrics | head -20

A healthy metrics endpoint returns Prometheus-format text with kubevirt_* metrics. Add a Vigilmon monitor on your exposed metrics endpoint:

  1. Add Monitor → HTTP.
  2. URL: https://virt-api.example.com/metrics (your exposed endpoint).
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: kubevirt_ (present in every valid KubeVirt metrics response).
  6. Label: KubeVirt Prometheus metrics.
  7. Click Save.

Step 6: Monitor SSL Certificates for the virt-api Webhook

KubeVirt's virt-api registers a mutating and validating webhook with Kubernetes. This webhook uses TLS, and if the certificate expires, Kubernetes API calls for VM resources will fail with webhook errors:

kubectl get secret -n kubevirt kubevirt-virt-api-certs -o jsonpath='{.data.tls\.crt}' | \
  base64 -d | openssl x509 -noout -enddate

For external SSL monitoring via Vigilmon, monitor the exposed virt-api endpoint certificate:

  1. Add Monitor → SSL Certificate.
  2. Domain: virt-api.example.com (your exposed virt-api endpoint).
  3. Alert when expiry is within: 30 days.
  4. Alert again: 14 days, 7 days, 3 days, 1 day.
  5. Click Save.

KubeVirt rotates its internal webhook certificates automatically in most configurations. However, if you expose virt-api externally with a custom certificate, monitor that certificate separately.

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | KubeVirt API (/apis/kubevirt.io/) | Non-200 or keyword missing | KubeVirt CRDs may be unregistered; check kubectl get kubevirt -n kubevirt | | virt-api health | Non-200 or ok missing | virt-api pod down; check kubectl get pods -n kubevirt | | virt-controller deployment | Non-200 or availableReplicas missing | Controller unavailable; VM state machines not processing | | Prometheus metrics | Non-200 or keyword missing | Metrics exporter down; check component health | | SSL certificate | < 30 days to expiry | Renew external certificate; webhook TLS may need rotation |

Alert after: 1 consecutive failure for virt-api health — a virt-api failure means all VM API operations fail immediately.

Common KubeVirt Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | virt-api pod crash or OOMKilled | virt-api health monitor fires within 60 s | | virt-controller unavailable | Deployment monitor shows 0 availableReplicas | | KubeVirt CRDs unregistered (bad upgrade) | KubeVirt API group endpoint returns 404 | | Webhook certificate expired | SSL monitor alerts at 30-day threshold; API calls fail | | Kubernetes API server unreachable | All monitors fire simultaneously | | Node running virt-handler drained | DaemonSet monitor shows reduced available pods | | Prometheus metrics scrape broken | Metrics endpoint monitor fires |

KubeVirt is the control plane for virtual machines in Kubernetes environments — when its components degrade, VM operations fail silently or with cryptic Kubernetes API errors that are hard to trace without external monitoring. Vigilmon gives you external visibility across the KubeVirt API, component health endpoints, pod availability, and Prometheus metrics so you catch failures before they impact VM workloads.

Start monitoring KubeVirt in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles