tutorial

How to Monitor Kubeconform with Vigilmon

Kubeconform is a fast Kubernetes manifest schema validation tool that validates YAML manifests against the upstream Kubernetes OpenAPI schema, Custom Resourc...

Kubeconform is a fast Kubernetes manifest schema validation tool that validates YAML manifests against the upstream Kubernetes OpenAPI schema, Custom Resource Definition schemas, or any custom schema set. It replaces the older kubeval with significantly faster performance and support for CRD schemas. Engineering teams embed Kubeconform in CI pipelines, pre-commit hooks, and GitOps validation workflows to catch invalid manifests before they reach the cluster.

When Kubeconform runs as a validation service or API in a CI/CD pipeline, its availability directly determines whether deployments are validated before they ship. In this tutorial you'll set up uptime and response-time monitoring for a Kubeconform validation service using Vigilmon — free tier, no credit card required.

Why a Kubeconform validation service needs external monitoring

A Kubeconform validation service can fail in ways that silently break your deployment safety net:

  • Validation service crash — if the HTTP wrapper around Kubeconform crashes, CI pipelines that call it may either fail (blocking all deployments) or skip validation (letting invalid manifests through), depending on how the pipeline handles errors
  • Schema mirror becomes stale — Kubeconform fetches schemas from a configured source; if the schema mirror is unreachable or returns outdated schemas, validation passes invalid CRDs or misses breaking API version changes
  • CRD schema registry out of sync — custom schema registries for CRDs need to be updated when CRD versions change; a stale registry causes Kubeconform to silently skip CRD validation while appearing to succeed
  • High latency under CI load — Kubeconform is fast for single runs but a shared service handling many concurrent CI pipeline requests can queue up and cause timeouts, leading CI to skip validation or show flaky results
  • Network policy blocking schema fetches — if Kubeconform fetches schemas at validation time (not pre-cached), a network policy blocking outbound HTTPS causes silent validation skips or crashes

External monitoring from Vigilmon watches the Kubeconform service health endpoint from outside the cluster, giving you an independent signal that manifest validation is available to your CI pipelines.

What you'll need

  • Kubeconform deployed as an HTTP validation service or webhook
  • The service reachable externally or via Ingress
  • A free Vigilmon account

Step 1: Deploy Kubeconform as a validation HTTP service

Wrap Kubeconform in a lightweight HTTP server that accepts manifest payloads for validation:

# Dockerfile
FROM ghcr.io/yannh/kubeconform:latest AS kubeconform

FROM python:3.12-slim
COPY --from=kubeconform /kubeconform /usr/local/bin/kubeconform

COPY server.py /app/server.py
WORKDIR /app
RUN pip install flask gunicorn

EXPOSE 8080
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:8080", "server:app"]

# server.py
import subprocess, tempfile, os, json
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.get("/healthz")
def healthz():
    return "OK", 200

@app.get("/readyz")
def readyz():
    result = subprocess.run(
        ["kubeconform", "--version"],
        capture_output=True, text=True, timeout=5
    )
    if result.returncode == 0:
        return "OK", 200
    return "kubeconform unavailable", 503

@app.post("/validate")
def validate():
    manifest = request.get_data(as_text=True)
    with tempfile.NamedTemporaryFile(suffix=".yaml", delete=False, mode="w") as f:
        f.write(manifest)
        fname = f.name
    try:
        result = subprocess.run(
            ["kubeconform", "-output", "json", "-summary", fname],
            capture_output=True, text=True, timeout=30
        )
        os.unlink(fname)
        return jsonify(json.loads(result.stdout or "{}")), 200
    except Exception as e:
        return jsonify({"error": str(e)}), 500

Deploy to Kubernetes:

# kubeconform-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kubeconform
  namespace: ci-tools
spec:
  replicas: 2
  selector:
    matchLabels:
      app: kubeconform
  template:
    metadata:
      labels:
        app: kubeconform
    spec:
      containers:
        - name: kubeconform
          image: your-registry/kubeconform-server:latest
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /readyz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 15
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 30
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 1000m
              memory: 512Mi
---
apiVersion: v1
kind: Service
metadata:
  name: kubeconform
  namespace: ci-tools
spec:
  selector:
    app: kubeconform
  ports:
    - port: 8080
      targetPort: 8080

Expose via Ingress:

# kubeconform-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kubeconform-ingress
  namespace: ci-tools
spec:
  ingressClassName: nginx
  rules:
    - host: validate.yourdomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kubeconform
                port:
                  number: 8080

kubectl apply -f kubeconform-deployment.yaml
kubectl apply -f kubeconform-ingress.yaml

# Verify endpoints
curl https://validate.yourdomain.com/healthz
# OK

curl https://validate.yourdomain.com/readyz
# OK

# Test with a valid manifest
curl -X POST https://validate.yourdomain.com/validate \
  -H "Content-Type: application/yaml" \
  --data-binary @test-manifest.yaml | jq .

Step 2: Understand Kubeconform validation output

A healthy /validate response looks like:

{
  "resources": [
    {
      "filename": "manifest.yaml",
      "kind": "Deployment",
      "name": "my-app",
      "version": "apps/v1",
      "status": "statusValid",
      "msg": ""
    }
  ],
  "summary": {
    "valid": 1,
    "invalid": 0,
    "errors": 0,
    "skipped": 0
  }
}

A high skipped count in the summary indicates that resource types are being skipped due to missing schemas — typically CRDs that are not in Kubeconform's schema registry. A high errors count on a test manifest indicates the validation engine itself is encountering problems.

Step 3: Set up HTTP monitoring in Vigilmon

With Kubeconform endpoints accessible, add them to Vigilmon:

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Add a monitor for each endpoint:

| Monitor name | URL | Expected status | |---|---|---| | Kubeconform health | https://validate.yourdomain.com/healthz | 200 | | Kubeconform readiness | https://validate.yourdomain.com/readyz | 200 |

  1. Set the check interval to 1 minute
  2. For the health monitor, under Expected response, match OK in the response body
  3. For the readiness monitor, also match OK — a non-OK response means the kubeconform binary itself is not responding
  4. Optionally add a POST monitor to /validate with a known-good test manifest to verify end-to-end validation:
    • Method: POST
    • Body: a minimal valid Kubernetes Deployment YAML
    • Expected response body match: statusValid
  5. Save each monitor

The POST validation monitor is the most valuable check — it verifies not just that the HTTP server is alive, but that the kubeconform binary can execute a full schema validation.

Step 4: Monitor the Kubeconform TCP port

Add a TCP-layer monitor:

  1. Go to Monitors → New Monitor
  2. Choose TCP Port
  3. Enter your Kubeconform hostname and port 8080
  4. Save the monitor

A TCP failure when pods are Running indicates an Ingress misconfiguration, a Service selector mismatch, or a network policy blocking the health check path.

Step 5: Configure alert channels

Kubeconform validation service downtime means CI pipelines are running without manifest validation — invalid configurations can reach production. Treat this as high urgency.

Email alerts

  1. Go to Alert Channels → Add Channel → Email
  2. Enter your platform engineering team and CI/CD on-call address
  3. Assign the channel to all Kubeconform monitors

Webhook alerts for incident routing

  1. Go to Alert Channels → Add Channel → Webhook
  2. Enter your incident management webhook URL (PagerDuty, Opsgenie, Slack)
  3. The payload Vigilmon sends:
{
  "monitor_name": "Kubeconform health",
  "status": "down",
  "url": "https://validate.yourdomain.com/healthz",
  "started_at": "2024-01-15T10:23:00Z",
  "duration_seconds": 120
}

Wire this alert to a runbook that immediately notifies CI pipeline owners that manifest validation is unavailable, applies a temporary hold on production deployments, and restarts the Kubeconform service.

Step 6: Correlate Vigilmon alerts with Kubeconform diagnostics

When you receive a Kubeconform downtime alert, run these checks:

# 1. Check pod status
kubectl get pods -n ci-tools -l app=kubeconform

# 2. Check pod logs for crash or schema errors
kubectl logs -n ci-tools -l app=kubeconform --tail=100

# 3. Verify kubeconform binary is intact inside the container
kubectl exec -n ci-tools -l app=kubeconform -- kubeconform --version

# 4. Test schema fetch directly
kubectl exec -n ci-tools -l app=kubeconform -- \
  kubeconform -schema-location default \
  /dev/stdin <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
  - name: test
    image: nginx:latest
EOF

# 5. Check for OOM kills
kubectl describe pod -n ci-tools -l app=kubeconform | grep -E "OOMKilled|Exit Code|Reason"

# 6. Check if schema source is reachable
kubectl exec -n ci-tools -l app=kubeconform -- \
  curl -s --max-time 5 https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.0/pod.json | head -5

# 7. Check Ingress and Service
kubectl describe ingress -n ci-tools kubeconform-ingress
kubectl describe svc -n ci-tools kubeconform
kubectl get endpoints -n ci-tools kubeconform

# 8. Load test the service to check for queue buildup
for i in $(seq 1 10); do
  time curl -s -X POST https://validate.yourdomain.com/validate \
    -H "Content-Type: application/yaml" \
    --data "apiVersion: v1\nkind: Pod\nmetadata:\n  name: test$i\nspec:\n  containers:\n  - name: c\n    image: nginx" &
done
wait

If Vigilmon shows the health endpoint responding but the end-to-end validation test is slow, Kubeconform is receiving requests but the kubeconform binary or schema fetches are taking too long — consider pre-caching schemas in the container image rather than fetching them at validation time.

Step 7: Create a status page for CI/CD validation infrastructure

Kubeconform validation is a deployment safety gate. Make its status visible to engineering leadership:

  1. Go to Status Pages → New Status Page
  2. Name it: "CI/CD Validation Infrastructure"
  3. Add your monitors: Kubeconform health, readiness, and end-to-end validation
  4. Share the URL with platform engineering, development team leads, and release managers

Summary

| What you set up | What it catches | |---|---| | HTTP monitor on /healthz | HTTP server crash, pod failure | | HTTP monitor on /readyz | kubeconform binary unavailable | | HTTP POST monitor on /validate | End-to-end validation pipeline failure | | TCP monitor on port 8080 | NetworkPolicy blocks, Service selector drift | | Email + webhook alert channels | Immediate notification when validation service is down | | Status page | Visibility into CI/CD validation gate health |

Kubeconform is the last line of defense against invalid Kubernetes manifests reaching your cluster. External monitoring ensures that the validation service is always available so your CI pipelines never silently bypass schema checks and let broken configurations into production.

Get started at vigilmon.online — free tier, no credit card required.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles