How to Monitor Headlamp with Vigilmon

Headlamp is an extensible, CNCF sandbox Kubernetes UI that makes it easy to visualize and manage your clusters through a clean browser interface and a rich plugin ecosystem. When Headlamp goes down — or its plugins stop loading — every developer and operator who depends on it loses visibility into the cluster. External uptime monitoring ensures you know before they do.

In this tutorial you'll configure Vigilmon to monitor your Headlamp server, track plugin health, and alert your team the moment cluster connectivity or the UI itself becomes unavailable.

Why Headlamp needs external monitoring

Headlamp runs as a stateless web server that proxies Kubernetes API requests from the browser. That architecture means there are several distinct failure modes that internal Kubernetes probes won't catch:

Headlamp server is down — the container is running but the HTTP server has crashed; users see a blank page
Plugin manifest unreachable — a CDN or object-storage URL for a plugin artifact 404s silently; the UI loads but features are missing
Cluster connectivity severed — the in-cluster service account token has expired or RBAC changed; the UI loads but every resource fetch returns 403
Multi-cluster proxy broken — one of several registered clusters loses connectivity; cross-cluster navigation silently fails
TLS certificate expired — browser enforces HTTPS strict; Headlamp becomes unreachable without an obvious error message

Headlamp's own liveness probe only tells you the process is alive. It cannot tell you whether the Kubernetes API is reachable, whether plugins are loading, or whether real users can authenticate.

What you'll need

Headlamp deployed in-cluster (Helm chart or raw manifests) or as a standalone desktop/server binary
Headlamp exposed via a Kubernetes Service of type LoadBalancer or through an Ingress controller
A free Vigilmon account

Step 1: Expose Headlamp and add a health endpoint

The official Headlamp Helm chart exposes a /healthz endpoint on the server. Verify it is reachable:

# Port-forward for a quick check
kubectl port-forward -n headlamp svc/headlamp 4466:80

curl -s http://localhost:4466/healthz
# Expected: {"status":"ok"}

If you are using the Helm chart, confirm the Service is exposed externally:

kubectl get svc -n headlamp
# NAME       TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)
# headlamp   LoadBalancer   10.96.42.7      203.0.113.25     80:31234/TCP

If EXTERNAL-IP is <pending>, either annotate the service for your cloud provider's LB controller or set up an Ingress:

# headlamp-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: headlamp
  namespace: headlamp
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - headlamp.example.com
      secretName: headlamp-tls
  rules:
    - host: headlamp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: headlamp
                port:
                  number: 80

Apply it:

kubectl apply -f headlamp-ingress.yaml

Step 2: Add a plugin health endpoint

Headlamp plugins are loaded as JavaScript modules from URLs listed in the Headlamp config. You can expose a lightweight plugin-manifest endpoint by adding a ConfigMap that Headlamp serves:

# headlamp-plugin-check.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: headlamp-plugin-manifest
  namespace: headlamp
data:
  plugins.json: |
    [
      {"name": "prometheus-metrics", "version": "1.2.0", "url": "https://cdn.example.com/plugins/prometheus-metrics.js"},
      {"name": "flux-cd", "version": "0.9.1", "url": "https://cdn.example.com/plugins/flux-cd.js"}
    ]

Then add a sidecar or init container that periodically verifies each plugin URL is reachable and surfaces the result at /plugin-health:

# In your Headlamp Deployment spec, add a sidecar:
- name: plugin-healthcheck
  image: curlimages/curl:8.5.0
  command:
    - /bin/sh
    - -c
    - |
      while true; do
        STATUS=0
        for URL in https://cdn.example.com/plugins/prometheus-metrics.js \
                   https://cdn.example.com/plugins/flux-cd.js; do
          curl -fsS --max-time 5 "$URL" -o /dev/null || STATUS=1
        done
        echo $STATUS > /tmp/plugin-status
        sleep 30
      done
  volumeMounts:
    - name: plugin-status
      mountPath: /tmp

Expose the status via a tiny HTTP server or use an existing sidecar pattern. For most teams, monitoring the main Headlamp /healthz endpoint is sufficient.

Step 3: Set up Vigilmon HTTP monitoring

Click Add Monitor → HTTP(S)
Set the URL to https://headlamp.example.com/healthz
Set Check interval to 1 minute
Set Expected status code to 200
Enable String assertion: assert the response body contains "status":"ok"
Set Timeout to 10 seconds
Name it Headlamp – Health
Click Save

Vigilmon will now hit your Headlamp health endpoint from multiple regions every minute and alert you if it goes down.

Step 4: Monitor cluster connectivity via a dedicated probe endpoint

Create a lightweight health endpoint inside the cluster that verifies Headlamp can reach the Kubernetes API. Add this as a separate Deployment:

# k8s-probe.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-api-probe
  namespace: headlamp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: k8s-api-probe
  template:
    metadata:
      labels:
        app: k8s-api-probe
    spec:
      serviceAccountName: headlamp
      containers:
        - name: probe
          image: bitnami/kubectl:latest
          command:
            - /bin/sh
            - -c
            - |
              while true; do
                if kubectl get --raw /healthz > /dev/null 2>&1; then
                  echo "OK" > /tmp/status
                else
                  echo "FAIL" > /tmp/status
                fi
                sleep 15
              done
          livenessProbe:
            exec:
              command: ["test", "-f", "/tmp/status"]
            periodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  name: k8s-api-probe
  namespace: headlamp
spec:
  selector:
    app: k8s-api-probe
  ports:
    - port: 8080
      targetPort: 8080

Then in Vigilmon, add a second HTTP monitor:

URL: https://headlamp.example.com/api/v1/namespaces (proxied through Headlamp)
Method: GET
Expected status code: 200 or 401 (401 means the API is reachable but requires auth — which is fine)
Name: Headlamp – Kubernetes API Reachability

Step 5: Multi-cluster access health

If you run Headlamp in multi-cluster mode, add one HTTP monitor per cluster endpoint:

https://headlamp.example.com/healthz              → Headlamp – Cluster: production
https://headlamp.example.com/healthz              → Headlamp – Cluster: staging

Use Vigilmon tags to group them:

Tag all monitors with headlamp for a single-pane status view
Tag by cluster name (production, staging) for granular alerting

Step 6: Configure alerts

In Vigilmon, go to Alert Channels → Add Channel
Add Slack, PagerDuty, or email
Set Alert after: 2 consecutive failures (avoids transient noise)
Set Recovery alert: enabled (know when it comes back)
Assign the alert channel to all headlamp-tagged monitors

A sample Slack message template:

:red_circle: *Headlamp DOWN*
Monitor: {{ monitor.name }}
URL: {{ monitor.url }}
Region: {{ check.region }}
Time: {{ check.timestamp }}
Error: {{ check.error }}

Step 7: Status page (optional)

If your team uses Headlamp as a shared ops tool, expose a public or private status page in Vigilmon:

Go to Status Pages → New Status Page
Add all headlamp-tagged monitors
Set visibility: Private (shared link only, for internal teams)
Share the URL in your team Slack channel

What to monitor — checklist

| Monitor | URL | Assertion | |---|---|---| | Headlamp UI health | /healthz | 200, body ok | | Kubernetes API reachability | /api/v1/namespaces | 200 or 401 | | Plugin CDN availability | Plugin JS URL | 200 | | TLS certificate validity | Full HTTPS URL | Cert expiry > 14 days |

Summary

Headlamp is a developer-facing tool with no tolerance for silent failures. A down Headlamp server means engineers can't inspect pods, view logs, or navigate resources — they're flying blind during an incident. With Vigilmon monitoring /healthz, cluster connectivity, and plugin availability, you get an early warning system that fires before anyone files a support ticket.