Monitoring kube-hunter in Kubernetes Security Pipelines with Vigilmon

kube-hunter is Aqua Security's open-source penetration testing tool for Kubernetes clusters. It probes your cluster for attack vectors—unauthenticated API server access, exposed dashboards, privilege escalation paths, and more. Running it as a scheduled CronJob gives you continuous regression detection; but if the job silently fails, you lose your security posture signal entirely.

This tutorial shows how to run kube-hunter as a scheduled Kubernetes CronJob, expose its health and results via a lightweight status API, and use Vigilmon to alert you the moment a scan stops completing or a new critical finding appears.

Why kube-hunter needs external monitoring

kube-hunter itself does not have a built-in alerting layer for job health. If the CronJob fails to start (image pull error, RBAC misconfiguration, resource pressure), you get silence—not an alert. Silent failure in a security scanning pipeline is indistinguishable from "no new findings," which is exactly the wrong mental model.

External monitoring catches:

• CronJob never ran — schedule drift, namespace deletion, or the kube-hunter ServiceAccount being revoked
• Scan completed but returned non-zero exit code — network errors, API server timeouts, or a partial scan that produced no report
• Report generation failed — the JSON output was not written to the expected ConfigMap or PVC
• Finding regression — a previously remediated vulnerability reappears in the latest scan

What you'll need

• A Kubernetes cluster (1.24+)
• kubectl access with permission to create CronJobs, ServiceAccounts, and ClusterRoles
• A free Vigilmon account

Step 1: Deploy kube-hunter as a scheduled CronJob

Create the RBAC resources and the CronJob:

# kube-hunter-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-hunter
  namespace: security
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-hunter
rules:
  - apiGroups: [""]
    resources: ["nodes", "pods", "services", "namespaces", "endpoints"]
    verbs: ["get", "list"]
  - apiGroups: ["apps"]
    resources: ["deployments", "daemonsets", "replicasets"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-hunter
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-hunter
subjects:
  - kind: ServiceAccount
    name: kube-hunter
    namespace: security

# kube-hunter-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: kube-hunter-scan
  namespace: security
spec:
  schedule: "0 2 * * *"        # daily at 02:00 UTC
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: kube-hunter
          restartPolicy: Never
          containers:
            - name: kube-hunter
              image: aquasec/kube-hunter:latest
              args:
                - "--pod"
                - "--report=json"
                - "--output=/reports/report.json"
              volumeMounts:
                - name: reports
                  mountPath: /reports
          # Sidecar: pushes heartbeat to Vigilmon after scan completes
          initContainers: []
          volumes:
            - name: reports
              emptyDir: {}

Apply both manifests:

kubectl apply -f kube-hunter-rbac.yaml
kubectl apply -f kube-hunter-cronjob.yaml

Step 2: Add a heartbeat sidecar

The simplest way to monitor job completion is to push a heartbeat to Vigilmon at the end of a successful scan. Wrap kube-hunter in a shell script that runs the scan and then POSTs the heartbeat:

# kube-hunter-cronjob.yaml (updated container section)
containers:
  - name: kube-hunter
    image: aquasec/kube-hunter:latest
    command: ["/bin/sh", "-c"]
    args:
      - |
        kube-hunter --pod --report=json --output=/reports/report.json
        EXIT_CODE=$?
        if [ $EXIT_CODE -eq 0 ]; then
          # Push heartbeat to Vigilmon — replace TOKEN with your monitor token
          wget -q -O- "https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_TOKEN" || true
        else
          echo "kube-hunter exited with code $EXIT_CODE — skipping heartbeat"
          exit $EXIT_CODE
        fi
    volumeMounts:
      - name: reports
        mountPath: /reports

Step 3: Add a findings status endpoint

Deploy a lightweight sidecar or standalone Pod that reads the latest report and exposes a /health endpoint returning HTTP 200 (no new critical findings) or HTTP 503 (critical findings detected):

# scan-status-server.py
from flask import Flask, jsonify
import json, os, time

app = Flask(__name__)
REPORT_PATH = os.environ.get("REPORT_PATH", "/reports/report.json")
MAX_AGE_HOURS = int(os.environ.get("MAX_AGE_HOURS", "25"))

@app.get("/health")
def health():
    if not os.path.exists(REPORT_PATH):
        return jsonify({"status": "no_report", "error": "Report not found"}), 503

    mtime = os.path.getmtime(REPORT_PATH)
    age_hours = (time.time() - mtime) / 3600
    if age_hours > MAX_AGE_HOURS:
        return jsonify({"status": "stale", "age_hours": round(age_hours, 1)}), 503

    with open(REPORT_PATH) as f:
        report = json.load(f)

    critical = [v for v in report.get("vulnerabilities", []) if v.get("severity") == "high"]
    if critical:
        return jsonify({"status": "critical_findings", "count": len(critical)}), 503

    return jsonify({"status": "ok", "age_hours": round(age_hours, 1)}), 200

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

# scan-status-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-hunter-status
  namespace: security
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-hunter-status
  template:
    metadata:
      labels:
        app: kube-hunter-status
    spec:
      containers:
        - name: status-server
          image: python:3.12-slim
          command: ["python", "/app/scan-status-server.py"]
          env:
            - name: REPORT_PATH
              value: "/reports/report.json"
            - name: MAX_AGE_HOURS
              value: "25"
          ports:
            - containerPort: 8080
          volumeMounts:
            - name: reports
              mountPath: /reports
      volumes:
        - name: reports
          persistentVolumeClaim:
            claimName: kube-hunter-reports
---
apiVersion: v1
kind: Service
metadata:
  name: kube-hunter-status
  namespace: security
spec:
  selector:
    app: kube-hunter-status
  ports:
    - port: 80
      targetPort: 8080

Step 4: Configure Vigilmon monitors

Heartbeat monitor (scan completion)

1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
2. Set Expected interval: 25h (daily scan with buffer)
3. Copy the heartbeat URL and paste it into your CronJob script as YOUR_HEARTBEAT_TOKEN
4. Set alert contacts for Slack and email

If the CronJob never pushes the heartbeat within 25 hours, Vigilmon fires an alert.

HTTP monitor (findings status)

1. Go to Monitors → New Monitor → HTTP
2. URL: https://your-cluster-ingress/kube-hunter/health (or the LoadBalancer IP)
3. Check interval: 5 minutes
4. Expected status: 200
5. Alert on: status 503 (critical findings) or timeout

Step 5: Test end-to-end

Trigger a manual scan run:

kubectl create job --from=cronjob/kube-hunter-scan manual-scan -n security
kubectl logs -f job/manual-scan -n security

Verify the heartbeat reaches Vigilmon:

curl -s https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_TOKEN
# {"status":"ok"}

Simulate a stale report by rolling back mtime:

kubectl exec -n security deploy/kube-hunter-status -- \
  touch -d "26 hours ago" /reports/report.json
curl -s http://localhost:8080/health
# {"status":"stale","age_hours":26.0}  →  HTTP 503 → Vigilmon alert fires

Vigilmon integration summary

| Signal | Monitor type | Alert condition | |--------|-------------|-----------------| | Scan ran and completed | Heartbeat | No ping in 25h | | No critical findings | HTTP /health | HTTP 503 | | Status server is up | HTTP | Timeout / non-200 |

With this setup Vigilmon gives you continuous assurance that your kube-hunter pipeline is alive and your cluster has not regressed to a known-vulnerable state—without polling Kubernetes logs manually.

Next steps

• Export the JSON report to a ConfigMap and diff it against the previous run to detect new findings
• Integrate kube-hunter results into your incident management workflow via Vigilmon webhooks
• Add a Vigilmon status page badge to your internal security dashboard so every team can see K8s security posture at a glance