How to Monitor kube-ps1 with Vigilmon

kube-ps1 is a shell prompt plugin for bash and zsh that displays the currently active Kubernetes context and namespace directly in the terminal prompt, giving engineers a constant visual reminder of which cluster and namespace their kubectl commands will target. In environments with multiple clusters — development, staging, production, and multiple regions — kube-ps1 is the primary guard against running a destructive command against the wrong cluster. When the kubeconfig that kube-ps1 reads becomes stale, the API server that validates the active context is unreachable, or a namespace displayed in the prompt is silently deleted, kube-ps1 continues showing an outdated context and namespace in the prompt without any indication that the underlying cluster state has changed.

In this tutorial you'll set up uptime monitoring for the Kubernetes infrastructure that kube-ps1 reads from using Vigilmon — free tier, no credit card required.

Why kube-ps1's dependencies need external monitoring

kube-ps1 reads from ~/.kube/config and optionally queries the Kubernetes API for live context data. Its failure modes span both local config and remote API:

• Stale context in the prompt — kube-ps1 displays the context and namespace stored in kubeconfig at shell initialization time; if another terminal switches the context after the shell was opened, kube-ps1 shows the old context while kubectl uses the new one — the prompt becomes misleading rather than protective
• Deleted namespace still shown in prompt — kube-ps1 reads the active namespace from kubeconfig metadata, not from the cluster; if a namespace is deleted from the cluster, kube-ps1 continues displaying it in the prompt while all kubectl commands in that namespace return 404 errors
• Expired credentials masking as a healthy context — kube-ps1 shows a context name and namespace regardless of whether the credentials embedded in that context are still valid; engineers believe they have a working connection when they do not
• kubeconfig file unavailable — if the kubeconfig file is deleted, on a network filesystem that becomes unavailable, or if file permissions change, kube-ps1 may render an empty or error prompt that engineers interpret as a shell issue rather than a Kubernetes connectivity issue
• Missing kube-ps1 binary after shell upgrade — if bash or zsh is upgraded and the kube-ps1 source line in .bashrc/.zshrc points to a path that no longer exists, the prompt silently drops the context indicator, removing the safety net without alerting the user

External monitoring from Vigilmon watches the Kubernetes API that kube-ps1's context data reflects and alerts you when the displayed context diverges from actual cluster state.

What you'll need

• kube-ps1 installed and active in your shell (bash or zsh)
• kubectl access to your monitored cluster
• A node or LoadBalancer endpoint to host the health check
• A free Vigilmon account

Step 1: Expose a health endpoint for kube-ps1's context data

kube-ps1 has no HTTP interface. Deploy a lightweight health exporter that validates the context, namespace, and cluster credentials that kube-ps1 reads from kubeconfig are accurate and active:

# kube-ps1-health.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-ps1-health
  namespace: observability
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-ps1-health-reader
rules:
  - apiGroups: [""]
    resources: ["namespaces", "nodes"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-ps1-health-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-ps1-health-reader
subjects:
  - kind: ServiceAccount
    name: kube-ps1-health
    namespace: observability
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-ps1-health
  namespace: observability
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-ps1-health
  template:
    metadata:
      labels:
        app: kube-ps1-health
    spec:
      serviceAccountName: kube-ps1-health
      containers:
        - name: checker
          image: bitnami/kubectl:latest
          command:
            - /bin/sh
            - -c
            - |
              while true; do
                # Check that the active context resolves to a reachable cluster
                CONTEXT=$(kubectl config current-context 2>/dev/null || echo "unknown")
                NS=$(kubectl config view --minify -o jsonpath='{.contexts[0].context.namespace}' 2>/dev/null || echo "default")
                NS=${NS:-default}

                # Verify the namespace shown in kube-ps1 actually exists
                if kubectl get namespace "$NS" > /dev/null 2>/tmp/ns_err; then
                  NS_STATUS="exists"
                else
                  NS_STATUS="missing"
                fi

                # Verify cluster is reachable (the state kube-ps1 implies is valid)
                if kubectl get nodes --no-headers > /tmp/nodes 2>/tmp/node_err; then
                  NODE_COUNT=$(wc -l < /tmp/nodes)
                  if [ "$NS_STATUS" = "exists" ]; then
                    echo "healthy context=$CONTEXT namespace=$NS nodes=$NODE_COUNT" > /tmp/status
                  else
                    echo "degraded context=$CONTEXT namespace=$NS ns_status=missing nodes=$NODE_COUNT" > /tmp/status
                  fi
                else
                  ERR=$(head -1 /tmp/node_err)
                  echo "unhealthy context=$CONTEXT namespace=$NS error=$ERR" > /tmp/status
                fi
                sleep 30
              done
        - name: health-server
          image: busybox:latest
          command:
            - /bin/sh
            - -c
            - |
              while true; do
                STATUS=$(cat /tmp/status 2>/dev/null || echo "starting")
                if echo "$STATUS" | grep -q "^healthy"; then
                  printf "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: ${#STATUS}\r\n\r\n$STATUS" | nc -l -p 8080 -q 1
                elif echo "$STATUS" | grep -q "^degraded"; then
                  printf "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: ${#STATUS}\r\n\r\n$STATUS" | nc -l -p 8080 -q 1
                else
                  printf "HTTP/1.1 503 Service Unavailable\r\nContent-Type: text/plain\r\nContent-Length: ${#STATUS}\r\n\r\n$STATUS" | nc -l -p 8080 -q 1
                fi
              done
          ports:
            - containerPort: 8080
              name: health
          volumeMounts:
            - name: status
              mountPath: /tmp
      volumes:
        - name: status
          emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: kube-ps1-health
  namespace: observability
spec:
  type: NodePort
  selector:
    app: kube-ps1-health
  ports:
    - name: health
      port: 8080
      targetPort: 8080
      nodePort: 30085

kubectl apply -f kube-ps1-health.yaml

# Verify
NODE_IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
curl http://$NODE_IP:30085/
# healthy context=production namespace=default nodes=3

Step 2: Validate that kube-ps1 prompt data matches actual cluster state

Before setting up monitoring, confirm that the context and namespace displayed in your prompt reflect the real cluster state:

#!/bin/bash
# check-kube-ps1-accuracy.sh

echo "=== kube-ps1 installation ==="
if [ -f /usr/local/opt/kube-ps1/share/kube-ps1.sh ] || \
   [ -f /usr/share/kube-ps1/kube-ps1.sh ] || \
   command -v kube_ps1 > /dev/null 2>&1; then
  echo "OK: kube-ps1 found"
else
  echo "WARNING: kube-ps1 not found in standard locations"
fi

echo ""
echo "=== Active context from kubeconfig ==="
KUBECONFIG_CTX=$(kubectl config current-context 2>/dev/null || echo "none")
echo "Context: $KUBECONFIG_CTX"

KUBECONFIG_NS=$(kubectl config view --minify -o jsonpath='{.contexts[0].context.namespace}' 2>/dev/null)
echo "Namespace: ${KUBECONFIG_NS:-default}"

echo ""
echo "=== Cluster reachability ==="
if kubectl get nodes --request-timeout=5s > /dev/null 2>&1; then
  echo "OK: cluster reachable for context $KUBECONFIG_CTX"
else
  echo "FAILED: cluster unreachable — kube-ps1 prompt is misleading"
fi

echo ""
echo "=== Namespace existence check ==="
NS=${KUBECONFIG_NS:-default}
if kubectl get namespace "$NS" > /dev/null 2>&1; then
  echo "OK: namespace '$NS' exists in cluster"
else
  echo "WARNING: namespace '$NS' shown in prompt does NOT exist in cluster"
fi

echo ""
echo "=== Credential validity ==="
kubectl auth whoami 2>/dev/null || kubectl auth can-i list namespaces 2>&1 | head -3

chmod +x check-kube-ps1-accuracy.sh
./check-kube-ps1-accuracy.sh

Step 3: Set up HTTP monitoring in Vigilmon

With the health endpoint running, add it to Vigilmon:

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS
3. Add monitors for kube-ps1's context infrastructure:

| Monitor name | URL | Expected status | |---|---|---| | kube-ps1 context health | http://your-node-ip:30085/ | 200 | | Kubernetes API livez | https://your-api-server:6443/livez | 200 | | Kubernetes API readyz | https://your-api-server:6443/readyz | 200 |

4. Set the check interval to 1 minute
5. Under Expected response, set status code 200 and match healthy in the response body for the health endpoint
6. For private CA clusters, add your CA certificate under TLS settings
7. Save each monitor

Step 4: Add a TCP monitor for the Kubernetes API server

Add a TCP monitor to detect network failures that make the cluster unreachable even though kube-ps1 still displays an active context:

1. Go to Monitors → New Monitor
2. Choose TCP Port
3. Enter your Kubernetes API server hostname and port 6443
4. Set the check interval to 1 minute
5. Save the monitor

A TCP failure on port 6443 while the kube-ps1 prompt still shows a healthy-looking context is the clearest sign that the prompt is misleading — the cluster shown is unreachable.

Step 5: Configure alert channels

kube-ps1 is a safety guardrail that tells engineers where their kubectl commands will land. When the cluster it shows is unreachable or the namespace it displays no longer exists, that guardrail provides false confidence. Alert the team immediately so they can verify their context before running cluster operations.

Email alerts

1. Go to Alert Channels → Add Channel → Email
2. Enter your SRE on-call and senior engineering addresses
3. Assign the channel to all kube-ps1 monitors

Webhook alerts

1. Go to Alert Channels → Add Channel → Webhook
2. Enter your incident management webhook URL (PagerDuty, Opsgenie, Slack)
3. Sample Vigilmon webhook payload:

{
  "monitor_name": "kube-ps1 context health",
  "status": "down",
  "url": "http://your-node-ip:30085/",
  "started_at": "2024-01-15T16:20:00Z",
  "duration_seconds": 90
}

Route this to a runbook that instructs engineers to run kubectl cluster-info before executing any cluster commands and to treat their terminal prompt context as unverified until the monitor recovers.

Step 6: Correlate Vigilmon alerts with kube-ps1 diagnostics

When you receive a kube-ps1 health downtime alert:

# 1. Verify what the kube-ps1 prompt actually shows vs. reality
echo "Prompt says context: $(kubectl config current-context)"
echo "Prompt says namespace: $(kubectl config view --minify -o jsonpath='{.contexts[0].context.namespace}')"

# 2. Test whether the displayed cluster is actually reachable
kubectl cluster-info

# 3. Verify the namespace shown in the prompt exists
ACTIVE_NS=$(kubectl config view --minify -o jsonpath='{.contexts[0].context.namespace}')
ACTIVE_NS=${ACTIVE_NS:-default}
kubectl get namespace "$ACTIVE_NS" && echo "OK: namespace exists" || echo "ALERT: namespace missing from cluster"

# 4. Check credential validity for the active context
kubectl auth can-i list pods

# 5. Look for recently deleted namespaces
kubectl get events --all-namespaces --field-selector reason=NamespaceDeleting 2>/dev/null | tail -5

# 6. Inspect the kube-ps1 health pod
kubectl logs -n observability -l app=kube-ps1-health -c checker --tail=20

# 7. If the namespace is missing, update the kubeconfig to point to an existing one
kubectl get namespaces
kubens <existing-namespace>   # if kubens is installed
# or
kubectl config set-context --current --namespace=<existing-namespace>

# 8. Verify kube-ps1 is sourced and active in the current shell
type kube_ps1 2>/dev/null || echo "kube-ps1 not active in this shell"

If the health check returns degraded with ns_status=missing, the namespace displayed in your prompt was deleted from the cluster. Update the active namespace via kubens or kubectl config set-context and confirm the health endpoint recovers to healthy.

Step 7: Create a status page for Kubernetes context safety

Make the accuracy of the kube-ps1 prompt visible to the whole engineering team:

1. Go to Status Pages → New Status Page
2. Name it: "Kubernetes Context Safety (kube-ps1)"
3. Add your kube-ps1 health monitor, API livez, and API readyz monitors
4. Share the URL with all engineers who use kubectl daily

When this page shows degraded, engineers across the team know to verify their context manually before running any cluster operations — even though their terminal prompt appears normal.

Summary

| What you set up | What it catches | |---|---| | HTTP monitor on health endpoint | Stale namespace in prompt, expired credentials, API unavailability | | HTTP monitor on livez/readyz | API server degradation that makes the displayed context unreliable | | TCP monitor on port 6443 | Network failures that disconnect the prompt context from the real cluster | | Email + webhook alert channels | Immediate notification when the kube-ps1 safety guardrail is misleading | | Status page | Team-wide visibility into kube-ps1 context accuracy |

kube-ps1 exists to prevent engineers from running commands against the wrong cluster. External monitoring ensures you know when the context it displays no longer matches cluster reality — before someone acts on a misleading prompt during a high-pressure incident.

Get started at vigilmon.online — free tier, no credit card required.