tutorial

Monitoring Knative Serverless Services with Vigilmon: Scale-to-Zero, Gateway Health, and Control Plane Checks

How to monitor Knative with Vigilmon — external HTTP checks that account for scale-to-zero behavior, ingress gateway TCP monitoring, Knative control plane health, and TLS certificate alerts.

How to Monitor Knative Serverless Services with Vigilmon

Knative transforms Kubernetes into a serverless platform — your applications scale automatically from zero to thousands of replicas based on HTTP traffic, without managing pods or deployments directly. It's an elegant abstraction, but it introduces monitoring challenges that neither traditional Kubernetes monitoring nor pure serverless tooling handles well.

When a Knative Service receives no traffic, it scales to zero — there are literally no pods running. When traffic arrives, Knative's activator component buffers the request while cold-starting a new pod. This scale-from-zero cycle is healthy behavior, but it means the absence of running pods is expected, not an alert condition. Your existing Kubernetes monitoring tools will constantly false-alarm about "no pods running" for perfectly healthy services.

At the same time, real failures do happen. The Knative control plane (Serving controller, activator, webhook) can crash. Cold start times can balloon when there's resource pressure on the cluster. Knative's ingress gateway (Istio, Kourier, or Ambassador) can have routing issues that prevent traffic from reaching services. And domain configuration errors silently break external accessibility.

This tutorial covers setting up effective external uptime monitoring for Knative using Vigilmon — one that accounts for scale-to-zero behavior and focuses on what actually matters: whether your services respond to requests.

Why Knative needs external monitoring

Knative's built-in observability (via Prometheus metrics and the Knative dashboard) gives you performance visibility. What it doesn't give you:

  • End-to-end reachability — can external users reach your service through the ingress gateway, DNS, and TLS termination? Kubernetes metrics won't show you routing problems at the edge
  • Cold start SLA violations — if cold start times exceed acceptable thresholds (say, > 5 seconds), users experience errors; internal metrics don't catch this from the user's perspective
  • Knative control plane failures — if the Serving controller crashes, existing services keep running, but new revisions can't be deployed; no obvious alert is generated
  • Ingress gateway downtime — Kourier or Istio outages stop all Knative traffic instantly; Kubernetes reports pods as healthy while every request fails
  • Certificate expiry — Knative can manage TLS certificates automatically, but cert renewal failures are silent until the cert actually expires
  • DNS misconfiguration — your custom domain doesn't resolve to the Knative gateway; the service appears healthy internally but is completely inaccessible externally

External monitoring from Vigilmon catches all of these because it makes real HTTP requests from outside your cluster — the same path your users take.

What you'll need

  • A Kubernetes cluster with Knative Serving installed
  • At least one deployed Knative Service accessible via HTTP(S)
  • A free Vigilmon account

Step 1: Deploy a Knative Service with a health endpoint

If you don't have a Knative Service yet, here's a minimal example:

# knative-hello.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: hello
  namespace: default
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/min-scale: "0"  # enable scale-to-zero
        autoscaling.knative.dev/max-scale: "10"
    spec:
      containers:
        - image: gcr.io/knative-samples/helloworld-go
          ports:
            - containerPort: 8080
          env:
            - name: TARGET
              value: "Vigilmon"
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 0
            periodSeconds: 3

kubectl apply -f knative-hello.yaml

# Get the service URL
kubectl get ksvc hello
# NAME    URL                                           LATESTCREATED   LATESTREADY   READY
# hello   https://hello.default.example.com            hello-00001     hello-00001   True

For a production service, make sure your application exposes a /health endpoint:

// Go example
http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"status":"ok"}`))
})

# Python/FastAPI example
@app.get("/health")
async def health():
    return {"status": "ok"}

Step 2: Configure your Knative ingress for external access

Knative Serving uses an ingress gateway to route external traffic. The gateway address is what you'll monitor in Vigilmon.

# For Kourier (default for many installations)
kubectl get svc -n kourier-system
# NAME               TYPE           EXTERNAL-IP      PORT(S)
# kourier            LoadBalancer   203.0.113.100    80:31080/TCP,443:31443/TCP

# For Istio
kubectl get svc -n istio-system istio-ingressgateway
# NAME                   TYPE           EXTERNAL-IP      PORT(S)
# istio-ingressgateway   LoadBalancer   203.0.113.200    80:31380/TCP,443:31443/TCP

Make sure your service URL resolves correctly:

# Test the service (this triggers a cold start if scaled to zero)
curl https://hello.default.example.com/health
# {"status":"ok"}

# Or using the IP directly with Host header (useful if DNS isn't set up yet)
curl -H "Host: hello.default.example.com" http://203.0.113.100/health

Step 3: Set up HTTP monitoring in Vigilmon

Now add your Knative service URL to Vigilmon:

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. URL: https://hello.default.example.com/health
  4. Check interval: 1 minute
  5. Expected response:
    • Status code: 200
    • Response body contains: "status":"ok"
  6. Name the monitor: "Knative hello service"
  7. Save the monitor

Important: scale-to-zero and Vigilmon's check interval

Vigilmon pinging your service every minute means the service will never scale to zero — the monitoring probe counts as traffic. This is intentional for production services: keeping a warm instance eliminates cold start latency for your real users. Set autoscaling.knative.dev/min-scale: "1" for services where you want this behavior explicitly.

For services where you genuinely want scale-to-zero (e.g., internal tools or batch processors), consider setting the Vigilmon check interval to 5 minutes or longer, and document that cold starts up to a few seconds are expected.

Multi-region consensus and cold starts

Vigilmon uses multi-region consensus — multiple probes from different locations must agree a service is down before firing an alert. This naturally handles the brief (< 1 second) window where a cold-starting Knative pod is being provisioned. A true outage will be confirmed by all probes simultaneously, while a temporary cold-start delay will only affect the one probe that happened to arrive during scale-up.

Step 4: Monitor the Knative ingress gateway

Your Knative services are only as reachable as the ingress gateway in front of them. Monitor the gateway itself:

  1. In Vigilmon, go to Monitors → New Monitor
  2. Choose TCP Port
  3. Host: 203.0.113.100 (your gateway's external IP), Port: 443
  4. Save as "Knative Gateway HTTPS Port"

Also add port 80:

  • Host: gateway IP, Port: 80 — "Knative Gateway HTTP Port"

If both TCP monitors go dark simultaneously, the gateway itself is down — not just one service.

Step 5: Monitor the Knative control plane

The Knative Serving controller handles deployments of new revisions. If it's unhealthy, your services keep serving traffic from existing revisions, but you lose the ability to deploy. Add a health check for the control plane components.

# Check control plane services
kubectl get pods -n knative-serving
# NAME                          READY   STATUS
# activator-xxx                 2/2     Running
# autoscaler-xxx                2/2     Running
# controller-xxx                2/2     Running
# webhook-xxx                   2/2     Running

# Check if any expose a health endpoint
kubectl get svc -n knative-serving

Deploy a health proxy similar to the KubeArmor approach:

# knative-control-health.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: health-script
  namespace: knative-serving
data:
  health.py: |
    import subprocess, json
    from http.server import HTTPServer, BaseHTTPRequestHandler

    DEPLOYMENTS = ['controller', 'autoscaler', 'activator', 'webhook']

    class Health(BaseHTTPRequestHandler):
        def do_GET(self):
            if self.path != '/health':
                self.send_response(404); self.end_headers(); return
            statuses = {}
            healthy = True
            for dep in DEPLOYMENTS:
                try:
                    r = subprocess.run(['kubectl', 'get', 'deploy', dep,
                                        '-n', 'knative-serving', '-o',
                                        'jsonpath={.status.readyReplicas}'],
                                       capture_output=True, text=True, timeout=5)
                    ready = int(r.stdout.strip() or 0)
                    statuses[dep] = ready
                    if ready == 0:
                        healthy = False
                except Exception:
                    statuses[dep] = -1
                    healthy = False
            code = 200 if healthy else 503
            body = json.dumps({"status": "ok" if healthy else "degraded",
                               "components": statuses}).encode()
            self.send_response(code)
            self.send_header('Content-Type', 'application/json')
            self.end_headers()
            self.wfile.write(body)
        def log_message(self, *args): pass

    HTTPServer(('0.0.0.0', 8090), Health).serve_forever()

Expose this via a LoadBalancer service and add an HTTP monitor in Vigilmon:

  • URL: http://<health-proxy-ip>:8090/health
  • Expected: status 200, body contains "status":"ok"
  • Name: "Knative Control Plane Health"

Step 6: Monitor TLS certificates

Knative can use cert-manager to provision and renew TLS certificates automatically. Certificate expiry is a common cause of Knative service outages.

Vigilmon automatically checks TLS certificate validity for HTTPS monitors. Configure it to alert before expiry:

  1. Edit your HTTPS monitor in Vigilmon
  2. Under SSL Certificate, enable "Alert when certificate expires within X days"
  3. Set the threshold to 30 days — enough time to investigate and fix cert renewal issues before they cause an outage

This catches cert-manager failures before they cause a full outage.

Step 7: Configure alert channels

Knative service failures are user-facing, so alerts should reach the right team quickly.

Email alerts

  1. Go to Alert Channels → Add Channel → Email
  2. Add your on-call or platform engineering email
  3. Assign to all Knative monitors

Slack webhook

  1. Go to Alert Channels → Add Channel → Webhook
  2. Paste your Slack #incidents channel webhook
  3. Assign to monitors

Example Vigilmon alert for a Knative service outage:

{
  "monitor_name": "Knative hello service",
  "status": "down",
  "url": "https://hello.default.example.com/health",
  "started_at": "2026-07-02T16:30:00Z",
  "duration_seconds": 120
}

Step 8: Diagnose Knative failures

When Vigilmon alerts on a Knative service, follow this runbook:

# 1. Check if the Knative Service is Ready
kubectl get ksvc hello
# READY should be True

# 2. Check the latest revision status
kubectl get revision -l serving.knative.dev/service=hello
# Check READY column

# 3. Inspect the revision for errors
kubectl describe revision hello-00001

# 4. Check if pods are being created (for a cold start)
kubectl get pods -l serving.knative.dev/service=hello
# If empty with traffic incoming, activator might be stuck

# 5. Check the activator logs
kubectl logs -n knative-serving -l app=activator -c activator --tail=50

# 6. Check the autoscaler
kubectl logs -n knative-serving -l app=autoscaler --tail=50

# 7. Test via gateway IP directly (bypasses DNS)
GATEWAY_IP=$(kubectl get svc -n kourier-system kourier -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl -H "Host: hello.default.example.com" http://$GATEWAY_IP/health

# 8. Check Knative controller for reconciliation errors
kubectl logs -n knative-serving -l app=controller --tail=50 | grep -i error

Common Knative failure patterns

| Vigilmon alert | Probable cause | First check | |----------------|----------------|-------------| | Service HTTP 503 | No healthy pods, failed cold start | kubectl get pods, activator logs | | Service HTTP 404 | Knative routing issue or domain misconfiguration | kubectl get ksvc, ingress routes | | Service TCP down | Ingress gateway down | kubectl get pods -n kourier-system | | TLS warning | Cert about to expire | kubectl get cert -A, cert-manager logs | | Control plane HTTP 503 | Controller or webhook down | kubectl get pods -n knative-serving |

Step 9: Create a status page for your Knative services

  1. In Vigilmon, go to Status Pages → New Status Page
  2. Name: "Serverless Services Status"
  3. Add monitors:
    • Individual Knative services
    • Knative Gateway
    • Knative Control Plane
  4. Group by service category (API, Processing, Internal)
  5. Publish and share

Recommended monitoring setup for Knative

| Monitor | Type | What it detects | |---------|------|-----------------| | https://service.example.com/health | HTTP | Service end-to-end reachability | | gateway-ip:443 | TCP | Ingress gateway HTTPS port | | gateway-ip:80 | TCP | Ingress gateway HTTP port | | Control plane health proxy | HTTP | Controller, activator, autoscaler, webhook | | TLS certificate check | SSL | Certificate expiry (via HTTPS monitor) |

What's next

With Vigilmon monitoring your Knative services externally, you'll catch outages from the user's perspective — the only perspective that matters for production availability.

Next steps for Knative observability maturity:

  • Latency monitoring: use Vigilmon's response time tracking to establish cold start baselines; a sudden increase in P95 latency often precedes reliability issues
  • Canary deployment verification: when deploying a new Knative revision, watch Vigilmon's response during the traffic split — if error rates rise, roll back immediately
  • Multi-service dependency mapping: for Knative services that call each other, monitor each service independently; a downstream failure cascades, but Vigilmon will show you exactly which service went dark first

Monitor your Knative serverless services from the outside in — start at vigilmon.online — free tier, no credit card required.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles