tutorial

How to Monitor Skaffold-Deployed Services with Vigilmon

A step-by-step guide to monitoring Kubernetes services deployed with Skaffold using Vigilmon — covering build pipeline health, endpoint checks, container health, and CI integration.

Skaffold handles the repetitive parts of Kubernetes development — it watches your source files, rebuilds container images, pushes them to a registry, and redeploys to your cluster. Whether you're running skaffold dev in a local loop or skaffold run in CI, Skaffold keeps your cluster synchronized with your code.

But Skaffold stops at deployment. Once your service is running, you need to know that it actually works — that the endpoint returns 200, that latency is within bounds, and that the build-deploy cycle didn't introduce a silent regression. Vigilmon provides external monitoring that runs independently of your Skaffold workflow and catches failures that Skaffold has no visibility into.

What Skaffold Cannot Tell You

Skaffold reports build success, push success, and deployment success. What it cannot report:

  • Whether the deployed container is serving HTTP traffic on the expected path
  • Whether external DNS resolves correctly after redeployment
  • Whether a new container image introduced a startup failure that only manifests under real traffic
  • Whether the Kubernetes Ingress is routing correctly after a profile switch
  • Whether response times degraded after a dependency update

These gaps are where monitoring lives. Skaffold and Vigilmon are complementary: Skaffold manages the inner loop, Vigilmon watches the outer edge.

What You'll Set Up

  • HTTP monitor for your Skaffold-deployed service endpoint
  • Heartbeat monitor to detect failed or stalled Skaffold pipelines in CI
  • Container health endpoint wired into your application
  • Alert routing to Slack or your preferred channel

You'll need a free Vigilmon account — no credit card required.

Step 1: Add a Health Endpoint to Your Application

Before Vigilmon can monitor your service, your application needs a health endpoint. This endpoint should verify real dependencies, not just that the process started.

# app.py — FastAPI example
from fastapi import FastAPI
from fastapi.responses import JSONResponse
import time
import os

app = FastAPI()
start_time = time.time()

@app.get("/health")
async def health():
    uptime = time.time() - start_time
    # Check any real dependencies here (DB, cache, etc.)
    return JSONResponse({
        "status": "ok",
        "uptime_seconds": round(uptime, 2),
        "version": os.getenv("APP_VERSION", "unknown")
    })

If you're using Skaffold profiles, inject the version via environment variable so you can confirm which build is actually running:

# skaffold.yaml
apiVersion: skaffold/v4beta11
kind: Config
build:
  artifacts:
    - image: my-api
      docker:
        dockerfile: Dockerfile
deploy:
  kubectl:
    manifests:
      - k8s/deployment.yaml
profiles:
  - name: staging
    deploy:
      kubectl:
        manifests:
          - k8s/deployment-staging.yaml

# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-api
  template:
    metadata:
      labels:
        app: my-api
    spec:
      containers:
        - name: my-api
          image: my-api
          env:
            - name: APP_VERSION
              value: "{{.IMAGE_TAG}}"
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10

Using {{.IMAGE_TAG}} as the version value means your health response always reflects the exact image deployed by Skaffold, making it trivial to correlate Vigilmon alerts with specific builds.

Step 2: Monitor Your Deployed Endpoint

Once your service is exposed via a Kubernetes Ingress or LoadBalancer, set up an HTTP monitor in Vigilmon:

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your service's external endpoint: https://staging-api.yourdomain.com/health
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: "status":"ok"
    • Response time threshold: 3000ms
  6. Save the monitor

Vigilmon probes from multiple independent geographic regions and requires consensus across locations before opening an incident. A transient network blip in one region won't wake your on-call — but a real service failure will.

Monitoring Multiple Skaffold Profiles

If you use Skaffold profiles for staging and production, create a separate monitor for each:

| Monitor Name | URL | Profile | |---|---|---| | [dev] my-api /health | https://dev-api.yourdomain.com/health | dev | | [staging] my-api /health | https://staging-api.yourdomain.com/health | staging | | [prod] my-api /health | https://api.yourdomain.com/health | production |

Grouping them in Vigilmon lets you see at a glance which environment is affected during an incident.

Step 3: Heartbeat Monitoring for Skaffold CI Pipelines

When running skaffold run in CI, a failed pipeline is a deployment gap. If your CI job exits non-zero, Skaffold stopped deploying — but nothing external knows that. Heartbeat monitoring flips the model: your pipeline reports in on success, and Vigilmon alerts if it goes silent.

Create the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
  2. Name it: skaffold-ci-deploy
  3. Set the expected interval: 1 day (or your actual deploy cadence)
  4. Set the grace period: 2 hours
  5. Save — you'll receive a unique URL like https://vigilmon.online/heartbeat/abc123xyz

Store the URL as a CI secret named VIGILMON_HEARTBEAT_URL.

Wire Into Your CI Pipeline

# .github/workflows/deploy.yml
name: Deploy via Skaffold

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up kubectl
        uses: azure/setup-kubectl@v3

      - name: Configure kubeconfig
        run: echo "${{ secrets.KUBECONFIG }}" | base64 -d > $HOME/.kube/config

      - name: Install Skaffold
        run: |
          curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64
          chmod +x skaffold && sudo mv skaffold /usr/local/bin/

      - name: Deploy with Skaffold
        run: skaffold run --profile=production

      - name: Notify Vigilmon heartbeat
        if: success()
        run: curl -fsS "${{ secrets.VIGILMON_HEARTBEAT_URL }}" > /dev/null

The heartbeat ping only fires on success(), so any failed Skaffold step — build error, push failure, deployment timeout — prevents the ping and triggers a Vigilmon alert after the grace period expires.

Step 4: Container Health Monitoring

Kubernetes readiness probes track pod health inside the cluster network. External monitoring tracks what external clients see. Both are necessary and check different things.

Add a Vigilmon SSL/TLS monitor alongside your HTTP monitor to catch certificate issues that Skaffold's --tls-verify flag doesn't cover post-deployment:

  1. In Vigilmon, go to Monitors → New Monitor
  2. Choose SSL Certificate
  3. Enter your domain: staging-api.yourdomain.com
  4. Set the expiry alert threshold: 30 days
  5. Save

A certificate that was valid at deploy time can expire during a sprint. Skaffold won't tell you — Vigilmon will.

Step 5: Alert Configuration

Slack Integration

  1. Create a Slack incoming webhook for your #deployments channel
  2. In Vigilmon, go to Alert Channels → New Channel → Webhook
  3. Paste the Slack webhook URL
  4. Assign the alert channel to all your Skaffold service monitors

A Vigilmon downtime alert sends a payload you can act on immediately:

{
  "monitor_name": "[staging] my-api /health",
  "status": "down",
  "url": "https://staging-api.yourdomain.com/health",
  "started_at": "2026-07-01T09:12:00Z",
  "duration_seconds": 0,
  "regions_failing": ["us-east", "eu-west"]
}

When this fires after a Skaffold deploy, you know immediately that the new image didn't come up cleanly.

Correlating Alerts With Builds

Add Skaffold's image tag to a custom response header in your health endpoint:

@app.get("/health")
async def health(response: Response):
    response.headers["X-Build-Version"] = os.getenv("APP_VERSION", "unknown")
    return {"status": "ok"}

Vigilmon's response body check captures this indirectly — if the version header changes after a Skaffold deploy and the service goes down, you have an exact build to roll back to.

Summary

Skaffold automates deployment; Vigilmon monitors what was deployed. Together they give you confidence in the full cycle — from code change to running service to confirmed availability.

| Concern | Tool | Coverage | |---|---|---| | Build success | Skaffold | Image build and push | | Deployment success | Skaffold | Pod rollout status | | External reachability | Vigilmon HTTP monitor | Full external path | | TLS certificate health | Vigilmon SSL monitor | Certificate expiry | | CI pipeline heartbeat | Vigilmon heartbeat | Deploy cadence | | Latency regression | Vigilmon response time threshold | External latency |

Start monitoring your Skaffold deployments for free at vigilmon.online — setup takes under five minutes and gives you immediate visibility into every deploy.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles