tutorial

How to Monitor Ko Build Pipelines with Vigilmon

Ko is Google's open-source tool for building and publishing Go container images — no Dockerfile required. It reads your Go module, compiles a static binary, ...

Ko is Google's open-source tool for building and publishing Go container images — no Dockerfile required. It reads your Go module, compiles a static binary, layers it into a minimal base image, and pushes it to a container registry in seconds. If you ship Go services to Kubernetes, Ko likely touches every deployment you make.

But a fast build tool is only valuable when it's working. Registry push failures, stale image digests, broken CI integrations, and artifact corruption are all real failure modes — and they're silent until a deployment fails in production. This tutorial shows you how to monitor your Ko Build pipeline with Vigilmon so you catch failures before your users do.

Why Ko Build pipelines need external monitoring

Ko is typically embedded in CI/CD workflows: a GitHub Actions job or a GitLab pipeline runs ko build on every merge, pushes the image to a registry, and a Kubernetes deploy rolls it out. Each step can fail independently:

  • Registry unavailability — the OCI registry (GCR, ECR, Docker Hub, GHCR) goes down or rate-limits your pushes; Ko exits 0 after building but the image was never pushed
  • Binary artifact corruption — a compiler toolchain update or caching bug produces a broken binary; the image exists in the registry but the container crashes on start
  • CI integration drift — a change to your CI runner's environment (Go version, module proxy, GONOSUMCHECK) silently breaks Ko's build graph
  • Tag immutability violations — the registry rejects a push because the tag already exists and immutability is enforced; Ko logs an error but your pipeline script ignores stderr
  • Health endpoint missing on published images — the Go service inside the container has no /healthz endpoint, so Kubernetes can't tell liveness from crash-loop

External monitoring closes the loop: after Ko pushes an image and your deployment rolls out, Vigilmon continuously verifies that the running service actually responds. A silent push failure or corrupt binary gets caught within minutes, not at the next on-call escalation.

What you'll need

  • A Go project using Ko for container builds
  • A container registry (GCR, ECR, GHCR, Docker Hub, or self-hosted)
  • A Kubernetes cluster or any container runtime to deploy to
  • A free Vigilmon account — no credit card needed

Step 1: Add a health endpoint to your Go service

Ko publishes whatever your main package builds. Before monitoring the deployment, make sure your service exposes a /healthz endpoint that Vigilmon can probe:

package main

import (
	"encoding/json"
	"net/http"
	"os"
	"runtime"
	"time"
)

var startTime = time.Now()

func healthHandler(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(map[string]interface{}{
		"status":   "ok",
		"uptime":   time.Since(startTime).String(),
		"goVersion": runtime.Version(),
		"hostname": func() string { h, _ := os.Hostname(); return h }(),
	})
}

func main() {
	http.HandleFunc("/healthz", healthHandler)
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("hello from Ko-built service"))
	})
	http.ListenAndServe(":8080", nil)
}

The /healthz endpoint returns a JSON payload with status, uptime, and runtime info. Vigilmon will look for HTTP 200 from this endpoint every minute from multiple geographic regions.

Step 2: Build and publish with Ko

Install Ko and configure it to push to your registry:

# Install Ko
go install github.com/google/ko@latest

# Set the registry prefix (GCR example)
export KO_DOCKER_REPO=gcr.io/your-project/your-service

# Build and push — Ko resolves the main package automatically
ko build ./cmd/server --platform=linux/amd64 --push

# For a Kubernetes YAML with image references replaced automatically
ko apply -f config/deployment.yaml

Ko outputs the full image digest on stdout:

gcr.io/your-project/your-service:abc123def456@sha256:...

Capture this digest in CI so you can verify the image is reachable after push:

IMAGE=$(ko build ./cmd/server --push --format=json | jq -r '.[]')
echo "Published: $IMAGE"

Step 3: Deploy and expose a probe URL

After ko apply, your Kubernetes deployment uses the freshly-published image. Expose your service with a LoadBalancer or Ingress so Vigilmon can reach it from outside the cluster:

# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: go-service
  namespace: production
spec:
  selector:
    app: go-service
  ports:
    - port: 80
      targetPort: 8080
  type: LoadBalancer

# deployment.yaml — let ko apply substitute the image
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-service
  namespace: production
spec:
  replicas: 2
  selector:
    matchLabels:
      app: go-service
  template:
    metadata:
      labels:
        app: go-service
    spec:
      containers:
        - name: go-service
          image: gcr.io/your-project/your-service  # ko apply rewrites this
          ports:
            - containerPort: 8080
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 15
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 10

After the LoadBalancer provisions, get the external IP:

kubectl get service go-service -n production
# NAME         TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)
# go-service   LoadBalancer   10.96.0.50    34.90.10.22    80:32000/TCP

Your probe URL is http://34.90.10.22/healthz.

Step 4: Set up Vigilmon monitoring

Log in to Vigilmon and add your first monitor:

  1. Click New Monitor → HTTP(S)
  2. Set the URL to http://34.90.10.22/healthz (or your domain if you have Ingress with TLS)
  3. Set the interval to 1 minute
  4. Under Response validation, require:
    • Status code: 200
    • Response body contains: "status":"ok"
  5. Set locations to at least 3 geographic regions for multi-region coverage
  6. Click Save

Vigilmon immediately starts probing from each configured region. If any region cannot reach your service, an incident is opened and alerts fire.

Step 5: Monitor registry push success in CI

Ko pipeline failures often happen silently in CI. Add a lightweight check job that verifies the published image is pullable after every Ko build:

# .github/workflows/build.yml
name: Build and Monitor

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: '1.22'

      - name: Install Ko
        run: go install github.com/google/ko@latest

      - name: Build and push
        env:
          KO_DOCKER_REPO: gcr.io/${{ secrets.GCP_PROJECT }}/go-service
        run: |
          IMAGE=$(ko build ./cmd/server --push --format=json | jq -r '.[]')
          echo "IMAGE=$IMAGE" >> $GITHUB_ENV
          echo "Published: $IMAGE"

      - name: Verify image is pullable
        run: |
          docker pull "$IMAGE" --quiet
          echo "Registry verification passed"

      - name: Ping Vigilmon deploy webhook
        run: |
          curl -s -X POST "${{ secrets.VIGILMON_WEBHOOK_URL }}" \
            -H "Content-Type: application/json" \
            -d '{"event":"deploy","image":"'"$IMAGE"'","ref":"'"$GITHUB_SHA"'"}'

The deploy webhook tells Vigilmon a new version was just pushed. Vigilmon correlates any degradation in the minutes after a deploy with the triggering commit.

Step 6: Configure alerts

In Vigilmon, open your monitor and go to Alerts:

  • Email — immediate notification when the check fails from any region
  • Slack — post to your #incidents channel with a link to the incident timeline
  • PagerDuty / OpsGenie — escalate to on-call after 2 consecutive failures (avoids noise from transient blips)

Set the confirmation threshold to 2 consecutive failures before opening an incident. This eliminates false positives from brief network hiccups while still catching real Ko pipeline or deployment failures within 2 minutes.

What you're now monitoring

With this setup, Vigilmon detects:

| Failure scenario | How Vigilmon catches it | |---|---| | Ko push failed, old image still running | Health endpoint returns stale /healthz JSON; manual correlation with CI log | | Corrupt binary in published image | Container crash-loops; /healthz times out or returns 5xx | | Registry down during deployment | Rolling update stalls; new pods never pass readiness; service degrades | | Kubernetes node failure removes all replicas | Service becomes unreachable from all Vigilmon regions | | CI environment change breaks Ko build | Deploy webhook stops firing; combined with uptime check, gap is visible |

Conclusion

Ko makes building and publishing Go container images remarkably simple — but simplicity doesn't mean failure-proof. Registry outages, corrupt artifacts, and CI drift all happen, and they're invisible until a deployment fails. Adding Vigilmon monitoring takes about five minutes: expose /healthz in your Go service, deploy with ko apply, and point Vigilmon at the external URL. From there, every Ko-driven deployment is continuously verified from multiple global regions, and any failure generates an alert before your users notice.

Sign up for a free Vigilmon account to get started.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles