tutorial

Monitoring Kargo with Vigilmon: API Health, Web UI Availability, Webhook Endpoint & SSL Certificate Alerts

How to monitor Kargo with Vigilmon — API health checks, web UI availability, webhook endpoint monitoring, and SSL certificate alerts for GitOps-based progressive delivery.

Kargo is the GitOps-based progressive delivery and promotion platform that automates multi-stage rollouts across Kubernetes environments — moving artifacts from dev to staging to production through promotion policies, automated verification steps, and manual approval gates. When Kargo's API server becomes unavailable, promotions stall mid-flight and stage configurations drift silently from their intended states. When the webhook endpoint is unreachable, Git repository push events that should trigger automatic promotions go unprocessed, leaving artifacts stranded in earlier stages. Vigilmon gives you external visibility into Kargo's operational health: the API server, web UI, webhook receiver, and SSL certificate expiry.

What You'll Build

  • A monitor on Kargo's API health endpoint to detect API server failures
  • A web UI availability check confirming the Kargo dashboard is accessible
  • A webhook endpoint check to ensure Git push events reach Kargo's promotion engine
  • An SSL certificate monitor for your Kargo domain
  • An alerting setup that distinguishes API layer failures from UI and webhook delivery failures

Prerequisites

  • Kargo installed in a Kubernetes cluster (version 0.6.0 or later)
  • Kargo API server and web UI exposed via an Ingress or LoadBalancer with a DNS name
  • A free account at vigilmon.online

Step 1: Understand Kargo's Health Endpoints

Kargo's API server exposes a liveness endpoint that confirms the gRPC-gateway and Kubernetes API connectivity are healthy:

curl https://kargo.example.com/healthz
# Returns: {"status":"SERVING"}

For an unauthenticated check:

curl -k https://kargo.example.com/healthz
# Returns HTTP 200 with a JSON status body

A healthy Kargo API server returns 200 with "status":"SERVING". If the API server pod is crashed, the Kubernetes API is unreachable, or the gateway container is unresponsive, you receive a connection error or a 502 from the ingress controller in front of Kargo.

Step 2: Create a Vigilmon HTTP Monitor for the API Health Endpoint

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://kargo.example.com/healthz.
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: SERVING (present in the response body when the API server is healthy).
  7. Click Save.

This monitor catches:

  • Kargo API server pod crashes or OOM kills
  • Kubernetes API server connectivity failures that prevent Kargo from reading stage configurations
  • Database (etcd-backed) access failures affecting stage status storage
  • Ingress controller failures that block external API access
  • Kargo upgrade failures that leave the API server in a degraded state

Alert sensitivity: Set to trigger after 1 consecutive failure. When the Kargo API is down, promotions cannot be approved, stage status cannot be queried, and the kargo CLI becomes non-functional for every operator in the organization.

Step 3: Monitor the Kargo Web UI

Kargo's web UI provides the dashboard where platform engineers view promotion pipelines, inspect stage health, approve manual promotion gates, and review freight history. A UI outage is a distinct failure mode from an API outage — the React frontend may fail to load while the backend gRPC server is still healthy:

curl https://kargo.example.com
# Returns HTML containing "Kargo" in the page content
  1. Add Monitor → HTTP.
  2. URL: https://kargo.example.com.
  3. Check interval: 2 minutes.
  4. Response timeout: 20 seconds.
  5. Expected status: 200.
  6. Keyword: Kargo (present in the web UI HTML).
  7. Label: Kargo web UI.
  8. Click Save.

When the web UI monitor fires but the /healthz monitor stays green, the issue is in the frontend static asset serving layer — typically a misconfigured Nginx container serving the React build, a broken ConfigMap mounting the UI configuration, or a resource constraint on the UI pod. Platform engineers lose dashboard access but CLI operations continue working.

Step 4: Monitor the Webhook Endpoint

Kargo integrates with Git repositories (GitHub, GitLab, Gitea) to receive push event webhooks. When a new image is pushed or a Git commit lands, the source repository sends a webhook to Kargo's webhook receiver endpoint, which triggers artifact discovery and optional automatic promotions. If this endpoint is unreachable, automatic promotions never start:

# Verify the webhook receiver endpoint is reachable
curl -I https://kargo.example.com/webhooks/github
# Returns HTTP 405 (expects POST) or 401/403 — confirms the endpoint exists
  1. Add Monitor → HTTP.
  2. URL: https://kargo.example.com/webhooks/github (adjust path for your Git provider: /webhooks/gitlab, /webhooks/gitea).
  3. Check interval: 5 minutes.
  4. Response timeout: 10 seconds.
  5. Expected status: 405 (Method Not Allowed confirms the endpoint is reachable and routing correctly).
  6. Label: Kargo webhook receiver.
  7. Click Save.

Why monitor this separately? The /healthz endpoint confirms the API server is running. The webhook endpoint sits behind a different routing path and can fail independently — for example, if a Kubernetes Service selector misconfiguration routes webhook traffic to the wrong pod, or if an ingress rule for the /webhooks path is accidentally deleted.

Step 5: Monitor the Kargo gRPC API Port via TCP

Kargo exposes its primary API over gRPC (used by the kargo CLI and the web UI backend). The gRPC port is typically the same as the HTTPS port (443) or a dedicated port depending on your ingress setup. TCP monitoring confirms the port accepts connections at the network level, catching ingress and load balancer failures that might not be reflected in HTTP-level checks:

# Test TCP connectivity to the gRPC port
nc -zv kargo.example.com 443
  1. Add Monitor → TCP.
  2. Host: kargo.example.com.
  3. Port: 443 (or your dedicated gRPC port).
  4. Check interval: 2 minutes.
  5. Label: Kargo gRPC TCP port.
  6. Click Save.

If the TCP monitor fires while the HTTP health check is green, you likely have an HTTP/1.1 vs. HTTP/2 routing mismatch — the ingress serves HTTP/1.1 for health checks but drops gRPC (HTTP/2) connections. This breaks the kargo CLI and the web UI's real-time stage updates while leaving the REST-accessible /healthz endpoint appearing healthy.

Step 6: Monitor SSL Certificates

Kargo uses TLS for all communication — between the CLI and the API server, between the web UI and the gRPC gateway, and between Git providers and the webhook receiver. An expired certificate breaks all three simultaneously and can be difficult to diagnose if you're not watching certificate expiry proactively:

openssl s_client -connect kargo.example.com:443 2>/dev/null | openssl x509 -noout -dates
  1. Add Monitor → SSL Certificate.
  2. Domain: kargo.example.com.
  3. Alert when expiry is within: 30 days.
  4. Alert again: 14 days, 7 days, 3 days, 1 day.
  5. Click Save.

Most Kargo deployments use cert-manager with Let's Encrypt for certificate issuance. Renewal failures are common after Kargo upgrades that change Ingress annotations, after namespace migrations, or when ACME HTTP-01 challenges are blocked by NetworkPolicies applied to the Kargo namespace. A 30-day window gives you time to catch a failed renewal before it becomes an outage.

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | /healthz | Non-200 or SERVING missing | Check Kargo API server pod; run kubectl get pods -n kargo | | Web UI | Non-200 or keyword missing | Frontend pod issue; check Kargo UI deployment and Nginx config | | Webhook receiver | Non-405 response | Webhook routing broken; automatic promotions will not trigger | | gRPC TCP port | Connection refused | gRPC port blocked; CLI and web UI real-time updates broken | | SSL certificate | < 30 days to expiry | Renew certificate; check cert-manager Certificate resource in kargo namespace |

Alert after: 1 consecutive failure for the API health and gRPC TCP monitors. 2 consecutive failures for UI and webhook monitors.

Common Kargo Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Kargo API server pod OOM killed | /healthz returns connection error; alert within 60 s | | Kubernetes API unavailable | Kargo cannot reconcile stages; API health monitor fires | | Web UI frontend crash | UI keyword monitor fires; API health stays green | | Webhook routing misconfiguration | Webhook monitor fires; Git push events silently lost | | gRPC/HTTP/2 ingress misconfiguration | gRPC TCP monitor fires; CLI and UI real-time updates broken | | SSL certificate expired | SSL monitor alerts; all connections fail including Git provider webhooks | | Kargo upgrade breaks API | /healthz returns non-SERVING after version update | | Ingress controller failure | All monitors fire simultaneously | | Promotion controller crash | Promotions stall; not caught by external monitoring — check Kargo controller logs | | Image repository polling failure | No new freight discovered; not externally detectable — requires internal metrics |

Kargo sits at the critical path of every multi-stage GitOps promotion — when it fails, artifact promotions stall between environments, leaving staging ahead of dev and production behind staging without anyone noticing until an engineer manually checks. Vigilmon gives you external visibility into every layer of Kargo's availability: the API server health, web dashboard, webhook receiver for automatic promotions, gRPC CLI port, and SSL certificate expiry, so you detect failures before they silently block your entire promotion pipeline.

Start monitoring Kargo in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles