Tanka is a Kubernetes configuration management tool from Grafana Labs that uses Jsonnet to describe cluster resources. Unlike raw YAML, Jsonnet lets you compose, inherit, and programmatically generate Kubernetes manifests — making it practical to manage large, multi-environment deployments with shared library components. Tanka handles the configuration and reconciliation of Kubernetes resources. What it does not do is tell you whether the pods Kubernetes scheduled are actually serving live traffic correctly. Vigilmon answers that question. It probes your service endpoints from multiple external locations on a continuous schedule, alerting your team immediately if availability degrades — independent of whether your cluster's internal health checks and readiness probes report all clear.
This tutorial covers adding external uptime monitoring to services deployed and managed through Tanka environments.
What You'll Build
- A
/healthendpoint in a service deployed via a Tanka environment - A Vigilmon HTTP monitor with JSON body assertions
- A heartbeat monitor for scheduled Tanka sync jobs
- Alert routing that keeps Kubernetes cluster alerts and external uptime alerts distinct
Prerequisites
- A Tanka project with at least one environment targeting a running cluster
- A public HTTPS endpoint for a service managed by Tanka
- A free account at vigilmon.online
Step 1: Add a Health Endpoint to Your Service
Tanka manages the Kubernetes manifests that deploy your service. The service itself needs a /health endpoint that Vigilmon can probe. Build it to exercise real runtime dependencies — not a static 200 that passes even when downstream dependencies have failed.
Go (standard library)
// internal/health/handler.go
package health
import (
"database/sql"
"encoding/json"
"net/http"
"os"
)
type response struct {
Status string `json:"status"`
Checks map[string]string `json:"checks"`
Host string `json:"host"`
}
func Handler(db *sql.DB) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
checks := map[string]string{}
ok := true
if err := db.PingContext(r.Context()); err != nil {
checks["database"] = "error: " + err.Error()
ok = false
} else {
checks["database"] = "ok"
}
host, _ := os.Hostname()
resp := response{
Checks: checks,
Host: host,
}
if ok {
resp.Status = "ok"
} else {
resp.Status = "degraded"
w.WriteHeader(http.StatusServiceUnavailable)
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
}
Tanka manifest for the Service and Ingress
Define the health check path in your Tanka Jsonnet so every environment automatically exposes /health:
// lib/service.libsonnet
local k = import 'k.libsonnet';
{
deployment(name, image, port): k.apps.v1.deployment.new(
name=name,
replicas=2,
containers=[
k.core.v1.container.new(name, image)
+ k.core.v1.container.withPorts([k.core.v1.containerPort.new('http', port)])
+ k.core.v1.container.livenessProbe.httpGet.withPath('/health')
+ k.core.v1.container.livenessProbe.httpGet.withPort('http')
+ k.core.v1.container.readinessProbe.httpGet.withPath('/health')
+ k.core.v1.container.readinessProbe.httpGet.withPort('http'),
],
),
}
Verify the endpoint after a Tanka apply:
tk apply environments/production/
kubectl get pods -n production
curl -s https://api.example.com/health | jq .
# {"status":"ok","checks":{"database":"ok"},"host":"api-deployment-abc123"}
Step 2: Configure Vigilmon HTTP Monitor
- Log in to Vigilmon → Add Monitor → HTTP.
- URL:
https://api.example.com/health - Check interval: 60 seconds.
- Response timeout: 10 seconds.
- Expected status code:
200. - JSON body assertion:
- Path:
status - Expected value:
ok
- Path:
- Click Save.
Vigilmon probes from multiple external locations and requires a quorum of probe agents to agree on a failure before generating an alert. This ensures that a single probe node losing routing to your cluster doesn't wake up your on-call engineer.
Monitor per Tanka environment
Each Tanka environment typically maps to a deployment stage. Create a dedicated Vigilmon monitor for each production environment endpoint:
| Tanka environment | Service URL | Interval |
|---|---|---|
| environments/production | https://api.example.com/health | 60 s |
| environments/staging | https://api-staging.example.com/health | 120 s |
| environments/canary | https://api-canary.example.com/health | 60 s |
Group monitors under a Monitor group named after the Tanka environment or cluster.
Step 3: Heartbeat Monitor for Tanka Sync Jobs
Teams often automate Tanka applies on a schedule — nightly config drift checks, automated rollouts, or GitOps reconciliation loops. If these scheduled sync jobs fail silently, configuration drift accumulates without anyone noticing. Vigilmon heartbeat monitors detect when a scheduled job stops running.
#!/bin/bash
# scripts/tanka-sync.sh
set -euo pipefail
ENVIRONMENT="${1:-environments/production}"
echo "Running Tanka diff check for ${ENVIRONMENT}..."
tk diff "${ENVIRONMENT}"
echo "Applying Tanka environment: ${ENVIRONMENT}..."
tk apply --dangerous-auto-approve "${ENVIRONMENT}"
echo "Sync complete — sending heartbeat"
curl -s -X POST "${VIGILMON_HEARTBEAT_URL}" --max-time 5
In Vigilmon, create a Heartbeat monitor with a grace period 30 minutes longer than your scheduled sync interval.
GitHub Actions integration
# .github/workflows/tanka-nightly-sync.yml
name: Nightly Tanka Sync
on:
schedule:
- cron: '0 3 * * *' # 3 AM UTC daily
jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Tanka
run: |
curl -LO https://github.com/grafana/tanka/releases/latest/download/tk-linux-amd64
chmod +x tk-linux-amd64
sudo mv tk-linux-amd64 /usr/local/bin/tk
- name: Configure kubeconfig
run: |
echo "${{ secrets.KUBECONFIG_B64 }}" | base64 -d > "$HOME/.kube/config"
- name: Run Tanka sync
env:
VIGILMON_HEARTBEAT_URL: ${{ secrets.VIGILMON_HEARTBEAT_URL }}
run: bash scripts/tanka-sync.sh environments/production
- name: Send heartbeat on success
if: success()
run: curl -X POST "${{ secrets.VIGILMON_HEARTBEAT_URL }}"
Store the heartbeat URL as a repository secret so it's available across workflow runs.
Step 4: TLS Certificate Monitoring
Services deployed by Tanka often use cert-manager to provision Let's Encrypt certificates. cert-manager renews certificates automatically, but the renewal mechanism itself can fail silently — especially if DNS propagation stalls or ACME challenges time out. Vigilmon's TLS monitor provides an external confirmation that the certificate is valid.
In Vigilmon → Add Monitor → SSL/TLS certificate:
- Host:
api.example.com - Port:
443 - Alert before expiry: 30 days
If cert-manager is configured to renew 60 days before expiry, the 30-day Vigilmon alert fires only if the renewal has failed — catching the failure with time to intervene.
Step 5: Alert Routing Strategy
Tanka-managed Kubernetes deployments typically feed alerts into cluster observability stacks: Prometheus Alertmanager, Grafana, or Loki. External uptime failures from Vigilmon are a separate, independent signal that should route through a distinct path so cluster-internal alerts don't mask service availability failures and vice versa.
| Failure mode | Source | Route to |
|---|---|---|
| Tanka apply failed | CI/CD pipeline | Slack #k8s-deploys + on-call |
| Pod crash loop or OOMKill | Kubernetes / Prometheus | PagerDuty + Slack #k8s-alerts |
| External endpoint down | Vigilmon | PagerDuty + Slack #prod-alerts |
| Nightly sync job missed | Vigilmon | Slack #infra-ops-critical |
| TLS certificate expiring | Vigilmon | DevOps team + cert-manager audit |
Configure Vigilmon alert channels under Alerts → Add channel:
- Email: on-call distribution list.
- PagerDuty webhook: primary on-call rotation for production availability failures.
- Slack webhook: a
#vigilmon-alertschannel for the SRE team.
Step 6: Public Status Page
Kubernetes cluster health dashboards are internal. When a production service experiences an outage, your users and API consumers have no visibility unless you publish it proactively. Vigilmon's public status page gives them a real-time availability view tied directly to the external probe results.
- In Vigilmon → Status Pages → Create.
- Add your production endpoint monitors and heartbeat monitors.
- Configure a custom domain (
status.example.com) or use the Vigilmon subdomain. - Embed a status badge in your API documentation or dashboard:
<img src="https://vigilmon.online/badge/<monitor-id>.svg" alt="API Status" />
What Vigilmon Adds to a Tanka Workflow
| Capability | Tanka | Vigilmon | |---|---|---| | Jsonnet-based manifest generation | Yes | No | | Multi-environment config inheritance | Yes | No | | Kubernetes resource reconciliation | Yes | No | | GitOps apply workflow | Yes | No | | External synthetic HTTP probing | No | Yes | | Multi-location quorum-based checks | No | Yes | | DNS resolution failure detection | No | Yes | | TLS/SSL certificate expiry alerts | No | Yes | | Scheduled job heartbeat monitoring | No | Yes | | Public status page | No | Yes | | Independent of cluster health | No | Yes |
Tanka gives you composable, programmable Kubernetes configuration management. Vigilmon gives you the external availability signal that tells you what users experience after Tanka has applied its manifests and Kubernetes has scheduled its pods. Together they close the gap between infrastructure state and user-facing service reliability.
Add external uptime monitoring to your Tanka-managed services today — register free at vigilmon.online.