tutorial

Monitoring Kratix with Vigilmon: Promise API Availability, Worker Cluster Pipeline Health, GitOps State Sync & SSL Certificate Alerts

How to monitor Kratix with Vigilmon — Promise API availability, worker cluster pipeline health, GitOps state repository sync status, and SSL certificate alerts so your internal developer platform built on Kratix stays reliable and observable.

Kratix is the platform engineering framework that lets platform teams define and deliver infrastructure capabilities as Kubernetes Promises — self-service APIs that tenant developers request and Kratix fulfills by running pipelines and delivering workloads to worker clusters. When the Kratix platform cluster API becomes unavailable, developers cannot request new Promise fulfillments. When the GitOps state repository that Kratix uses to declare desired state on worker clusters becomes stale, running workloads drift from the configuration developers requested. Vigilmon gives you external visibility into the Kratix platform cluster, GitOps state repository synchronization, worker cluster availability, and the TLS certificates protecting every endpoint so you know immediately when your internal developer platform is degraded.

What You'll Build

  • HTTP monitors on the Kratix platform cluster API to detect Promise API failures
  • A monitor for the GitOps state repository (Gitea, GitHub) synchronizing state to worker clusters
  • Worker cluster API monitors to detect failures before developers notice missing resources
  • SSL certificate monitoring for the platform cluster and GitOps endpoints
  • An alert runbook mapping Vigilmon findings to Kratix and FluxCD remediation commands

Prerequisites

  • Kratix v0.18+ installed on a platform cluster with at least one worker cluster registered
  • At least one Promise and one fulfilled ResourceRequest deployed
  • A GitOps state store (Gitea, GitHub, or Flux bucket) configured
  • FluxCD installed on worker clusters for GitOps reconciliation
  • Network access to your platform cluster and GitOps endpoints from an external monitor
  • A free account at vigilmon.online

Step 1: Understand Kratix's Observability Surface

Kratix operates across two layers — the platform cluster where Promises are defined and requested, and worker clusters where Promise pipelines deliver actual resources:

# Check Kratix platform components
kubectl get pods -n kratix-platform-system
kubectl logs -n kratix-platform-system deployment/kratix-platform-controller-manager --tail=30

# List all defined Promises
kubectl get promises -A
kubectl describe promise postgresql

# List ResourceRequests (developer fulfillment requests)
kubectl get resourcerequests -A
kubectl describe resourcerequest my-postgres-request -n default

# Check worker cluster registrations
kubectl get destinations -A
kubectl describe destination worker-cluster-1

# Check the GitOps state store (Kratix writes desired state here)
kubectl get gitstatestores -A
kubectl describe gitstatestore default

# Check FluxCD on a worker cluster for GitOps sync status
kubectl get kustomizations -n flux-system --context=worker-cluster-1

The key observability signals are:

  • Platform cluster API: All Promise definitions and ResourceRequest submissions go through the platform cluster API. Unavailability blocks all developer self-service requests.
  • GitOps state repository: Kratix writes the desired state for each worker cluster to a Git repository (or Flux bucket). If writes fail, worker clusters stop receiving new configuration.
  • Worker cluster availability: Worker clusters reconcile GitOps state via FluxCD. If a worker cluster is unreachable, resource fulfillments targeting it fail silently.
  • FluxCD on worker clusters: FluxCD pulls from the GitOps state repository and applies manifests to the worker cluster. FluxCD failure means new Promise fulfillments are queued but never applied.
  • TLS certificates: The platform API and GitOps endpoints use TLS; certificate expiry breaks connectivity for both developer clients and worker cluster FluxCD controllers.

Step 2: Monitor the Kratix Platform Cluster API

The platform cluster is where developers submit ResourceRequest objects to request infrastructure. Monitor its health endpoint:

# Check platform cluster API health
kubectl get --raw /healthz
# Expected: "ok"

# Find the platform cluster API endpoint
kubectl cluster-info
# Output: Kubernetes control plane is running at https://platform.kratix.example.com:6443

# Test the platform API health endpoint
curl -k https://platform.kratix.example.com:6443/healthz
# Expected: "ok"

# Verify Promise CRDs are registered
kubectl api-resources | grep promise
# Output: promises, resourcerequests

# Test that Kratix webhook is responding
kubectl auth can-i create resourcerequest --as=developer@example.com

In Vigilmon:

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://platform.kratix.example.com:6443/healthz.
  3. Check interval: 2 minutes.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: ok.
  7. Label: kratix platform api.
  8. Click Save.

Add a TCP monitor for a connectivity check independent of HTTP and TLS:

  1. Add Monitor → TCP.
  2. Host: platform.kratix.example.com.
  3. Port: 6443.
  4. Check interval: 1 minute.
  5. Label: kratix platform tcp.
  6. Click Save.

Step 3: Monitor the GitOps State Repository

Kratix writes desired state to a Git repository that FluxCD on worker clusters polls and applies. If the state repository is unreachable, new Promise fulfillments are written to the platform but never delivered to workers:

# Check the configured GitStateStore
kubectl get gitstatestores -A -o jsonpath='{range .items[*]}{.metadata.name}: {.spec.url}{"\n"}{end}'

# For Gitea (common self-hosted option with Kratix)
curl https://gitea.example.com/api/v1/repos/kratix/state-store
# Expected: 200 with repository metadata

# Check Gitea health
curl https://gitea.example.com/-/health
# Expected: "OK"

# For GitHub-backed state stores, check the API
curl -H "Authorization: token $GITHUB_TOKEN" \
  https://api.github.com/repos/your-org/kratix-state-store

# Verify Kratix can write to the state store
kubectl logs -n kratix-platform-system deployment/kratix-platform-controller-manager | \
  grep -i "statestore\|git" | tail -20

In Vigilmon (for Gitea):

  1. Add Monitor → HTTP.
  2. URL: https://gitea.example.com/-/health.
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: OK.
  6. Label: kratix gitea state store.
  7. Click Save.

Also add a TCP monitor:

  1. Add Monitor → TCP.
  2. Host: gitea.example.com.
  3. Port: 443.
  4. Check interval: 2 minutes.
  5. Label: kratix state store tcp.
  6. Click Save.

Step 4: Monitor Worker Cluster API Availability

Kratix delivers Promise fulfillments to worker clusters. Each worker cluster must be available for its FluxCD controller to apply GitOps state:

# List registered worker clusters (Destinations)
kubectl get destinations -A
kubectl get destination worker-cluster-1 -o jsonpath='{.status.conditions}'

# Check worker cluster API health
curl -k https://worker1.example.com:6443/healthz
# Expected: "ok"

# Check FluxCD on the worker cluster
kubectl get kustomizations -n flux-system --context=worker-cluster-1
kubectl get helmreleases -n flux-system --context=worker-cluster-1

# Verify Promise resources are delivered to the worker
kubectl get pods -A --context=worker-cluster-1 | grep my-postgres

In Vigilmon for each worker cluster:

  1. Add Monitor → HTTP.
  2. URL: https://worker1.example.com:6443/healthz.
  3. Check interval: 5 minutes.
  4. Expected status: 200.
  5. Keyword: ok.
  6. Label: kratix worker-1 api.
  7. Click Save.

Add TCP monitors for each worker:

  1. Add Monitor → TCP.
  2. Host: worker1.example.com.
  3. Port: 6443.
  4. Check interval: 2 minutes.
  5. Label: kratix worker-1 tcp.
  6. Click Save.

Step 5: Monitor FluxCD on Worker Clusters

FluxCD is the GitOps reconciler on worker clusters that pulls from the Kratix state repository and applies manifests. If FluxCD is degraded, Promise fulfillments queue in Git but never reach the cluster:

# Check FluxCD pod health on a worker cluster
kubectl get pods -n flux-system --context=worker-cluster-1
kubectl logs -n flux-system deployment/source-controller --context=worker-cluster-1 --tail=30

# Check FluxCD GitRepository source for the Kratix state store
kubectl get gitrepositories -n flux-system --context=worker-cluster-1
# Look for: "Ready=True" and recent "Last Fetched At"

# Check Kustomization apply status
kubectl get kustomizations -n flux-system --context=worker-cluster-1
# Look for: "Ready=True" and recent "Last Applied Revision"

# Port-forward FluxCD webhook receiver health
kubectl port-forward -n flux-system svc/webhook-receiver 9292:9292 --context=worker-cluster-1
curl http://localhost:9292/healthz
# Expected: {"status":"ok"}

Expose FluxCD health on each worker cluster:

# worker-flux-health-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: flux-health
  namespace: flux-system
spec:
  rules:
  - host: worker1-flux-health.example.com
    http:
      paths:
      - path: /healthz
        pathType: Exact
        backend:
          service:
            name: webhook-receiver
            port:
              number: 9292
  tls:
  - hosts:
    - worker1-flux-health.example.com
    secretName: flux-health-tls

In Vigilmon:

  1. Add Monitor → HTTP.
  2. URL: https://worker1-flux-health.example.com/healthz.
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: ok.
  6. Label: kratix worker-1 flux health.
  7. Click Save.

Step 6: Monitor SSL Certificates

Kratix relies on TLS for every connection: platform cluster API, GitOps state repository, and worker cluster APIs. An expired certificate breaks the GitOps reconciliation loop silently:

# Check platform cluster certificate
openssl s_client -connect platform.kratix.example.com:6443 2>/dev/null | \
  openssl x509 -noout -dates

# Check Gitea state store certificate
openssl s_client -connect gitea.example.com:443 2>/dev/null | \
  openssl x509 -noout -dates

# Check worker cluster certificates
openssl s_client -connect worker1.example.com:6443 2>/dev/null | \
  openssl x509 -noout -dates

# List cert-manager certificates across the platform cluster
kubectl get certificates -A

# Check FluxCD GitRepository TLS verification status
kubectl get gitrepository -n flux-system --context=worker-cluster-1 \
  -o jsonpath='{.items[0].status.conditions}'

In Vigilmon:

  1. Add Monitor → SSL Certificate.
  2. Domain: platform.kratix.example.com.
  3. Port: 6443.
  4. Alert when expiry is within: 30 days.
  5. Alert again: 14 days, 7 days, 3 days, 1 day.
  6. Click Save.

Repeat for gitea.example.com and each workerN.example.com.

Step 7: Configure Alerting and Runbook

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Immediate Action | |---|---|---| | Kratix platform API HTTP | Non-200 | kubectl get pods -n kratix-platform-system; check controller logs; developer ResourceRequests will fail | | Kratix platform API TCP | Connection refused | Check platform cluster node health; escalate to platform SRE | | GitOps state store HTTP | Non-200 | Check Gitea/GitHub status; Kratix cannot write state — new fulfillments queued but not delivered | | Worker cluster API | Non-200 | kubectl get nodes --context=worker1; check cloud provider; affected Promise deliveries stalled | | FluxCD worker health | Non-200 | kubectl get pods -n flux-system --context=worker1; check source-controller logs for git pull errors | | SSL certificate | < 30 days | kubectl get certificates; trigger cert-manager renewal; expired cert breaks GitOps loop |

Escalation policy: Platform cluster failures are P1 — developers cannot request new infrastructure. State store failures are also P1 — new fulfillments are written to the platform but silently never delivered. Worker cluster failures are P2 per worker — only developers targeting that specific worker are affected.

Common Kratix Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Kratix controller OOM killed during parallel Promise pipeline execution | Platform API health stays green; controller pod restarts; pipeline jobs stall mid-execution | | Gitea state store disk full — Kratix writes fail silently | State store TCP check succeeds; HTTP check returns 500; Vigilmon alerts; FluxCD sees no new commits | | Worker cluster cert-manager fails to renew API certificate | Worker SSL monitor catches 30 days before FluxCD starts failing TLS verification of worker API | | FluxCD on worker cluster OOM killed during large manifest apply | FluxCD health goes down; Kustomizations show stale last-applied; Promise deliveries stop | | Platform cluster control plane upgrade causes brief API unavailability | Platform API TCP monitor catches connection refused; developer requests return 503 during window | | Kratix Promise pipeline image pull fails — all pipelines stall | Platform API health green; pipeline pods in ImagePullBackOff; surfaces via Kratix controller logs | | Worker cluster node pool scale-down evicts running FluxCD pod | FluxCD health goes down on that worker; GitOps reconciliation stalls; detected before tenant reports | | State store repository grows too large — FluxCD clone times out | State store HTTP check succeeds; FluxCD Kustomization shows Source not ready; add branch-based state partitioning |

Kratix enables platform teams to build internal developer platforms as Promise-based APIs, but its distributed architecture — platform cluster, GitOps state repository, and multiple worker clusters — creates failure modes across every layer. A state repository outage silently queues fulfillments that are never delivered. A worker FluxCD failure lets configuration drift for all tenants on that worker. Vigilmon provides external monitoring across the entire delivery chain: the platform API, the GitOps state repository, each worker cluster, and the TLS certificates that protect every link. When any component degrades, you know within minutes with enough context to distinguish a platform-wide incident from a single worker failure.

Start monitoring Kratix in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles