Monitoring Timoni with Vigilmon: Module Registry Availability, Bundle Apply Health, Flux Controller Sync & SSL Certificate Alerts

Timoni is the Kubernetes package manager that uses CUE configuration language to define, distribute, and apply cluster packages as OCI artifacts. When the OCI registry hosting Timoni modules becomes unavailable, timoni apply commands fail and automated GitOps pipelines using Timoni bundles stall silently. When the Flux controller reconciling Timoni bundles loses connectivity to the registry or the Kubernetes API, drift accumulates undetected. Vigilmon gives you external visibility into the OCI module registry, your Flux GitOps controller, the Kubernetes API endpoints Timoni depends on, and the SSL certificates protecting the entire artifact distribution chain so you know immediately when your package management pipeline has stalled.

What You'll Build

• An HTTP monitor on your OCI module registry to detect artifact push and pull failures
• A monitor for the Flux GitOps controller managing Timoni bundles
• Kubernetes API server health monitoring via HTTP and TCP
• SSL certificate monitoring for your OCI registry and Flux webhook endpoints
• An alert runbook mapping Vigilmon findings to specific timoni, flux, and kubectl remediation commands

Prerequisites

• Timoni v0.22+ installed and configured with kubectl access to a cluster
• An OCI registry (GitHub Container Registry, Docker Hub, Zot, Harbor) hosting Timoni modules
• Flux v2.2+ installed in the cluster for GitOps bundle reconciliation (optional but common)
• A Kubernetes cluster with kubectl access
• A free account at vigilmon.online

Step 1: Understand Timoni's Observability Surface

Timoni itself is a CLI tool with no server component — its observability surface comes from the systems it depends on: the OCI registry, the Kubernetes API, and optionally a Flux controller for GitOps automation:

# Verify Timoni can reach the registry and pull a module
timoni mod pull oci://ghcr.io/stefanprodan/modules/podinfo:6.5.4 \
  --output ./podinfo-module

# Check that Timoni can apply a bundle to the cluster
timoni bundle apply -f ./timoni-bundle.cue --dry-run

# If using Flux, check Timoni bundle reconciliation status
kubectl get timoniinstances -A
kubectl get fluxcd -A

# Check the OCI registry API
curl https://ghcr.io/v2/
# Expected: 200 with WWW-Authenticate header (anonymous access returns 401, which confirms the registry is up)

# Check registry for a specific module tag
crane ls ghcr.io/your-org/modules/your-module

The key signals are:

• OCI registry availability: timoni apply and timoni mod pull fail immediately if the registry returns 5xx or refuses connections.
• Flux TimoniInstance reconciliation: If you use Flux to manage Timoni bundles, the Flux controller must be running and able to reach the registry.
• Kubernetes API health: Timoni applies resources directly via the Kubernetes API. API server degradation causes apply operations to hang or fail.
• TLS certificates on the registry: Timoni uses standard OCI client libraries that reject expired or self-signed certificates unless explicitly configured to skip verification.

Step 2: Monitor Your OCI Module Registry

The OCI registry is the central artifact store for all Timoni modules. Verify it exposes a standard /v2/ API endpoint:

# Test registry API endpoint (returns 200 or 401 when healthy)
curl -o /dev/null -s -w "%{http_code}" https://ghcr.io/v2/
# Expected: 401 (unauthenticated) — this means the registry API is up

# Test with authentication
echo $GITHUB_TOKEN | docker login ghcr.io -u $GITHUB_USER --password-stdin
curl -H "Authorization: Bearer $TOKEN" https://ghcr.io/v2/

# For self-hosted registries (Zot, Harbor, Distribution)
curl https://registry.example.com/v2/
# Expected: 200 or 401

In Vigilmon:

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: https://ghcr.io/v2/ (or your registry URL).
3. Check interval: 5 minutes.
4. Response timeout: 15 seconds.
5. Expected status: 401 (unauthenticated OCI registry API returns 401, not 403 or 5xx, when healthy).
6. Label: timoni oci registry.
7. Click Save.

Note on 401 vs 200: The OCI Distribution Spec mandates that /v2/ returns 401 Unauthorized with a WWW-Authenticate header for unauthenticated requests. A 401 response confirms the registry is running correctly. A 5xx or connection failure confirms it is not.

For self-hosted registries that return 200 on /v2/, adjust the expected status accordingly.

Step 3: Monitor a Self-Hosted OCI Registry (Zot or Harbor)

If you run a self-hosted OCI registry with Timoni modules, monitor its server health directly:

# Zot exposes a health endpoint
curl https://registry.example.com/v2/_catalog
# Lists all repositories — confirms authenticated access works

# Harbor exposes a ping endpoint
curl https://harbor.example.com/api/v2.0/ping
# Expected: "Pong"

# Check registry pod status
kubectl get pods -n registry
kubectl logs -n registry deployment/zot --tail=30

For Harbor or Zot:

1. Add Monitor → HTTP.
2. URL: https://harbor.example.com/api/v2.0/ping (Harbor) or https://registry.example.com/v2/ (Zot/Distribution).
3. Check interval: 5 minutes.
4. Expected status: 200 (Harbor ping) or 401 (Zot/Distribution v2 API).
5. Keyword: Pong (Harbor only).
6. Label: timoni self-hosted registry.
7. Click Save.

Add a TCP monitor on the registry port for an independent connectivity check:

1. Add Monitor → TCP.
2. Host: registry.example.com.
3. Port: 443.
4. Check interval: 2 minutes.
5. Label: timoni registry tcp.
6. Click Save.

Step 4: Monitor Flux Controller for Timoni Bundle Reconciliation

When using Flux to reconcile Timoni bundles via TimoniInstance custom resources, the Flux source and kustomize controllers must be running. A Flux controller crash means bundle updates stop being applied while kubectl get timoniinstances shows stale state:

# Check Flux controller pod status
kubectl get pods -n flux-system

# Check Flux controller logs
kubectl logs -n flux-system deployment/source-controller --tail=30
kubectl logs -n flux-system deployment/kustomize-controller --tail=30

# List Timoni instances and their reconciliation status
kubectl get timoniinstances -A -o wide

# Check for stalled reconciliation
kubectl describe timoniinstance -n default your-bundle-instance
# Look for: "Last Attempted Revision" vs "Last Applied Revision"

If Flux exposes a health endpoint externally:

# Port-forward to check Flux webhook receiver
kubectl port-forward -n flux-system svc/webhook-receiver 9292:9292
curl http://localhost:9292/healthz
# Expected: {"status":"ok"}

In Vigilmon, expose and monitor the Flux webhook receiver:

# flux-webhook-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: flux-webhook-receiver
  namespace: flux-system
spec:
  rules:
  - host: flux.example.com
    http:
      paths:
      - path: /healthz
        pathType: Exact
        backend:
          service:
            name: webhook-receiver
            port:
              number: 9292
  tls:
  - hosts:
    - flux.example.com
    secretName: flux-tls

kubectl apply -f flux-webhook-ingress.yaml

Then in Vigilmon:

1. Add Monitor → HTTP.
2. URL: https://flux.example.com/healthz.
3. Check interval: 2 minutes.
4. Expected status: 200.
5. Keyword: ok.
6. Label: flux controller health.
7. Click Save.

Step 5: Monitor Kubernetes API Server Availability

Timoni applies resources directly to the Kubernetes API. If the API server becomes unavailable, timoni apply operations hang and GitOps reconciliation queues up:

# Check Kubernetes API server health
kubectl get --raw /healthz
# Expected: "ok"

# Check API server readiness
kubectl get --raw /readyz
# Expected: "ok"

# Find your API server address
kubectl cluster-info
# Output: Kubernetes control plane is running at https://api.example.com:6443

In Vigilmon:

1. Add Monitor → HTTP.
2. URL: https://api.example.com:6443/healthz (your API server address).
3. Check interval: 2 minutes.
4. Response timeout: 10 seconds.
5. Expected status: 200.
6. Keyword: ok.
7. Label: kubernetes api server.
8. Click Save.

Also add a TCP monitor on port 6443 for a lower-level connectivity check independent of TLS and HTTP routing:

1. Add Monitor → TCP.
2. Host: api.example.com.
3. Port: 6443.
4. Check interval: 1 minute.
5. Label: kubernetes api server tcp.
6. Click Save.

Step 6: Monitor SSL Certificates

Timoni uses OCI client libraries that validate TLS certificates strictly. A certificate expiry on your module registry causes all timoni apply, timoni mod pull, and timoni bundle apply commands to fail with TLS errors:

# Check the OCI registry certificate
openssl s_client -connect ghcr.io:443 2>/dev/null | \
  openssl x509 -noout -dates

# Check a self-hosted registry certificate
openssl s_client -connect registry.example.com:443 2>/dev/null | \
  openssl x509 -noout -dates

# Check the Flux webhook receiver certificate
openssl s_client -connect flux.example.com:443 2>/dev/null | \
  openssl x509 -noout -dates

# Check the Kubernetes API server certificate
openssl s_client -connect api.example.com:6443 2>/dev/null | \
  openssl x509 -noout -dates

In Vigilmon:

1. Add Monitor → SSL Certificate.
2. Domain: registry.example.com (for self-hosted) or your primary module registry domain.
3. Alert when expiry is within: 30 days.
4. Alert again: 14 days, 7 days, 3 days, 1 day.
5. Click Save.

Repeat for flux.example.com and api.example.com.

Step 7: Configure Alerting and Runbook

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Immediate Action | |---|---|---| | OCI registry | Non-401/200 | Check registry status page; verify DNS resolution; timoni mod pull oci://registry.example.com/module:tag manually | | Flux controller health | Non-200 | kubectl get pods -n flux-system → check crashes; kubectl logs -n flux-system deployment/source-controller --tail=30 | | Kubernetes API TCP | Connection refused | Check control plane node health; verify kube-apiserver process; contact cloud provider if managed cluster | | Kubernetes API HTTP | Non-200 | kubectl get --raw /healthz from a node; check etcd health if self-managed | | Registry TCP | Connection refused | Check registry pod/service; verify Ingress configuration; kubectl get pods -n registry | | SSL certificate | < 30 days to expiry | kubectl get certificates; trigger cert-manager renewal; update registry TLS secret manually if needed |

Escalation policy: Route OCI registry failures to your platform team immediately — Timoni bundle failures cascade to every workload managed by GitOps automation. A registry outage that lasts longer than your Flux reconciliation retry timeout will cause pending configuration changes to queue indefinitely.

Common Timoni Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | GHCR outage prevents module pulls during CI/CD pipeline | OCI registry monitor catches non-401 response; alert within 5 min | | Self-hosted registry runs out of disk space | Registry TCP check succeeds; HTTP check returns 500; alert immediately | | cert-manager fails to renew registry TLS certificate | SSL monitor catches 30 days before timoni apply starts failing with TLS errors | | Flux source controller OOM killed during large bundle reconciliation | Flux health endpoint goes down; reconciliation stalls; alert within 2 min | | Kubernetes API server upgrade causes brief unavailability | API TCP monitor catches connection refused; timoni apply operations fail during upgrade window | | Timoni module pushed with invalid CUE schema breaks all consumers | Not directly caught by Vigilmon; but registry HTTP monitor confirms the push endpoint was reachable | | Flux kustomize controller loses RBAC permission after policy change | Flux health stays green; TimoniInstances show stale revision; monitor /readyz endpoint for controller errors | | Harbor database failure corrupts image index | Harbor ping returns 200; but manifest pull returns 500 — add a specific manifest pull check for critical modules |

Timoni shifts Kubernetes package management toward typed, schema-validated CUE configurations that are distributed as OCI artifacts — but that distribution pipeline depends on an OCI registry that must stay available, a Flux controller that must keep reconciling, and a Kubernetes API server that must accept apply operations. Vigilmon provides independent external monitoring of every layer: the registry availability, the Flux controller health, and the Kubernetes API connectivity that Timoni depends on at every step. When any part of the pipeline stalls, you know within minutes with enough diagnostic context to identify whether the problem is the registry, the controller, or the cluster itself.

Start monitoring Timoni in under 5 minutes — register free at vigilmon.online.