tutorial

How to Monitor KubeSlice with Vigilmon (HTTP + TCP + Alerts)

KubeSlice is a CNCF project that creates application slices — encrypted network tunnels that connect workloads across multiple Kubernetes clusters. It operat...

KubeSlice is a CNCF project that creates application slices — encrypted network tunnels that connect workloads across multiple Kubernetes clusters. It operates on a hub-spoke model: a KubeSlice Controller runs in a central hub cluster and manages slice configuration, while KubeSlice Worker operators run in each spoke cluster and handle the actual gateway pods and inter-cluster routing. When a Worker operator crashes, a gateway pod fails, or the slice subnet routing breaks, workloads in different clusters lose connectivity to each other while both clusters individually report healthy. Neither cluster's standard readiness checks can see the cross-cluster path, so failures accumulate silently until users hit errors.

In this tutorial you'll set up comprehensive uptime monitoring for your KubeSlice deployment using Vigilmon — free tier, no credit card required.

Why KubeSlice deployments need external monitoring

KubeSlice introduces a class of failure that neither the hub cluster nor the spoke clusters can surface independently:

  • Worker operator crashes — the KubeSlice Worker deployment in a spoke cluster exits; slice gateway pods are no longer reconciled; existing connections drop and new pods cannot join the slice, while the spoke cluster's own health checks remain green
  • Gateway pod failures — the slice gateway pod that terminates the IPSec/VXLAN tunnel goes CrashLoopBackOff; inter-cluster traffic on port 4500 is silently dropped; workloads in the spoke cluster lose their route to workloads in other clusters
  • Slice subnet routing breaks — the overlay route for the slice subnet (e.g. 192.168.0.0/16) is withdrawn from the gateway; pods that send traffic to cross-cluster service IPs get no response and no ICMP error, making the failure look like application-level timeouts
  • Inter-cluster DNS resolution fails — the KubeSlice DNS plugin that resolves <service>.<slice>.svc.slice.local stops responding; applications see NXDOMAIN for cross-cluster service names while intra-cluster DNS continues to work normally
  • AppPod connectivity to slice gateway breaks — a network policy change or node-level iptables flush removes the route between application pods and the local slice gateway; the gateway pod is Running but no application traffic reaches it

These failures are especially damaging because Kubernetes node and pod conditions only reflect local cluster state. A slice that appears fully provisioned in both clusters can have zero functional cross-cluster connectivity.

What you'll need

  • A KubeSlice deployment with at least one hub cluster running the Controller and one spoke cluster running the Worker operator
  • A workload slice with at least one AppPod exposed through a SliceServiceExport or a test service reachable from outside the cluster
  • A free Vigilmon account (sign up takes 30 seconds)

Step 1: Expose a health endpoint for monitoring

KubeSlice does not expose a single external health URL by default. You need to expose the Controller's readiness endpoint and deploy a cross-cluster probe workload that validates the full slice data path.

First, verify your KubeSlice components are running:

# Check the Controller in the hub cluster (context: hub)
kubectl --context hub get pods -n kubeslice-hub
# NAME                                          READY   STATUS    RESTARTS   AGE
# kubeslice-controller-manager-6d8f9b7c-vr4xp   2/2     Running   0          3d

# Check the Worker operator in a spoke cluster (context: spoke-1)
kubectl --context spoke-1 get pods -n kubeslice-system
# NAME                                        READY   STATUS    RESTARTS   AGE
# kubeslice-worker-operator-5c9d8f6b4-tn7mk   1/1     Running   0          3d

# Check slice gateway pods
kubectl --context spoke-1 get pods -n kubeslice-system -l kubeslice.io/pod-type=slicegateway
# NAME                          READY   STATUS    RESTARTS   AGE
# ksvc-gw-spoke-1-hub-0-abcde   1/1     Running   0          3d

# Check the slice object
kubectl --context hub get slice -n kubeslice-hub
# NAME          AGE
# prod-slice    3d

Deploy a cross-cluster probe service that accepts HTTP requests and returns a JSON health response. Apply this in the spoke cluster so Vigilmon can validate the full path from outside:

# kubeslice-probe.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kubeslice-probe
  namespace: kubeslice-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kubeslice-probe
  template:
    metadata:
      labels:
        app: kubeslice-probe
    spec:
      containers:
        - name: probe
          image: nginx:alpine
          ports:
            - containerPort: 80
          volumeMounts:
            - name: config
              mountPath: /etc/nginx/conf.d
          readinessProbe:
            httpGet:
              path: /healthz
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 10
      volumes:
        - name: config
          configMap:
            name: kubeslice-probe-nginx
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: kubeslice-probe-nginx
  namespace: kubeslice-system
data:
  default.conf: |
    server {
      listen 80;
      location /healthz {
        return 200 '{"status":"ok","component":"kubeslice-probe","slice":"prod-slice"}';
        add_header Content-Type application/json;
      }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: kubeslice-probe
  namespace: kubeslice-system
spec:
  type: LoadBalancer
  selector:
    app: kubeslice-probe
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP

kubectl --context spoke-1 apply -f kubeslice-probe.yaml

# Get the LoadBalancer IP
kubectl --context spoke-1 get svc kubeslice-probe -n kubeslice-system
# NAME              TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)   AGE
# kubeslice-probe   LoadBalancer   10.96.44.120    203.0.113.71    80/TCP    2m

# Confirm the health endpoint responds
curl http://203.0.113.71/healthz
# {"status":"ok","component":"kubeslice-probe","slice":"prod-slice"}

Step 2: Set up HTTP monitoring in Vigilmon

With your probe endpoint exposed, add it to Vigilmon:

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS as the type
  3. Set the URL to your probe endpoint: http://203.0.113.71/healthz
  4. Set the check interval to 1 minute
  5. Under Expected response, set:
    • Status code: 200
    • Response body contains: "status":"ok"
  6. Save the monitor

This monitor validates several layers of the KubeSlice stack at once: the Worker operator must be reconciling the namespace, the probe pod must be scheduled and running, and the LoadBalancer service must be routing correctly. If any of these fail, Vigilmon catches it within a minute.

Why external monitoring catches what Kubernetes misses

KubeSlice slice health is not reflected in standard Kubernetes node or pod conditions. A spoke cluster can show all pods Running while its slice gateway has a broken IPSec association and all cross-cluster traffic is being silently dropped. Vigilmon probes from outside both clusters over the public network, so it validates the actual reachable state — not just what Kubernetes reports internally. Its multi-region consensus model prevents false positives from transient network blips.

Step 3: Monitor TCP ports and the KubeSlice gateway endpoint

Add monitors to detect failures at the gateway and operator level:

Slice gateway TCP check (IPSec/VXLAN port):

  1. Go to Monitors → New Monitor
  2. Choose TCP Port
  3. Enter: Host 203.0.113.71, Port 4500
  4. Save the monitor

KubeSlice Controller health via Ingress:

Expose the Controller's readiness probe endpoint through an Ingress in the hub cluster so Vigilmon can probe it directly:

# kubeslice-controller-ingress.yaml
apiVersion: v1
kind: Service
metadata:
  name: kubeslice-controller-health
  namespace: kubeslice-hub
spec:
  selector:
    control-plane: controller-manager
  ports:
    - port: 8081
      targetPort: 8081
      name: health
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kubeslice-controller-health
  namespace: kubeslice-hub
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /healthz/ready
spec:
  rules:
    - host: kubeslice-ctrl.your-hub-cluster.com
      http:
        paths:
          - path: /healthz/ready
            pathType: Exact
            backend:
              service:
                name: kubeslice-controller-health
                port:
                  number: 8081

kubectl --context hub apply -f kubeslice-controller-ingress.yaml

Add an HTTP monitor in Vigilmon for https://kubeslice-ctrl.your-hub-cluster.com/healthz/ready expecting status 200 and body ok.

Also add a TCP monitor for the probe service port directly:

  1. Go to Monitors → New Monitor
  2. Choose TCP Port
  3. Enter: Host 203.0.113.71, Port 80
  4. Save the monitor

Step 4: Configure alert channels

A KubeSlice failure silently severs connectivity between clusters — services fail, but each cluster looks healthy on its own dashboard.

Email alerts

  1. In Vigilmon, go to Alert Channels → Add Channel → Email
  2. Enter your platform team's on-call email
  3. Assign the channel to all KubeSlice monitors

Webhook alerts (Slack, PagerDuty, etc.)

  1. Go to Alert Channels → Add Channel → Webhook
  2. Enter your webhook URL
  3. Assign the channel to your monitors

Example payload when the KubeSlice probe goes down:

{
  "monitor_name": "KubeSlice Probe Health",
  "status": "down",
  "url": "http://203.0.113.71/healthz",
  "started_at": "2024-01-15T11:20:00Z",
  "duration_seconds": 90
}

Run these diagnostics immediately when an alert fires:

# Check Worker operator in spoke cluster
kubectl --context spoke-1 get pods -n kubeslice-system
kubectl --context spoke-1 logs -n kubeslice-system -l app=kubeslice-worker-operator --tail=100

# Check slice gateway pod status
kubectl --context spoke-1 get pods -n kubeslice-system -l kubeslice.io/pod-type=slicegateway
kubectl --context spoke-1 describe pod -n kubeslice-system -l kubeslice.io/pod-type=slicegateway

# Check slice status in hub cluster
kubectl --context hub get slice -n kubeslice-hub -o yaml | grep -A20 status

# Check Controller logs for reconciliation errors
kubectl --context hub logs -n kubeslice-hub -l control-plane=controller-manager --tail=200 | grep -i "error\|failed\|reconcil"

# Check gateway connectivity on port 4500
nc -zv 203.0.113.71 4500

# Inspect slice subnet routing in spoke
kubectl --context spoke-1 get slicegateways -n kubeslice-system -o wide

Step 5: Create a public status page

Multi-cluster platforms built on KubeSlice often serve internal development teams or external APIs. Give them visibility.

  1. In Vigilmon, go to Status Pages → New Status Page
  2. Give it a name: "KubeSlice Multi-Cluster Status"
  3. Add your monitors: KubeSlice Probe HTTP, Gateway TCP 4500, Probe TCP 80, Controller Health
  4. Group them: Slice Connectivity, Gateway Ports, Controller Health
  5. Publish the page

Share this URL with application teams whose services depend on cross-cluster connectivity through KubeSlice slices.

Putting it all together

Here's a summary of the monitors to create for a KubeSlice deployment:

| Monitor | Type | What it catches | |---------|------|-----------------| | http://203.0.113.71/healthz | HTTP | Worker operator, pod scheduling, LoadBalancer routing | | 203.0.113.71:4500 | TCP | IPSec/VXLAN gateway port reachability | | 203.0.113.71:80 | TCP | Probe service TCP reachability | | https://kubeslice-ctrl.your-hub-cluster.com/healthz/ready | HTTP | KubeSlice Controller readiness |

And the KubeSlice diagnostics to run when an alert fires:

# Quick triage: Worker operator or gateway pod?
kubectl --context spoke-1 get pods -n kubeslice-system
kubectl --context spoke-1 get slicegateways -n kubeslice-system

# Check Worker operator reconciliation errors
kubectl --context spoke-1 logs -n kubeslice-system deploy/kubeslice-worker-operator | grep -E "error|failed|reconcil"

# Check gateway IPSec associations
kubectl --context spoke-1 exec -n kubeslice-system -l kubeslice.io/pod-type=slicegateway -- ip xfrm state list

# Verify slice subnet routes are present
kubectl --context spoke-1 exec -n kubeslice-system -l kubeslice.io/pod-type=slicegateway -- ip route show

# Check events in slice namespace
kubectl --context spoke-1 get events -n kubeslice-system --sort-by='.lastTimestamp' | tail -20

# Check Controller reconciliation in hub
kubectl --context hub logs -n kubeslice-hub deploy/kubeslice-controller-manager | grep -E "error|quota|401|403"

What's next

  • SSL certificate monitoring — add SSL monitors for any HTTPS endpoints exported through KubeSlice ServiceExports
  • Heartbeat monitoring — deploy a cross-cluster job that pings a slice service from one cluster to another on a schedule and sends a Vigilmon heartbeat on success
  • Per-slice coverage — if you run multiple application slices, create a dedicated monitor set per slice to isolate which slice is degraded when an alert fires

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles