tutorial

How to Monitor Kube-router Kubernetes Networking with Vigilmon

Kube-router is an all-in-one Kubernetes networking solution that replaces kube-proxy, provides BGP-based pod networking, and includes a network policy enforc...

Kube-router is an all-in-one Kubernetes networking solution that replaces kube-proxy, provides BGP-based pod networking, and includes a network policy enforcer — all in a single DaemonSet. That consolidation is powerful, but it also means a single kube-router failure can simultaneously break service routing, pod-to-pod communication, and network policy enforcement. Kubernetes will keep reporting pods as Running while traffic silently fails.

This tutorial shows you how to set up external monitoring for services running on a kube-router cluster using Vigilmon, so you detect networking failures from the outside — where your users are.

Why kube-router clusters need external monitoring

Kube-router's failure modes span multiple networking layers, and most of them are invisible to Kubernetes health probes:

  • BGP route withdrawal — kube-router maintains BGP sessions with upstream routers to advertise pod CIDRs; a session drop withdraws all routes, making pods unreachable cluster-wide
  • IPVS rule corruption — kube-router programs IPVS rules instead of iptables for service routing; a crash or upgrade can leave rules inconsistent, causing service traffic to drop silently
  • Network policy rule stale — after a kube-router restart, network policy rules may be momentarily absent, causing traffic that should be blocked to flow, or vice versa
  • BGP peer asymmetry — kube-router on some nodes may lose peer connections while others stay up, causing intermittent routing failures that depend on which node handles a request
  • DSR mode failure — in Direct Server Return mode, the return path bypasses kube-router; a routing table misconfiguration causes asymmetric routing and dropped replies

None of these surface as pod health failures. External monitoring from multiple regions is the authoritative signal for reachability.

What you'll need

  • A Kubernetes cluster running kube-router (installed as a DaemonSet)
  • At least one service accessible externally (via NodePort, LoadBalancer, or Ingress)
  • A free Vigilmon account

Step 1: Verify kube-router is functioning and your service is reachable

# Check kube-router DaemonSet — all nodes should show DESIRED == READY
kubectl get ds kube-router -n kube-system

# Check individual pod logs for BGP session status
kubectl logs -n kube-system -l k8s-app=kube-router --tail=50 | grep -E "BGP|peer|ESTABLISHED|ACTIVE"

# List your externally accessible services
kubectl get svc -A --field-selector spec.type!=ClusterIP

# Test from outside the cluster
curl http://<NODE_IP>:<NODE_PORT>/health

If BGP sessions show as ACTIVE (trying to connect) rather than ESTABLISHED, you have a BGP peer issue that needs fixing before monitoring will be meaningful.

Add a health endpoint

Your application needs a /health endpoint for HTTP monitoring:

// Node.js / Express
app.get('/health', (req, res) => {
  res.json({ status: 'ok', service: 'my-api' });
});

# Expose it via NodePort for external access
apiVersion: v1
kind: Service
metadata:
  name: my-api
spec:
  type: NodePort
  selector:
    app: my-api
  ports:
    - port: 3000
      targetPort: 3000
      nodePort: 30080

Step 2: Set up HTTP monitoring

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL:
    • http://<NODE_IP>:30080/health for NodePort access
    • https://api.yourdomain.com/health if you have a domain with an Ingress
  4. Check interval: 1 minute
  5. Expected status: 200
  6. Optional body check: "status":"ok"
  7. Save

Vigilmon's multi-region probes validate reachability from outside your cluster. Since kube-router handles both pod routing and service routing, a single Vigilmon alert can indicate either layer has failed.

Monitor multiple node IPs for DaemonSet failures

Because kube-router runs on every node, partial failures (one node losing BGP) cause intermittent errors. If your NodePort service could be served by any node, monitor multiple node IPs:

Create separate monitors for:

  • http://node-1.example.com:30080/health
  • http://node-2.example.com:30080/health
  • http://node-3.example.com:30080/health

If one goes red while others stay green, a specific kube-router pod has a problem on that node.

Step 3: Monitor TCP connectivity

TCP monitoring catches port-level failures independently of the HTTP application:

  1. Go to Monitors → New Monitor → TCP Port
  2. Host: your node IP or domain
  3. Port: your NodePort (e.g., 30080)
  4. Interval: 1 minute
  5. Save

For services using kube-router's DSR (Direct Server Return) mode, TCP monitoring is especially valuable because DSR failures often manifest as TCP-level drops (SYN sent, no SYN-ACK) rather than HTTP errors.

Step 4: Diagnose kube-router networking failures

When Vigilmon alerts, use these commands to correlate:

# 1. Check kube-router pod health across all nodes
kubectl get pods -n kube-system -l k8s-app=kube-router -o wide

# 2. Check BGP session status on a specific node
NODE_POD=$(kubectl get pods -n kube-system -l k8s-app=kube-router \
  -o jsonpath='{.items[0].metadata.name}')
kubectl exec -n kube-system $NODE_POD -- gobgp neighbor

# 3. Check IPVS rules for your service
kubectl exec -n kube-system $NODE_POD -- ipvsadm -Ln

# 4. Verify service ClusterIP is present in IPVS
SVC_IP=$(kubectl get svc my-api -o jsonpath='{.spec.clusterIP}')
kubectl exec -n kube-system $NODE_POD -- ipvsadm -Ln | grep $SVC_IP

# 5. Check BGP routes are being advertised
kubectl exec -n kube-system $NODE_POD -- gobgp global rib

# 6. Network policy rules
kubectl exec -n kube-system $NODE_POD -- iptables -L KUBE-ROUTER-INPUT -n -v

The combination of Vigilmon's external signal and these diagnostics tells you exactly which kube-router component failed.

Step 5: Configure alert channels

  1. Go to Alert Channels → Add Channel → Email and enter your team's ops email
  2. Go to Alert Channels → Add Channel → Webhook for Slack or PagerDuty integration

Example automated remediation triggered by a Vigilmon webhook:

#!/bin/bash
# auto-remediate.sh — triggered by Vigilmon webhook
# Attempt to restart kube-router on nodes with failed BGP sessions

FAILED_NODES=$(kubectl get pods -n kube-system -l k8s-app=kube-router \
  --field-selector=status.phase!=Running \
  -o jsonpath='{.items[*].spec.nodeName}')

for NODE in $FAILED_NODES; do
  echo "Restarting kube-router on $NODE"
  POD=$(kubectl get pods -n kube-system -l k8s-app=kube-router \
    --field-selector=spec.nodeName=$NODE \
    -o jsonpath='{.items[0].metadata.name}')
  kubectl delete pod -n kube-system $POD
done

Step 6: Heartbeat monitoring for BGP-dependent workloads

If you have scheduled jobs that depend on cross-node communication routed by kube-router's BGP, heartbeat monitors catch failures that HTTP monitoring misses:

  1. Go to Monitors → New Monitor → Heartbeat
  2. Name: cross-node-job
  3. Expected interval: match your job's schedule
  4. Copy the heartbeat URL

In your job:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: cross-node-sync
spec:
  schedule: "*/15 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: sync
              image: your-registry/sync:latest
              command:
                - /bin/sh
                - -c
                - |
                  ./sync-job --target http://my-api.production.svc.cluster.local/api
                  curl -sf https://vigilmon.online/heartbeat/YOUR_TOKEN
          restartPolicy: OnFailure

A missing heartbeat means the job failed — whether due to kube-router routing issues, pod scheduling, or application errors.

Step 7: Create a status page

  1. Go to Status Pages → New Status Page
  2. Name: Kubernetes Network Services
  3. Add monitors: HTTP checks per node, TCP checks, heartbeats
  4. Group by: Control Plane, Application Services, Scheduled Jobs
  5. Publish

Share the URL with your SRE team and stakeholders.

Monitoring coverage map

| Monitor | Type | Catches | |---------|------|---------| | http://node-1:30080/health | HTTP | App errors, kube-router IPVS failure on node-1 | | http://node-2:30080/health | HTTP | Partial DaemonSet failure on node-2 | | node-1.example.com:30080 | TCP | DSR mode failures, TCP-layer drops | | https://api.yourdomain.com/health | HTTP | Ingress + full routing stack | | cross-node-job | Heartbeat | BGP-dependent inter-node job failures |

What's next

  • BGP peer alerting — hook kube-router's BGP state changes into a webhook that creates a Vigilmon maintenance window when peer sessions are intentionally disrupted during upgrades
  • Canary monitoring — during kube-router upgrades (rolling DaemonSet updates), monitor per-node endpoints to catch regressions before the rollout completes
  • SSL certificate monitoring — if your Ingress terminates TLS, Vigilmon tracks cert expiry independently of cert-manager

Get started free at vigilmon.online — no credit card, monitors are live in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles