How to Monitor Multus CNI Network Interfaces with Vigilmon

Multus CNI lets Kubernetes pods attach multiple network interfaces — typically one for cluster traffic and additional interfaces for storage networks, SR-IOV, or direct hardware access. When Multus works correctly, pods get the connectivity they need at the speed they need it. When Multus fails, pods come up with missing interfaces and no clear error surfaced to the operator.

In this tutorial you'll set up monitoring for your Multus CNI-managed network infrastructure using Vigilmon — free tier, no credit card required.

Why Multus CNI needs external monitoring

Multus failures are subtle. The cluster continues to schedule pods, but those pods silently lack the network interfaces they depend on:

• Network attachment definition misconfiguration — a NetworkAttachmentDefinition change breaks attachment; new pods start without the secondary interface and fail only at the application layer
• CNI plugin binary gone — a node upgrade removes a CNI plugin binary that Multus delegates to; the next pod scheduled to that node fails attachment with a cryptic runtime error
• IPAM pool exhaustion — the secondary network's IP address pool runs out; pods queue indefinitely waiting for interface assignment
• SR-IOV device plugin saturation — SR-IOV virtual function allocations are exhausted; latency-sensitive workloads fail to schedule
• Multus webhook failure — the Multus admission webhook becomes unresponsive; pods are scheduled without any annotation processing

External monitoring of services running on Multus-managed interfaces catches the real-world impact of these failures — not just the controller state.

What you'll need

• A Kubernetes cluster with Multus CNI installed
• At least one workload using secondary network interfaces
• A free Vigilmon account (sign up takes 30 seconds)

Step 1: Deploy a health-check service on your secondary network

The most reliable way to monitor Multus is to run a service that uses a secondary interface and verify it's reachable.

First, confirm your NetworkAttachmentDefinition is in place:

# monitoring-network.yaml
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: monitoring-net
  namespace: network-monitoring
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "monitoring-net",
      "type": "macvlan",
      "master": "eth1",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.200.0/24",
        "rangeStart": "192.168.200.10",
        "rangeEnd": "192.168.200.50"
      }
    }

Deploy a lightweight health-check pod that listens on the secondary interface:

# multus-health-pod.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: multus-health-probe
  namespace: network-monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: multus-health-probe
  template:
    metadata:
      labels:
        app: multus-health-probe
      annotations:
        k8s.v1.cni.cncf.io/networks: monitoring-net
    spec:
      containers:
        - name: probe
          image: nginx:alpine
          ports:
            - containerPort: 80
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 10

Expose this pod via a NodePort on the secondary network node:

# multus-health-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: multus-health-svc
  namespace: network-monitoring
spec:
  type: NodePort
  selector:
    app: multus-health-probe
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30880

kubectl apply -f monitoring-network.yaml
kubectl apply -f multus-health-pod.yaml
kubectl apply -f multus-health-svc.yaml

Verify the secondary interface was attached:

kubectl exec -n network-monitoring deploy/multus-health-probe -- ip addr show
# Should show eth0 (cluster network) AND net1 (secondary from monitoring-net)

Step 2: Set up HTTP monitoring in Vigilmon

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS as the type
3. Set the URL to your health probe service, e.g. http://your-node.example.com:30880/
4. Set the check interval to 1 minute
5. Under Expected response, set status code to 200
6. Save the monitor

If Multus fails to attach the secondary interface, your health probe pod will fail its readiness check, the service will have no endpoints, and Vigilmon will immediately trigger an alert.

Step 3: Monitor Multus CNI TCP connectivity

For TCP-based services on secondary interfaces (storage, SR-IOV data plane, bare-metal networking), add TCP port monitors directly:

1. In Vigilmon, go to Monitors → New Monitor
2. Choose TCP Port
3. Enter the external IP or hostname of the node running your secondary-network service and its port
4. Save the monitor

This is especially important for SR-IOV workloads. A TCP monitor on the SR-IOV data-plane port catches VF allocation failures within seconds of them occurring.

Example — monitoring a high-performance UDP/TCP service on a secondary interface:

# Check which node is running SR-IOV workloads
kubectl get pods -n sriov-workloads -o wide

# Verify the VF is bound
kubectl exec -n sriov-workloads <pod-name> -- ip link show

Then add a TCP monitor for the data port in Vigilmon.

Step 4: Add a Multus attachment verification CronJob

Periodically verify that Multus is actually attaching interfaces — not just that the controller is running:

# multus-verify-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: multus-verify
  namespace: network-monitoring
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: verify
              image: bitnami/kubectl:latest
              command:
                - /bin/sh
                - -c
                - |
                  # Check that the probe pod has a secondary interface
                  POD=$(kubectl get pod -n network-monitoring \
                    -l app=multus-health-probe \
                    -o jsonpath='{.items[0].metadata.name}')
                  
                  # Verify annotation shows attached networks
                  NETWORKS=$(kubectl get pod -n network-monitoring $POD \
                    -o jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}')
                  
                  if echo "$NETWORKS" | grep -q "monitoring-net"; then
                    curl -fsS "https://hb.vigilmon.online/your-heartbeat-token"
                    echo "Multus attachment verified"
                  else
                    echo "ERROR: secondary interface not attached" >&2
                    exit 1
                  fi

Create a Heartbeat monitor in Vigilmon with a 10-minute interval and configure the CronJob to ping it on success. If the CronJob stops pinging — because Multus attachment is broken — you'll get an alert.

Step 5: Configure alert channels

1. In Vigilmon, go to Alert Channels → Add Channel → Webhook
2. Enter your Slack or PagerDuty webhook URL
3. Assign all Multus monitors to the channel

When an alert fires, use these commands to diagnose:

# Check Multus pod
kubectl get pods -n kube-system | grep multus

# Check recent attachment events
kubectl get events -n network-monitoring --sort-by=.metadata.creationTimestamp

# Check network status annotations on probe pod
kubectl get pod -n network-monitoring \
  -l app=multus-health-probe \
  -o jsonpath='{.items[0].metadata.annotations}' | python3 -m json.tool

# Check CNI plugin binaries are present on nodes
kubectl get nodes -o wide
# SSH to node and verify:
# ls /opt/cni/bin/

Step 6: Create a status page for network infrastructure

If your Multus-managed networks underpin storage or inter-service communication for multiple teams:

1. In Vigilmon, go to Status Pages → New Status Page
2. Name it "Secondary Network Infrastructure"
3. Add your monitors and publish

Putting it all together

| Monitor | Type | What it catches | |---------|------|-----------------| | http://node:30880/ | HTTP | Multus attachment failures, pod readiness failures | | node:<data-plane-port> | TCP | SR-IOV or secondary network data plane failures | | Attachment verification heartbeat | Heartbeat | Silent Multus CNI delegation failures |

Key diagnostic commands when Vigilmon alerts:

# Multus controller health
kubectl get pods -n kube-system -l app=multus

# Failed pod annotations (missing network attachments)
kubectl get pods -A -o json | \
  jq '.items[] | select(.metadata.annotations["k8s.v1.cni.cncf.io/networks"] != null) | 
  {name: .metadata.name, status: .metadata.annotations["k8s.v1.cni.cncf.io/network-status"]}'

# NetworkAttachmentDefinition health
kubectl get net-attach-def -A

What's next

• Node-level CNI monitoring — add monitors for individual nodes hosting SR-IOV workloads to detect node-level NIC or driver failures
• IPAM pool monitoring — expose IPAM pool utilization via a custom metrics endpoint and alert when pools approach exhaustion
• Multi-cluster coverage — if you run Multus across multiple clusters for different network topologies, create monitor groups for each cluster

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.