tutorial

How to Monitor OVS-CNI with Vigilmon (HTTP + TCP + Alerts)

OVS-CNI is a Kubernetes CNI plugin that integrates pods with Open vSwitch (OVS), the production-grade virtual switch widely used in OpenStack, NFV infrastruc...

OVS-CNI is a Kubernetes CNI plugin that integrates pods with Open vSwitch (OVS), the production-grade virtual switch widely used in OpenStack, NFV infrastructure, and SDN deployments. Rather than creating simple virtual ethernet pairs, OVS-CNI attaches pod interfaces directly to OVS bridges, enabling fine-grained flow-table control, VLAN tagging, QoS policies, and hardware offload support on SmartNICs. It is commonly paired with Multus CNI to give pods secondary network interfaces governed by OVS flow rules — making it the CNI of choice for telco network functions, virtual CPE platforms, and multi-tenant network virtualisation environments. When OVS-CNI is working correctly, pods receive OVS-managed interfaces, flow rules are installed, and traffic traverses the OVS datapath at near-hardware speeds. When it fails — because the OVS daemon is down, a flow rule is misconfigured, or a VLAN assignment conflicts — pod traffic is silently dropped by OVS without any Kubernetes-visible error. Vigilmon provides the external monitoring that OVS-CNI cannot give itself: continuous uptime checks from outside your infrastructure that detect datapath failures before they impact production traffic. You can add your first monitor for free, no credit card required.

Why OVS-CNI needs external monitoring

  • OVS daemon (ovs-vswitchd) crash silently drops all pod traffic managed by OVS bridges. OVS-CNI attaches pod interfaces to OVS bridges whose forwarding is handled by the ovs-vswitchd daemon. If ovs-vswitchd crashes — due to an OVS upgrade, a kernel module conflict, or a DPDK configuration error — all traffic on OVS-managed pod interfaces is silently dropped. The pod interfaces remain present in the pod (ip link show shows them as UP), the CNI reports no error, and Kubernetes readiness probes that use loopback or in-pod network paths continue to pass. An external HTTP monitor probing a pod endpoint reachable via the OVS bridge detects this datapath failure immediately.

  • Missing OVS flow rule after pod restart causes selective traffic blackhole. OVS-CNI installs OpenFlow rules that forward traffic between the pod interface and the appropriate OVS bridge port. If the pod is restarted and the CNI call to install flow rules fails silently — due to a transient OVS database unavailability, a race condition during rapid pod churn, or a bug in the flow installation logic — the pod receives an interface but no forwarding rules. Traffic destined for that pod enters the OVS bridge but has no matching flow rule and is dropped by the default OVS NORMAL action or by a drop miss rule. Kubernetes reports the pod as ready, but no external traffic reaches it. An external HTTP monitor probing the pod service detects this selective blackhole.

  • VLAN misconfiguration isolates pod traffic on the wrong broadcast domain. OVS-CNI supports VLAN tagging on bridge ports, enabling multi-tenant network isolation. If a NetworkAttachmentDefinition is updated with an incorrect VLAN ID — due to a copy-paste error, a miscommunication between network and platform teams, or an automated provisioning tool deploying the wrong template — newly created pods are attached to the wrong VLAN. Traffic from those pods is broadcast on the wrong VLAN and never reaches the intended recipients. Existing pods on the correct VLAN continue to work, making the issue visible only for the subset of pods created after the misconfiguration. An external HTTP monitor targeting a service backed by newly created pods detects this VLAN isolation failure.

  • OVS database (ovsdb-server) unavailability prevents new pod interfaces from being attached. OVS-CNI communicates with ovsdb-server to create bridge ports and record interface metadata. If ovsdb-server is unavailable — because it is restarting after a crash, because its socket file is missing after a node OS update, or because the OVS package was upgraded and the database schema changed — all new CNI calls fail with a socket connection error. New pods fail to get OVS-managed interfaces and either stay in ContainerCreating or start with only a loopback interface. Pods created before the ovsdb-server failure retain their interfaces, so active traffic continues to flow while the cluster's ability to scale the workload is silently broken. An external HTTP monitor with a synthetic canary pod that is periodically redeployed detects this inability to attach new interfaces.

  • Hardware offload misconfiguration on SmartNICs silently misroutes DPDK flows. OVS-CNI in DPDK mode with SmartNIC hardware offload submits flow rules to the NIC's hardware pipeline. After a SmartNIC firmware update, a BIOS change affecting SR-IOV, or a reconfiguration of the OVS-DPDK binding, hardware-offloaded flows may be installed correctly in OVS's software flow table but not pushed down to the NIC's hardware pipeline. Traffic that should be offloaded instead hits the software path, causing a dramatic latency increase and throughput reduction. From Kubernetes' perspective, everything appears normal. An external HTTP monitor with a response time threshold detects the latency spike that indicates hardware offload failure before it causes application-level SLA violations.

What you'll need

  • A running Kubernetes cluster with OVS-CNI configured (typically via Multus CNI and an OVS NetworkAttachmentDefinition)
  • Open vSwitch installed and running on the Kubernetes nodes (ovs-vswitchd and ovsdb-server)
  • The hostname or IP for an OVS-CNI workload endpoint, for example http://ovs-workload.example.com
  • A free Vigilmon account — monitors start running in under a minute with no credit card required

Step 1: Expose OVS-CNI workload health endpoints

Verify OVS-CNI is attaching interfaces correctly and that workload endpoints are reachable before configuring Vigilmon monitors.

# Check OVS daemons are running on nodes
kubectl get nodes -o wide | awk 'NR>1 {print $1}' | while read NODE; do
  echo -n "Node $NODE: "
  kubectl debug node/$NODE -it --image=ubuntu \
    -- bash -c "systemctl is-active openvswitch-switch 2>/dev/null || ovs-vsctl show 2>/dev/null | head -1" 2>/dev/null
done

# Check OVS bridge configuration on a node
ovs-vsctl show
# Expected output:
# b1234567-...
#     Bridge "br-pod"
#         Port "br-pod"
#             Interface "br-pod"
#                 type: internal
#         Port "veth12345678"
#             Interface "veth12345678"

# Verify a pod has an OVS-managed secondary interface
kubectl exec -n <namespace> <ovs-pod> -- ip addr show
# Expected: net1 interface with an IP address

# Check the OVS port for this pod's interface
kubectl exec -n <namespace> <ovs-pod> -- ip link show net1
IFACE=$(kubectl exec -n <namespace> <ovs-pod> -- ip link show net1 | grep -oP 'net1@\K[^:]+')
ovs-vsctl list port $IFACE

# Verify flow rules are installed for this pod
ovs-ofctl dump-flows br-pod | grep <pod-ip>
# Expected: flow entries matching the pod's IP or MAC

# Test HTTP endpoint
curl -I http://ovs-workload.example.com/health
# Expected:
# HTTP/1.1 200 OK

If the OVS workload is on a private network:

# Check OVS bridge connectivity
ovs-vsctl get bridge br-pod datapath_type
ovs-appctl ofproto/trace br-pod in_port=1,tcp,nw_src=<pod-ip>,nw_dst=<dest-ip>,tp_dst=80

# Check DPDK port status (if using DPDK)
ovs-appctl dpif/show

# Check hardware offload status (if using TC offload or SmartNIC)
ovs-appctl dpctl/show | grep offloaded

Step 2: Set up HTTP monitoring in Vigilmon

Add an HTTP monitor for the OVS-CNI workload health endpoint:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Monitor type to HTTP.
  3. Enter the URL: http://ovs-workload.example.com/health
  4. Set Check interval to 60 seconds and Expected status code to 200.
  5. Click Save Monitor — Vigilmon will run the first check within a minute.

Add a latency threshold monitor for hardware offload detection

OVS hardware offload failures cause latency spikes before causing outright failures. A tight response time threshold catches this:

  1. Click Add Monitor again.
  2. Set Monitor type to HTTP.
  3. Enter the URL: http://ovs-workload.example.com/health
  4. Set Expected status code to 200 and Timeout to 5 seconds.
  5. Set a Response time threshold of 500ms — significantly below normal offloaded-path latency. A sudden increase in response times indicates the DPDK software path is serving traffic that should be hardware-offloaded.
  6. Click Save Monitor.

Why external monitoring catches what internal checks miss

OVS-CNI failures are invisible to Kubernetes. When ovs-vswitchd drops traffic due to a missing flow rule, a pod appears completely healthy from Kubernetes' perspective — readiness probes pass, pod events are clean, and the OVS interface is UP. Vigilmon monitors from entirely outside your infrastructure, sending probes through the same OVS datapath that your production traffic uses. It detects flow table holes, VLAN isolation failures, and DPDK forwarding degradations that application-internal checks cannot observe because those checks either bypass the OVS datapath entirely or run on the same host and miss the failure mode.

Step 3: Monitor OVS-CNI's TCP port

A TCP monitor verifies that the OVS-CNI workload endpoint is accepting connections at the transport layer independently of application health logic or OVS flow complexity.

  1. In Vigilmon, click Add Monitor.
  2. Set Monitor type to TCP.
  3. Enter Host: ovs-workload.example.com and Port: 80 (or the relevant port).
  4. Set Check interval to 60 seconds.
  5. Click Save Monitor.

Discover ports for OVS-CNI workloads:

# List services for OVS-backed workloads
kubectl get svc -A -o wide

# Check OVS flow entries for a specific destination port
ovs-ofctl dump-flows br-pod | grep tp_dst=80

# Verify TCP connectivity through OVS
nc -zv ovs-workload.example.com 80

# Trace a TCP connection through OVS to verify forwarding
ovs-appctl ofproto/trace br-pod \
  in_port=<uplink-port>,tcp,nw_src=<external-ip>,nw_dst=<pod-ip>,tp_dst=80
# Expected: final action should be "output:<pod-port>" not "drop"

Step 4: Configure alert channels

Email alerts

  1. In Vigilmon, go to Alert Channels and click Add Channel.
  2. Choose Email and enter the address that should receive notifications (for example, nfv-ops@example.com or a platform engineering on-call list).
  3. Assign this channel to all OVS-CNI monitors created in Steps 2 and 3.
  4. Click Save — Vigilmon will send a test email to confirm delivery.

Webhook alerts

For Slack, PagerDuty, or a network operations platform, add a webhook channel:

  1. In Alert Channels, click Add Channel and choose Webhook.
  2. Paste your endpoint URL.
  3. Vigilmon will POST the following JSON payload when a monitor transitions to down:
{
  "monitor_id": "mon_ovs_cni_health",
  "monitor_name": "OVS-CNI workload /health",
  "status": "down",
  "previous_status": "up",
  "checked_at": "2026-07-12T09:14:22Z",
  "response_time_ms": null,
  "status_code": null,
  "error": "Connection refused",
  "url": "http://ovs-workload.example.com/health"
}
  1. Assign all OVS-CNI monitors to this channel and click Save.

Runbook: diagnostics to run when an alert fires

#!/bin/bash
# OVS-CNI alert runbook — run these in order when Vigilmon fires

echo "=== Pod status ==="
kubectl get pods -A | grep <ovs-workload>

echo "=== OVS daemon status ==="
systemctl status openvswitch-switch 2>/dev/null || \
  ovs-vsctl show 2>/dev/null | head -5

echo "=== OVS bridge summary ==="
ovs-vsctl show

echo "=== OVS pod interface ==="
kubectl exec -n <namespace> <ovs-pod> -- ip addr show net1 2>/dev/null || \
  echo "Interface query failed — pod may be down"

echo "=== OVS flow rules for pod IP ==="
POD_IP=$(kubectl get pod -n <namespace> <ovs-pod> -o jsonpath='{.status.podIP}')
ovs-ofctl dump-flows br-pod 2>/dev/null | grep "$POD_IP" | head -20

echo "=== HTTP endpoint response ==="
curl -sv http://ovs-workload.example.com/health 2>&1 | tail -20

echo "=== OVS datapath statistics ==="
ovs-dpctl show 2>/dev/null

echo "=== OVS error log ==="
journalctl -u openvswitch-switch --since "10 minutes ago" 2>/dev/null | \
  grep -i "error\|warn\|fail" | tail -20

echo "=== Port connectivity ==="
nc -zv ovs-workload.example.com 80

echo "=== Service endpoints ==="
kubectl get endpoints -n <namespace>

echo "=== Recent cluster events ==="
kubectl get events -A --sort-by='.lastTimestamp' | \
  grep -i "cni\|ovs\|network\|attach" | tail -20

Step 5: Create a public status page

A Vigilmon status page gives your NFV operations team and application owners a single URL to check OVS-CNI workload health during incidents — no cluster access or OVS console required.

  1. In Vigilmon, navigate to Status Pages and click New Status Page.
  2. Give it a name such as OVS-CNI Network Functions and choose a subdomain like ovs-network-status.vigilmon.page.
  3. Add all monitors created in this tutorial: the /health HTTP monitor, the latency threshold HTTP monitor, and the TCP port monitor.
  4. Optionally add a description noting that these workloads use OVS-CNI for SDN-managed pod networking and that a latency alert may indicate hardware offload degradation before a full outage occurs.
  5. Click Publish — the page is immediately accessible and updates in real time.

Share this URL in your NFV operations runbooks and network function documentation. When traffic anomalies are reported in OVS-managed pods, the status page is the first place to check whether the OVS datapath is forwarding correctly.

Putting it all together

| Monitor | Type | What it catches | |---|---|---| | http://ovs-workload.example.com/health | HTTP | OVS daemon down, missing flow rules, VLAN misconfiguration | | http://ovs-workload.example.com/health (latency) | HTTP | Hardware offload failure, DPDK software path fallback, datapath congestion | | ovs-workload.example.com:80 | TCP | Transport-level failure, OVS port not forwarding TCP |

# Quick-reference diagnostic one-liners when a Vigilmon alert fires

# 1. Are OVS daemons running?
systemctl is-active openvswitch-switch || echo "OVS NOT RUNNING"

# 2. Is the health endpoint responding?
curl -sf http://ovs-workload.example.com/health || echo "Health check FAILED"

# 3. Are there flow rules for the pod IP?
POD_IP=$(kubectl get pod -n <namespace> <ovs-pod> -o jsonpath='{.status.podIP}')
ovs-ofctl dump-flows br-pod | grep "$POD_IP" | wc -l

# 4. Is port 80 open?
nc -zv ovs-workload.example.com 80 && echo "Port open" || echo "Port CLOSED"

# 5. What is the OVS bridge port list?
ovs-vsctl list-ports br-pod

# 6. Are there OVS error messages?
journalctl -u openvswitch-switch --since "5 minutes ago" | grep -i error

# 7. What does OVS flow trace say for the failing path?
ovs-appctl ofproto/trace br-pod tcp,nw_dst=<pod-ip>,tp_dst=80 | tail -5

# 8. Are service endpoints healthy?
kubectl get endpoints -n <namespace> | grep <service-name>

What's next

  • Heartbeat monitoring for OVS flow table health — Configure a Kubernetes CronJob that queries ovs-ofctl dump-flows on each node, verifies that the expected minimum number of flow rules is present for active pods, and POSTs a heartbeat to a Vigilmon dead-man's-switch endpoint. If the flow count drops below the threshold — indicating ovs-vswitchd cleared flows without reinstalling them — the heartbeat stops and Vigilmon fires an alert before traffic impact is reported.
  • SSL certificate monitoring — If your OVS-CNI workloads expose HTTPS endpoints, add a Vigilmon SSL monitor to alert before TLS certificate expiry causes connection rejections that appear as unexplained OVS forwarding failures to operators who are not aware of the certificate lifecycle.
  • DPDK throughput baseline tracking — Use Vigilmon's response time history to establish a latency baseline for OVS-DPDK workload endpoints. A sustained latency increase consistent with DPDK software-path fallback typically appears hours before the workload's throughput degrades to the point of causing application-visible errors, giving operators time to reinitialise the hardware offload pipeline before an SLA breach.

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles