Monitoring Calico with Vigilmon: Typha Health, Felix Availability, BGP Peering Heartbeats & SSL Certificates

Calico is Tigera's open-source CNI plugin and network policy engine trusted to secure and connect workloads at massive scale — from single-node dev clusters to multi-cloud production environments running hundreds of thousands of pods. Calico's Felix agent programs iptables and eBPF rules on every node, while Typha scales policy distribution in large clusters. When Felix crashes or Typha loses quorum, network policy enforcement silently degrades and pods that should be blocked can reach each other freely. Vigilmon gives you external visibility into Calico's control plane health, BGP peering status, and certificate validity so you know your security posture is sound before a policy enforcement gap becomes a breach.

What You'll Build

• An HTTP monitor on the Calico API server or Typha health endpoint
• A TCP monitor confirming the Typha port is reachable
• Heartbeat monitors verifying Felix is running on all nodes
• SSL certificate monitoring for your Calico API endpoint
• Alerting runbook mapped to Calico failure modes

Prerequisites

• A Kubernetes cluster with Calico installed as the CNI
• kubectl access to the calico-system or kube-system namespace
• Optionally: Typha deployed and accessible (auto-deployed when node count exceeds 50)
• A free account at vigilmon.online

Step 1: Verify Calico Component Health

Calico exposes health endpoints on each component. Check them to confirm they are reporting healthy:

# Check Calico node (Felix) status on each node
kubectl get pods -n calico-system -l app.kubernetes.io/name=calico-node

# Check Typha health (if deployed)
kubectl get pods -n calico-system -l app.kubernetes.io/name=calico-typha

# Query Typha health directly from within the cluster
curl -s http://calico-typha.calico-system.svc.cluster.local:9098/readiness

A healthy Typha returns ready. Felix health can be checked per-node via calicoctl node status.

Step 2: Create a Vigilmon HTTP Monitor for the Calico API Server

If you run Calico Enterprise or Tigera Operator with an external API server, set up an HTTP monitor for availability:

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: https://calico-api.yourdomain.com/healthz (replace with your Calico API server URL).
3. Check interval: 60 seconds.
4. Response timeout: 10 seconds.
5. Expected status: 200.
6. Label: Calico API Server Health.
7. Click Save.

For open-source Calico without a separate API server, use the heartbeat monitors in Steps 4–5 as your primary health signals.

Step 3: Monitor the Typha Port with TCP

Typha aggregates NetworkPolicy updates from the Kubernetes API server and fans them out to Felix agents. It listens on port 5473 by default. A Typha failure in large clusters means Felix agents can no longer receive policy updates:

1. Add Monitor → TCP.
2. Host: calico-typha.yourdomain.com (your externally-accessible Typha endpoint, or a NodePort).
3. Port: 5473.
4. Check interval: 2 minutes.
5. Label: Calico Typha Port.
6. Click Save.

If Typha is not externally exposed, use the heartbeat in Step 5 as your primary Typha health signal.

Step 4: Monitor Felix (calico-node) Health via a Heartbeat

Felix is the per-node agent that programs network policy into iptables or eBPF. A Felix failure on a node means that node stops enforcing NetworkPolicy — pods on it may communicate freely regardless of policy rules. Set up a CronJob that verifies all calico-node DaemonSet pods are ready:

Step 4a — Create a Heartbeat monitor in Vigilmon:

1. Add Monitor → Heartbeat.
2. Name: Calico Felix DaemonSet Health.
3. Expected interval: 5 minutes.
4. Grace period: 3 minutes.
5. Copy the generated ping URL (e.g., https://vigilmon.online/ping/abc123).

Step 4b — Deploy the CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: calico-felix-heartbeat
  namespace: calico-system
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: calico-node
          containers:
            - name: check
              image: bitnami/kubectl:latest
              command:
                - sh
                - -c
                - |
                  DESIRED=$(kubectl get daemonset calico-node -n calico-system \
                    -o jsonpath='{.status.desiredNumberScheduled}')
                  READY=$(kubectl get daemonset calico-node -n calico-system \
                    -o jsonpath='{.status.numberReady}')
                  if [ "$DESIRED" -eq "$READY" ] && [ "$READY" -gt "0" ]; then
                    curl -fsS -m 10 https://vigilmon.online/ping/abc123
                  fi
          restartPolicy: OnFailure

Step 5: Monitor Typha Availability via a Heartbeat

Typha is a Deployment (not a DaemonSet) and Calico targets at least one ready replica. A Typha failure in clusters with many nodes means Felix agents fall back to polling the Kubernetes API directly, increasing API server load, and eventually fail to receive policy updates:

Step 5a — Create a Heartbeat monitor in Vigilmon:

1. Add Monitor → Heartbeat.
2. Name: Calico Typha Health.
3. Expected interval: 5 minutes.
4. Grace period: 5 minutes.
5. Copy the ping URL.

Step 5b — Deploy the CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: calico-typha-heartbeat
  namespace: calico-system
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: calico-node
          containers:
            - name: check
              image: bitnami/kubectl:latest
              command:
                - sh
                - -c
                - |
                  READY=$(kubectl get deployment calico-typha -n calico-system \
                    -o jsonpath='{.status.readyReplicas}' 2>/dev/null || echo 0)
                  if [ "${READY:-0}" -ge "1" ]; then
                    curl -fsS -m 10 https://vigilmon.online/ping/abc123
                  fi
          restartPolicy: OnFailure

Step 6: Monitor the Calico API SSL Certificate

Calico's internal TLS certificates (used for Felix-Typha communication and the Calico API server) must remain valid. An expired certificate causes Felix to lose connectivity to Typha and breaks calicoctl operations:

1. Add Monitor → SSL Certificate.
2. Domain: calico-api.yourdomain.com (or your Calico API ingress endpoint).
3. Alert when expiry is within: 30 days.
4. Alert again at: 14 days, 7 days, 3 days, 1 day.
5. Click Save.

For self-managed Calico certificates, also monitor the internal CA rotation schedule and set a calendar reminder aligned with your cert TTL.

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | Calico API server | Non-200 response | Check tigera-operator pod; review Calico CRD status | | Typha TCP port | Connection refused | kubectl get pods -n calico-system -l app=calico-typha | | Felix heartbeat | No ping within 8 min | kubectl get ds calico-node -n calico-system; check NotReady nodes | | Typha heartbeat | No ping within 10 min | kubectl logs -n calico-system -l app=calico-typha --tail=50 | | SSL certificate | < 30 days | Rotate Calico CA via calicoctl; renew Ingress TLS |

Common Calico Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Felix pod OOMKilled on a node | DaemonSet heartbeat detects ready count drop | | Typha deployment crashlooping | Typha heartbeat expires; Felix API server load spikes | | BGP peer session drops | Felix logs show route distribution errors; heartbeat eventually fires | | iptables rule corruption | Felix restarts; DaemonSet heartbeat briefly stops | | Calico-node image pull failure on new node | DaemonSet heartbeat detects desired vs ready mismatch | | Calico API server TLS cert expired | SSL monitor alerts at 30-day threshold | | NetworkPolicy not enforced on node | Felix heartbeat still pings, but calicoctl node status shows issues |

Calico is often invisible when it works and catastrophic when it doesn't — a silent Felix failure means workloads bypass your security perimeter without any application-layer error. Vigilmon's external monitoring of Felix DaemonSet health, Typha availability, and SSL certificates gives you independent verification that your network policy enforcement layer is operating correctly across every node in your cluster.

Start monitoring Calico in under 5 minutes — register free at vigilmon.online.