tutorial

How to Monitor PureLB Load Balancers with Vigilmon

PureLB brings the `LoadBalancer` service type to bare-metal Kubernetes clusters — no cloud provider required. It allocates external IP addresses from local a...

PureLB brings the LoadBalancer service type to bare-metal Kubernetes clusters — no cloud provider required. It allocates external IP addresses from local address pools, announces them via ARP or BGP, and routes traffic directly to node ports. But bare-metal networking is unforgiving: a misconfigured IPAM pool, a silent BGP peer withdrawal, or a NIC that stops responding to ARP requests can make a service unreachable while Kubernetes still reports every pod as Running.

This tutorial shows you how to set up external monitoring for PureLB-managed services using Vigilmon, so you know about bare-metal reachability failures before your users do.

Why PureLB services need external monitoring

PureLB's job is to make LoadBalancer services work on bare metal. Its failure modes are different from cloud load balancers and from the application layer that Kubernetes normally watches:

  • IP allocation exhausted — the IPAM pool runs out of addresses; new services stay in <pending> state indefinitely but existing pods report healthy
  • ARP announcement failure — PureLB allocates an IP but the ARP broadcast never reaches the upstream switch, so the IP is unreachable despite being "assigned"
  • BGP peer disconnect — in BGP mode a peer router withdraws routes silently; traffic stops flowing with no Kubernetes-layer event
  • Virtual IP binding lost — a node reboot or network interface reconfiguration can cause PureLB to lose the virtual IP binding; the service appears assigned but is unreachable
  • LBNodePort misconfiguration — traffic reaches the assigned IP but the node-port forwarding rule is wrong, leading to connection resets at the TCP layer

None of these trigger Kubernetes liveness or readiness probe failures, because the pods themselves are healthy. External monitoring is the only layer that catches reachability from outside the cluster.

What you'll need

  • A Kubernetes cluster with PureLB installed and at least one ServiceGroup configured
  • A service of type LoadBalancer with an allocated external IP
  • A free Vigilmon account

Step 1: Verify your PureLB service is reachable

Before adding monitoring, confirm PureLB has allocated an IP and the service responds from outside the cluster:

# Check service has an external IP from PureLB
kubectl get svc -A --field-selector spec.type=LoadBalancer

# Example output:
# NAMESPACE    NAME       TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
# production   my-api     LoadBalancer   10.96.42.100    192.168.1.200   80:31234/TCP   10m

# Verify PureLB assigned it (check the allocator annotation)
kubectl describe svc my-api -n production | grep -i purelb
# Annotations: purelb.io/allocated-by: default
#              purelb.io/allocated-from: default

# Test reachability from outside the cluster (from your laptop or a jump host)
curl http://192.168.1.200/health
# {"status":"ok"}

If the curl fails but kubectl shows an external IP, you have a bare-metal reachability issue — ARP, BGP, or interface binding — that's exactly what Vigilmon will catch going forward.

Add a health endpoint to your application

Your application should expose a lightweight /health route:

# Flask example
@app.route('/health')
def health():
    return {'status': 'ok', 'service': 'my-api'}, 200

// Go example
http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"status":"ok"}`))
})

Step 2: Monitor the HTTP layer via the PureLB-assigned IP

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your service's external IP and health path:
    • http://192.168.1.200/health (for a local network service)
    • https://api.yourdomain.com/health (if you have a domain pointing to the PureLB IP)
  4. Set check interval: 1 minute
  5. Expected status code: 200
  6. Optional body assertion: "status":"ok"
  7. Save the monitor

Vigilmon probes from multiple geographic regions. For a bare-metal cluster on a private network, use your publicly routable IP or ensure the PureLB IP is reachable from the monitoring probes. For clusters on private networks, pair Vigilmon with a heartbeat monitor (see Step 5).

Step 3: Monitor TCP connectivity to the load-balanced port

HTTP monitoring validates the application layer. TCP monitoring validates that the port is reachable at all — catching routing failures before they cause HTTP-level errors:

  1. Go to Monitors → New Monitor
  2. Choose TCP Port
  3. Enter:
    • Host: 192.168.1.200 (the PureLB-assigned IP)
    • Port: 80 (or whichever port your service exposes)
  4. Set check interval: 1 minute
  5. Save the monitor

If Vigilmon's TCP monitor goes red while the HTTP monitor is still green — or vice versa — the diagnostics tell you exactly which layer broke. TCP red + HTTP green is impossible under normal conditions; seeing it indicates a probe routing anomaly. TCP red + HTTP red points to the PureLB allocation or announcement layer.

Monitor multiple ports if your service exposes them

# List all ports on your LoadBalancer services
kubectl get svc -A --field-selector spec.type=LoadBalancer \
  -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}: {.status.loadBalancer.ingress[0].ip} ports={.spec.ports[*].port}{"\n"}{end}'

Create one TCP monitor per critical port.

Step 4: Check PureLB component health

PureLB runs two key components you can check with kubectl:

# PureLB allocator — handles IP allocation from ServiceGroups
kubectl get pods -n purelb -l component=allocator

# PureLB lbnodeagent — runs on every node, handles ARP/BGP announcements
kubectl get pods -n purelb -l component=lbnodeagent

# Check for events related to allocation failures
kubectl get events -n purelb --sort-by='.lastTimestamp' | tail -20

# Check ServiceGroup pool utilization
kubectl get servicegroups -A
kubectl describe servicegroup default -n purelb

Set up a Vigilmon webhook alert that triggers an automated check of these components when the load balancer IP goes unreachable:

#!/bin/bash
# alert-handler.sh — called by your webhook receiver when Vigilmon fires
echo "=== PureLB allocator status ===" 
kubectl get pods -n purelb -l component=allocator

echo "=== PureLB lbnodeagent status ==="
kubectl get pods -n purelb -l component=lbnodeagent

echo "=== Recent PureLB events ==="
kubectl get events -n purelb --sort-by='.lastTimestamp' | tail -10

echo "=== ARP cache on affected node ==="
ip neigh show

Step 5: Heartbeat monitoring for batch workloads

If you run Kubernetes CronJobs that depend on PureLB-served endpoints (for example, jobs that call internal APIs on load-balanced IPs), use Vigilmon heartbeats to verify they're completing:

  1. Go to Monitors → New Monitor → Heartbeat
  2. Give it a name: batch-job-via-purelb
  3. Set the expected interval: match your CronJob schedule (e.g., 60 minutes for an hourly job)
  4. Copy the heartbeat URL

In your CronJob spec, add a final step that pings the heartbeat on success:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: data-sync
spec:
  schedule: "0 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: sync
              image: your-registry/data-sync:latest
              command:
                - /bin/sh
                - -c
                - |
                  # Call API via PureLB-assigned IP
                  ./sync --endpoint http://192.168.1.200/api/v1/sync
                  # Signal success to Vigilmon
                  curl -s https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_TOKEN
          restartPolicy: OnFailure

Vigilmon alerts if the heartbeat isn't received within the window — catching job failures, pod scheduling issues, and PureLB connectivity problems all in one signal.

Step 6: Configure alert channels

  1. Email — go to Alert Channels → Add Channel → Email, enter your ops email
  2. Webhook (Slack/PagerDuty) — go to Alert Channels → Add Channel → Webhook, paste your Slack incoming webhook or PagerDuty Events API URL

Example Vigilmon alert payload:

{
  "monitor_name": "PureLB my-api HTTP",
  "status": "down",
  "url": "http://192.168.1.200/health",
  "started_at": "2026-01-15T14:30:00Z",
  "duration_seconds": 65
}

Route this to PagerDuty for on-call paging, or post it to Slack with enriched context:

# Slack notification enriched with PureLB context
curl -X POST "$SLACK_WEBHOOK" \
  -H 'Content-type: application/json' \
  -d "{
    \"text\": \":red_circle: PureLB service down: http://192.168.1.200/health\",
    \"attachments\": [{
      \"text\": \"$(kubectl get pods -n purelb --no-headers 2>&1 | head -5)\"
    }]
  }"

Step 7: Build a status page for your bare-metal services

A public status page lets stakeholders check service health without needing kubectl access:

  1. Go to Status Pages → New Status Page
  2. Name it: Bare-Metal Kubernetes Services
  3. Add all your monitors: HTTP checks, TCP checks, heartbeats
  4. Group them by function: API Services, Database Layer, Batch Jobs
  5. Publish

Share the URL in your team wiki, runbook, and incident response docs.

Monitoring summary

| Monitor | Type | Detects | |---------|------|---------| | http://192.168.1.200/health | HTTP | App errors, pod crashes, PureLB IP routing failure | | 192.168.1.200:80 | TCP | Port unreachability, node-port forwarding failure | | 192.168.1.200:443 | TCP | TLS termination failure (if applicable) | | batch-job-via-purelb | Heartbeat | CronJob failure, PureLB connectivity loss for internal callers |

Diagnosing a PureLB outage

When Vigilmon alerts you, work through this checklist:

# 1. Is the PureLB IP still assigned?
kubectl get svc my-api -n production -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

# 2. Are PureLB pods running?
kubectl get pods -n purelb

# 3. Is the IP on any node interface?
ip addr show | grep 192.168.1.200

# 4. Is ARP resolving from outside?
arping -I eth0 192.168.1.200   # run from a host on the same subnet

# 5. BGP route present? (if using BGP mode)
birdc show route 192.168.1.200/32  # or vtysh equivalent

# 6. Application pods healthy?
kubectl get pods -n production -l app=my-api
kubectl logs -n production -l app=my-api --tail=50

The combination of Vigilmon's external signal and these kubectl commands pinpoints whether the failure is in the application, the PureLB allocation, or the underlying bare-metal network.

What's next

  • Multi-pool monitoring — if you use multiple PureLB ServiceGroups for different service classes, create separate Vigilmon monitor groups for each pool
  • Certificate monitoring — if your PureLB services terminate TLS, Vigilmon's SSL certificate expiry alerts catch cert-manager failures before certs lapse
  • Latency trending — Vigilmon's response time history shows you if bare-metal routing is introducing unexpected latency compared to your cloud-hosted baseline

Get started free at vigilmon.online — no credit card required, monitors are live in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles