Monitoring Netavark with Vigilmon

Netavark is the default network stack for Podman 4+, replacing the older CNI-based setup with a purpose-built Rust binary that manages bridge networks, port forwarding, and container DNS. It runs silently in the background — which means when it breaks, you often find out through application errors rather than explicit network failure messages. Vigilmon gives you a monitoring layer on top of Netavark so container network failures surface as immediate alerts rather than slow-burning support tickets.

What You'll Set Up

• HTTP uptime monitors for containerized services running behind Netavark networks
• TCP port monitors for container port mappings
• Cron heartbeat monitors for container-to-container connectivity checks
• Aardvark-dns health verification via a custom health endpoint
• Alert channels for container network failures

Prerequisites

• Podman 4.0+ with Netavark as the active network backend
• At least one container network with running containers
• A free Vigilmon account

Verify Netavark is your active backend:

podman info --format '{{.Host.NetworkBackend}}'
# Should output: netavark

Step 1: Monitor Container HTTP Services

The most direct way to monitor containers using Netavark is to watch the exposed HTTP endpoints. When Netavark's port forwarding rules are misconfigured or the bridge interface fails, HTTP monitors catch it immediately.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter the host URL with the mapped port: http://your-host:8080/health.
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

For each containerized service with a port mapping, add a dedicated monitor. If you have a container exposing port 8080:

podman run -d -p 8080:8080 --name myapp myapp:latest

Vigilmon probes http://host:8080/health every minute. If Netavark's iptables rules are dropped or the bridge interface goes down, the monitor fails before your application users notice.

Step 2: Add Health Endpoints to Your Containers

Containers should expose a lightweight /health route that returns a meaningful status. Here are examples for common runtimes:

Node.js

app.get('/health', (req, res) => {
  res.json({ status: 'ok', network: 'netavark', uptime: process.uptime() });
});

Python / FastAPI

@app.get("/health")
async def health():
    return {"status": "ok"}

Go

http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.Write([]byte(`{"status":"ok"}`))
})

Build health checks into your container images as a default, not an afterthought. Netavark failures that break port forwarding will immediately fail these probes, giving you a clear signal.

Step 3: TCP Port Monitors for Non-HTTP Services

Not all containers speak HTTP. Databases, message brokers, and custom TCP services still need coverage. Add a TCP monitor for each exposed port:

1. Click Add Monitor → TCP Port.
2. Enter your host IP or hostname.
3. Set Port to the host-side mapped port (e.g., 5432 for PostgreSQL, 6379 for Redis).
4. Set Check interval to 1 minute.
5. Click Save.

# Example: PostgreSQL container with Netavark
podman run -d -p 5432:5432 --name pg postgres:16

# Example: Redis container
podman run -d -p 6379:6379 --name redis redis:7

Add Vigilmon TCP monitors for ports 5432 and 6379. When Netavark's port forwarding table is corrupted or iptables rules are flushed, the TCP monitor detects the failure within one check interval.

Step 4: Heartbeat Monitoring for Container-to-Container Connectivity

Netavark's bridge networks provide internal connectivity between containers on the same host. This internal traffic never hits the host port mapping layer — so HTTP monitors on external ports won't catch internal network failures.

Add a heartbeat monitor for internal connectivity:

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set the expected ping interval to 5 minutes.
3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/YOUR_TOKEN.

Create a connectivity checker container that tests internal DNS and pings:

#!/bin/bash
# container-net-check.sh — runs on the host via cron

VIGILMON_URL="https://vigilmon.online/heartbeat/YOUR_TOKEN"
NETWORK="myapp-network"
TARGET_CONTAINER="myapp-backend"

# Check that containers can resolve each other via Aardvark DNS
RESOLVED=$(podman run --rm --network "$NETWORK" alpine \
  nslookup "$TARGET_CONTAINER" 2>&1)

if echo "$RESOLVED" | grep -q "Address:"; then
  curl -sf "$VIGILMON_URL" > /dev/null
else
  echo "$(date): DNS resolution failed for $TARGET_CONTAINER" >> /var/log/netavark-check.log
fi

Schedule every 5 minutes:

*/5 * * * * /usr/local/bin/container-net-check.sh

Aardvark-dns (Netavark's companion DNS server) resolves container names inside Podman networks. If Aardvark-dns fails — which can happen after host reboots or Podman upgrades — inter-container service discovery breaks while host-side port mappings remain intact. This heartbeat catches that specific failure mode.

Step 5: Monitor Podman Network Status via Sidecar

Build a lightweight monitoring sidecar that inspects Netavark network health and exposes it over HTTP:

#!/usr/bin/env python3
# netavark_health.py
from http.server import HTTPServer, BaseHTTPRequestHandler
import subprocess
import json

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != '/health':
            self.send_response(404)
            self.end_headers()
            return

        try:
            result = subprocess.run(
                ['podman', 'network', 'ls', '--format', 'json'],
                capture_output=True, text=True, timeout=10
            )
            networks = json.loads(result.stdout)
            active = [n for n in networks if n.get('Name') != 'podman']

            body = json.dumps({
                'status': 'ok',
                'networks': len(active),
                'backend': 'netavark'
            }).encode()
            self.send_response(200)
        except Exception as e:
            body = json.dumps({'status': 'error', 'message': str(e)}).encode()
            self.send_response(503)

        self.send_header('Content-Type', 'application/json')
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, format, *args):
        pass

if __name__ == '__main__':
    HTTPServer(('127.0.0.1', 9101), Handler).serve_forever()

Run as a systemd service and add a Vigilmon HTTP monitor for http://localhost:9101/health. This gives you a dedicated Netavark health signal separate from individual container health.

Step 6: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add your preferred channel (Slack, email, PagerDuty, webhook).
2. Set Consecutive failures before alert to 2 for HTTP and TCP monitors — container restarts can cause brief probe failures during the restart window.
3. Leave heartbeat monitors at 1 missed interval — a missed DNS heartbeat means at least 5 minutes of broken name resolution.

Group your monitors by Podman network in Vigilmon's labels for easier triage when a specific network's bridge interface fails.

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP | Container port mapping (host:port/health) | Port forwarding failure, container crash | | TCP | Exposed database/broker ports | Netavark iptables rule loss | | Cron heartbeat | Aardvark-dns container resolution script | Internal DNS failure, bridge network down | | HTTP sidecar | Podman network list endpoint | Netavark daemon crash |

Netavark handles Podman's networking silently — which is great for operations but dangerous for visibility. With Vigilmon monitoring exposed ports, internal connectivity, and DNS resolution, you get comprehensive coverage of every layer where Netavark can fail, long before your application logs start showing connection refused errors.