Watchtower Monitoring with Vigilmon: Heartbeats, HTTP API Checks & Update Failure Alerts

Watchtower ran its scheduled update check last Tuesday. Then it didn't run Wednesday. Or Thursday. Or Friday. Your containers were running stale images with unpatched CVEs for four days before you noticed — not because anything crashed, but because no one was watching whether Watchtower itself was actually running its job.

Watchtower is a Docker container that automatically updates your running containers to the latest image versions. It's the set-and-forget solution for keeping self-hosted services current. But Watchtower is itself just another Docker container: it can exit silently, get stuck, lose its Docker socket connection, or never start its scheduled job after a host reboot. When that happens, your containers drift further behind with every passing day.

Vigilmon gives you the external monitoring that catches Watchtower failures — whether that's a crashed container, a skipped update run, or a broken Docker socket connection.

What You'll Build

• A Vigilmon heartbeat monitor that confirms Watchtower runs its scheduled check
• An HTTP API health check (for setups with Watchtower's HTTP API enabled)
• Alert channels to notify your team when update runs stop

Prerequisites

• Watchtower running as a Docker container (Watchtower v1.7.x or later recommended)
• A free account at vigilmon.online

Step 1: Heartbeat Monitor for Scheduled Update Runs

The most important thing to monitor about Watchtower is whether it's actually running its scheduled update check. Watchtower doesn't have a built-in health dashboard or status API that reflects whether the cron-style schedule is executing — it just runs, checks for updates, and exits or waits for the next cycle.

Vigilmon's heartbeat monitor is the right tool here. You configure Watchtower to send an HTTP ping to Vigilmon after each update run, and Vigilmon alerts you if a ping doesn't arrive within the expected window.

Set Up the Heartbeat Monitor

1. Log in to Vigilmon and click New Monitor → Heartbeat.
2. Set the Name to Watchtower Update Run.
3. Set the Expected interval to match your Watchtower schedule. If Watchtower runs every 24 hours, set interval to 24 hours with a 2 hour grace period.
4. Save the monitor and copy the Heartbeat URL (looks like https://vigilmon.online/heartbeat/<your-token>).

Configure Watchtower to Ping Vigilmon

Watchtower supports lifecycle hooks via the WATCHTOWER_NOTIFICATIONS_HOSTNAME and HTTP notification mechanism. The simplest approach uses the --http-api-update flag combined with a post-check script, or Watchtower's built-in notification webhook support.

Option A: Use Watchtower's webhook notifications

Watchtower supports generic webhook notifications that fire after each update cycle. Configure it in your docker-compose.yml:

services:
  watchtower:
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WATCHTOWER_SCHEDULE=0 0 4 * * *  # Run at 4 AM daily
      - WATCHTOWER_NOTIFICATIONS=http
      - WATCHTOWER_NOTIFICATION_HTTP_URL=https://vigilmon.online/heartbeat/<your-token>
      - WATCHTOWER_NOTIFICATION_HTTP_HEADERS=Content-Type:application/json
    restart: unless-stopped

Watchtower will POST a JSON payload to the heartbeat URL after each update run. Vigilmon accepts any non-error HTTP POST to the heartbeat URL as a valid ping.

Option B: Wrap Watchtower with a cron job

If you prefer to keep Watchtower's notification settings simple, run it on-demand via a cron job that pings Vigilmon after completion:

# Add to crontab: run Watchtower and ping Vigilmon on success
0 4 * * * docker run --rm -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once && curl -fsS https://vigilmon.online/heartbeat/<your-token> > /dev/null

Replace <your-token> with your actual Vigilmon heartbeat token. The --run-once flag makes Watchtower check for updates and exit, and the curl ping only fires if Watchtower exits cleanly (no &&-breaking errors).

Now if Watchtower's scheduled run fails, crashes, or simply stops being scheduled (e.g., after a host reboot that didn't restore the cron job), Vigilmon will alert you after the expected interval plus grace period passes.

Step 2: HTTP API Health Check (Optional)

Watchtower optionally exposes an HTTP API on port 8080 when launched with the --http-api-update and --http-api-token flags. This API allows programmatic triggering of update checks and can be used as a health signal.

Enable the API in your Docker Compose configuration:

services:
  watchtower:
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WATCHTOWER_HTTP_API_UPDATE=true
      - WATCHTOWER_HTTP_API_TOKEN=my-secret-token
      - WATCHTOWER_HTTP_API_PERIODIC_POLLS=true
    ports:
      - "8080:8080"
    restart: unless-stopped

Test the API is running:

curl -H "Authorization: Bearer my-secret-token" \
  http://your-host:8080/v1/update

A 200 response confirms Watchtower's API is alive. Note: this endpoint triggers an update check — use the metrics endpoint (/v1/metrics) for a read-only health signal if it's available in your version.

Add an HTTP monitor in Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to http://your-watchtower-host:8080/v1/update.
3. Set Method to HEAD or GET depending on your needs.
4. Add a request header: Authorization: Bearer my-secret-token.
5. Set Expected status code to 200.
6. Set Check interval to 5 minutes.
7. Save.

Security note: Only expose this port if your host's firewall restricts access to trusted IPs. Do not expose the Watchtower API to the public internet without additional authentication layers.

Step 3: Docker Socket Connectivity Alert

Watchtower communicates with the Docker daemon via the Unix socket at /var/run/docker.sock. If this socket mount is missing, the Docker daemon restarts in an incompatible mode, or permissions change, Watchtower will exit immediately with an error — often without any visible crash in monitoring dashboards.

The heartbeat monitor in Step 1 is your primary signal here: if Watchtower can't connect to the Docker socket, it exits without completing its run, and the heartbeat ping never fires.

For additional coverage, you can run a lightweight Docker health check alongside Watchtower that verifies Docker itself is responsive:

# Check that Docker daemon is healthy from within the host
docker info --format '{{.ServerVersion}}'

Add this as a cron job that pings a second Vigilmon heartbeat endpoint if Docker responds correctly:

# /etc/cron.d/docker-health
*/5 * * * * root docker info > /dev/null 2>&1 && curl -fsS https://vigilmon.online/heartbeat/<docker-health-token> > /dev/null

Create a second heartbeat monitor in Vigilmon named Docker Socket Health with a 10-minute interval and 3-minute grace period.

Step 4: Alert Channels

Go to Notifications → New Channel in Vigilmon and configure:

• Email — direct alerts to your infrastructure team's shared inbox
• Webhook — post to a Slack or Discord channel your team checks daily

A missed heartbeat alert looks like:

🔴 HEARTBEAT MISSED: Watchtower Update Run
Last ping: 26 hours ago (expected every 24 hours)
Monitor: Watchtower Update Run

When Watchtower runs again and the heartbeat resumes:

✅ RECOVERED: Watchtower Update Run
Gap: 26 hours (1 missed expected ping)

Step 5: Status Page

Go to Status Pages → New Status Page in Vigilmon. Add your Watchtower monitors and set visibility to Private (this is infrastructure monitoring, not a public-facing status page).

Share the private status URL with your operations team so they have a single place to check whether scheduled maintenance tasks are running across your fleet.

What You've Built

| Scenario | How Vigilmon catches it | |---|---| | Watchtower container crashes | Heartbeat misses expected update-run ping | | Docker socket disconnected | Watchtower exits, heartbeat ping never fires | | Host reboot clears scheduled cron | Heartbeat gap detected after 24h+ | | Watchtower HTTP API down | HTTP monitor detects non-200 | | Docker daemon unhealthy | Docker socket heartbeat misses ping | | Update run completes but notifications fail | Webhook notification channel catches it |

Watchtower is one of those tools that works quietly in the background for months — until it doesn't. A single missed reboot recovery, a changed Docker socket path, or a permissions error can silently stop all automatic updates across your entire container fleet. External heartbeat monitoring is the only way to know it's still doing its job.

Start monitoring your Watchtower instance today — register free at vigilmon.online.