tutorial

Monitoring Your Database Infrastructure with Vigilmon

Database outages are application outages. When your PostgreSQL primary goes down, your web app returns 500 errors. When Redis becomes unreachable, your sessi...

Database outages are application outages. When your PostgreSQL primary goes down, your web app returns 500 errors. When Redis becomes unreachable, your session store fails and users are logged out. When a read replica falls behind, your reporting queries time out or return stale data.

Most uptime monitoring tools focus on HTTP endpoints — which is right for user-facing services — but database infrastructure requires a different approach. This guide covers how to monitor database infrastructure with Vigilmon using TCP port checks, HTTP health endpoints for database proxies, backup job heartbeats, and read replica monitoring.


TCP Port Monitoring: The Foundation

Database services expose TCP ports, not HTTP endpoints. Vigilmon's TCP monitoring checks that a specific host and port are accepting connections — which is the right level of verification for a database service.

Common database ports to monitor:

| Database | Default Port | What TCP monitoring confirms | |---|---|---| | PostgreSQL | 5432 | Primary is running and accepting connections | | MySQL / MariaDB | 3306 | Database server is up and connectable | | Redis | 6379 | Cache/session store is reachable | | MongoDB | 27017 | Document store is accepting connections | | Elasticsearch | 9200 | Search cluster is up (HTTP port) | | Cassandra | 9042 | Cluster is accepting CQL connections |

To add a TCP monitor in Vigilmon:

  1. Click Add MonitorTCP
  2. Enter your database host and port: db.internal.yourdomain.com:5432
  3. Set check interval (1–5 minutes depending on criticality)
  4. Configure alert routing

TCP monitoring catches the scenario where your application health endpoint shows green — because the app server is running — while the database has gone away and every query is timing out. The app layer can report healthy while the database layer is down; TCP monitoring checks the database layer directly.


HTTP Health Endpoints for Database Proxies

Modern database infrastructure commonly uses proxies and connection poolers that do sit behind HTTP endpoints. These are excellent targets for HTTP monitoring:

PgBouncer

PgBouncer exposes an admin interface but not a standard HTTP health endpoint by default. Many teams add a thin health check via a sidecar or configure a dedicated port. If you expose a health endpoint, monitor it:

https://db-proxy.internal.yourdomain.com:8080/health

Expected response: HTTP 200 with a body indicating connected pool status.

ProxySQL

ProxySQL exposes a web interface on port 6080 by default:

http://db-proxy.internal.yourdomain.com:6080/

For production monitoring, expose a dedicated health endpoint that returns 200 when ProxySQL has active backends and 503 when it doesn't. Monitor that endpoint at 1-minute intervals.

RDS Proxy / Cloud Database Proxies

AWS RDS Proxy, Google Cloud SQL Proxy, and similar managed proxies are typically accessed via a hostname that routes through the managed endpoint. Monitor the proxy endpoint's TCP port rather than the underlying instance — the proxy endpoint is what your application connects to, so that's the reachability that matters.

PgBouncer Stats via TCP

Even without an HTTP health endpoint, PgBouncer listens on its configured port. A TCP check on port 6432 (PgBouncer default) confirms the proxy is running and accepting connections — useful as a quick-fail indicator before you build a full HTTP health endpoint.


Backup Job Heartbeat Monitoring

Database backups failing silently is one of the most dangerous infrastructure failure modes: you don't discover the gap until you need a restore, which is exactly when missing backups are most catastrophic.

Heartbeat monitoring closes this gap. Configure your backup job to ping a Vigilmon heartbeat URL on successful completion:

#!/bin/bash
# backup-postgres.sh

set -e

BACKUP_FILE="backup_$(date +%Y%m%d_%H%M%S).sql.gz"

pg_dump -h $DB_HOST -U $DB_USER $DB_NAME | gzip > /backups/$BACKUP_FILE

# Upload to S3
aws s3 cp /backups/$BACKUP_FILE s3://your-backup-bucket/postgres/

# Signal successful completion to Vigilmon
curl -fsS -m 10 https://vigilmon.online/ping/your-heartbeat-id

echo "Backup completed: $BACKUP_FILE"

If the backup script fails at any point — pg_dump returns an error, the S3 upload fails, the network is unavailable — the curl command never runs, the heartbeat ping doesn't arrive, and Vigilmon alerts you.

Set your heartbeat window based on your typical backup duration:

  • Nightly full backup that takes ~15 minutes: set window to 25–30 minutes
  • Weekly backup that takes 2 hours: set window to 3 hours
  • Point-in-time recovery WAL archiving: set window to 2–3× your WAL archive interval

The general rule: set the window at 150–200% of typical job duration. This absorbs occasional slowdowns — large transaction logs, slower I/O during peak hours — without generating false positives, while still catching genuine failures within a reasonable detection window.


Read Replica Monitoring

Read replicas serve a different purpose than primary monitoring: they protect your reporting queries, analytics, and any read-heavy workloads that your application explicitly routes to replicas.

TCP Monitoring for Read Replicas

Monitor each read replica's TCP port independently from your primary:

  • db-primary.internal:5432 — primary instance
  • db-replica-1.internal:5432 — first read replica
  • db-replica-2.internal:5432 — second read replica

When a replica goes down, queries routed to it fail. Depending on your ORM or connection pooling configuration, your application may or may not automatically fail over to the primary or another replica. Monitoring each replica lets you know which nodes are healthy before the application layer discovers a problem.

Replication Lag via Health Endpoints

TCP monitoring confirms a replica is running; it doesn't confirm it's current. A replica that's running but 2 hours behind is unhealthy for most use cases.

Build a lightweight health endpoint on your database proxy or application server that checks replication lag:

# /health/db-replica endpoint
import psycopg2

def check_replica_lag():
    conn = psycopg2.connect(host="db-replica-1.internal", ...)
    cur = conn.cursor()
    cur.execute("""
        SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))
        AS lag_seconds
    """)
    lag = cur.fetchone()[0]
    if lag > 300:  # 5 minutes
        return 503, {"status": "lagging", "lag_seconds": lag}
    return 200, {"status": "ok", "lag_seconds": lag}

Monitor this endpoint with Vigilmon HTTP checks. When replication lag exceeds your threshold and the endpoint returns 503, Vigilmon alerts your team — before application queries start returning stale data.


Recommended Database Monitor Setup

| Monitor | Type | Target | Interval | Alert | |---|---|---|---|---| | PostgreSQL primary | TCP | db-primary:5432 | 1 min | Slack + PagerDuty | | MySQL primary | TCP | db-primary:3306 | 1 min | Slack + PagerDuty | | Redis cache | TCP | cache.internal:6379 | 1 min | Slack + PagerDuty | | PgBouncer proxy | TCP | db-proxy:6432 | 1 min | Slack | | Read replica 1 | TCP | replica-1:5432 | 5 min | Slack | | Read replica 2 | TCP | replica-2:5432 | 5 min | Slack | | Replica lag check | HTTP | /health/db-replica | 5 min | Slack | | Nightly backup job | Heartbeat | (per run) | — | Slack + PagerDuty | | Daily reconciliation job | Heartbeat | (per run) | — | Slack |

Start with the primary database TCP checks and the backup heartbeat — these catch the two most critical failure modes. Add replica monitoring as your read traffic grows.


Multi-Region Consensus for Database Endpoints

One nuance with internal database TCP monitoring: if your probe originates from inside your VPC or private network, it shares the same failure domain as the database. A network partition that isolates your database from your application will also isolate the probe from the database — both stop seeing it simultaneously, and the alert fires correctly.

Vigilmon's external probes are useful for database proxies and any database access points that are reachable externally (typically via private VPN or via a public-facing replica endpoint). For strictly internal database ports not exposed outside your network, supplement Vigilmon with an internal health check that monitors the database from inside the network.


Getting Started

Database monitoring with Vigilmon requires only the TCP monitoring feature and heartbeat monitoring — both available on the free tier. Start with your primary database TCP port, your most critical caching service, and your nightly backup heartbeat.

From there, add read replica monitoring and health endpoints for proxies as your infrastructure complexity grows.

Start monitoring at vigilmon.online — 5 monitors free, 1-minute check intervals, multi-region consensus, no credit card required.


Tags: #database #monitoring #postgresql #mysql #redis #devops #uptime #vigilmon #sre #infrastructure

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →