Monitoring AWS RDS with Vigilmon: Multi-Engine Health Checks, Failover Detection & Application-Layer Availability

AWS RDS automates backups, patching, and failover, but managed infrastructure doesn't protect your application from connection pool exhaustion, Multi-AZ failover lag, parameter group mismatches, or storage capacity failures. CloudWatch provides engine-level metrics, but it can't tell you whether your application can actually execute a query right now. Vigilmon fills that gap: HTTP health checks on your RDS-connected services probe real database connectivity, catch failover events, and alert on application-level failures independently of AWS's own monitoring stack.

This tutorial shows you how to build reliable monitoring for RDS-backed applications using Vigilmon.

What You'll Build

• A health endpoint that probes real RDS connectivity for MySQL and PostgreSQL
• Multi-AZ failover detection via application-layer checks
• A Vigilmon HTTP monitor with timeout and assertion settings tailored to RDS
• Connection pool health reporting
• An alerting strategy for instance unavailability and storage pressure

Prerequisites

• AWS account with one or more RDS instances (MySQL, PostgreSQL, MariaDB, or SQL Server)
• Database credentials or IAM database authentication configured
• An application connected to RDS (any runtime)
• A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your RDS Service

Add a /health endpoint that runs a lightweight query to verify real database connectivity and gather instance health signals.

Node.js (pg — RDS PostgreSQL)

// routes/health.js
import { Pool } from 'pg';

const pool = new Pool({
  host: process.env.RDS_HOST,
  port: parseInt(process.env.RDS_PORT || '5432', 10),
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
  max: 2,
  connectionTimeoutMillis: 5000,
  idleTimeoutMillis: 10000,
});

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  try {
    const result = await pool.query(`
      SELECT
        version() AS pg_version,
        pg_is_in_recovery() AS is_standby,
        (SELECT count(*) FROM pg_stat_activity WHERE state = 'active') AS active_connections,
        (SELECT setting::int FROM pg_settings WHERE name = 'max_connections') AS max_connections
    `);
    const row = result.rows[0];
    const latencyMs = Date.now() - start;

    checks.db = 'ok';
    checks.latencyMs = latencyMs;
    checks.pgVersion = row.pg_version;
    checks.isStandby = row.is_standby;
    checks.activeConnections = parseInt(row.active_connections, 10);
    checks.maxConnections = parseInt(row.max_connections, 10);
    checks.connectionUtilizationPct = Math.round(
      (checks.activeConnections / checks.maxConnections) * 100
    );

    // Flag if the instance is unexpectedly in standby/recovery
    if (row.is_standby) {
      checks.db = 'in recovery — Multi-AZ failover may be in progress';
      ok = false;
    }

    // Warn if connection utilization is high
    const maxUtilPct = parseInt(process.env.MAX_CONN_UTILIZATION_PCT || '80', 10);
    if (checks.connectionUtilizationPct > maxUtilPct) {
      checks.connections = `high utilization: ${checks.connectionUtilizationPct}%`;
      ok = false;
    } else {
      checks.connections = 'ok';
    }

    // Flag slow queries
    if (latencyMs > parseInt(process.env.QUERY_WARN_MS || '2000', 10)) {
      checks.db = 'slow';
      ok = false;
    }
  } catch (err) {
    checks.db = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'rds-service',
    checks,
  });
}

Node.js (mysql2 — RDS MySQL / MariaDB)

// routes/health.js
import mysql from 'mysql2/promise';

const pool = mysql.createPool({
  host: process.env.RDS_HOST,
  port: parseInt(process.env.RDS_PORT || '3306', 10),
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
  connectionLimit: 2,
  connectTimeout: 5000,
});

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  try {
    const [rows] = await pool.execute(`
      SELECT
        @@version AS mysqlVersion,
        @@read_only AS readOnly,
        @@max_connections AS maxConnections
    `);
    const row = rows[0];
    const latencyMs = Date.now() - start;

    checks.db = 'ok';
    checks.latencyMs = latencyMs;
    checks.mysqlVersion = row.mysqlVersion;
    checks.maxConnections = row.maxConnections;

    if (row.readOnly === 1) {
      checks.db = 'read-only — Multi-AZ failover may be in progress';
      ok = false;
    }

    if (latencyMs > parseInt(process.env.QUERY_WARN_MS || '2000', 10)) {
      checks.db = 'slow';
      ok = false;
    }
  } catch (err) {
    checks.db = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'rds-service',
    checks,
  });
}

Python (psycopg2 — RDS PostgreSQL)

# routers/health.py
import os, time, psycopg2
from psycopg2 import pool as pg_pool
from fastapi import APIRouter

router = APIRouter()

db_pool = pg_pool.SimpleConnectionPool(
    1, 2,
    host=os.environ['RDS_HOST'],
    port=int(os.environ.get('RDS_PORT', '5432')),
    user=os.environ['DB_USER'],
    password=os.environ['DB_PASSWORD'],
    dbname=os.environ['DB_NAME'],
    connect_timeout=5,
)

@router.get("/health")
async def health():
    checks = {}
    ok = True
    start = time.monotonic()
    conn = None

    try:
        conn = db_pool.getconn()
        with conn.cursor() as cur:
            cur.execute("""
                SELECT
                    version(),
                    pg_is_in_recovery(),
                    (SELECT count(*) FROM pg_stat_activity WHERE state = 'active'),
                    (SELECT setting::int FROM pg_settings WHERE name = 'max_connections')
            """)
            version, is_standby, active_conns, max_conns = cur.fetchone()
            latency_ms = int((time.monotonic() - start) * 1000)

            checks['db'] = 'ok'
            checks['latencyMs'] = latency_ms
            checks['pgVersion'] = version
            checks['activeConnections'] = active_conns
            checks['maxConnections'] = max_conns
            utilization_pct = round(active_conns / max_conns * 100)
            checks['connectionUtilizationPct'] = utilization_pct

            if is_standby:
                checks['db'] = 'in recovery — Multi-AZ failover may be in progress'
                ok = False

            max_util = int(os.environ.get('MAX_CONN_UTILIZATION_PCT', '80'))
            if utilization_pct > max_util:
                checks['connections'] = f'high utilization: {utilization_pct}%'
                ok = False
            else:
                checks['connections'] = 'ok'

    except Exception as e:
        checks['db'] = f'error: {e}'
        ok = False
    finally:
        if conn:
            db_pool.putconn(conn)

    return {
        'status': 'ok' if ok else 'degraded',
        'service': 'rds-service',
        'checks': checks,
    }

Step 2: Configure the Vigilmon HTTP Monitor

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: your service's health endpoint (e.g. https://api.example.com/health).
3. Check interval: 60 seconds.
4. Response timeout: 10 seconds.
5. Expected status: 200.
6. JSON assertion: path status, expected value ok.

Tip: RDS Multi-AZ failover typically completes in 60–120 seconds. Set your Vigilmon alert to trigger after 2 consecutive failures (2 minutes) to avoid false positives during the failover window while still catching genuine outages promptly.

Step 3: Storage Pressure Monitoring

RDS instances can become unavailable when storage is full. Surface storage metrics in your health endpoint by querying the database directly:

// PostgreSQL — check database size vs. instance storage
export async function storageHealthHandler(req, res) {
  const checks = {};
  let ok = true;

  try {
    const result = await pool.query(`
      SELECT
        pg_database_size(current_database()) AS db_size_bytes,
        pg_size_pretty(pg_database_size(current_database())) AS db_size_human
    `);
    const { db_size_bytes, db_size_human } = result.rows[0];
    checks.dbSizeHuman = db_size_human;
    checks.dbSizeBytes = parseInt(db_size_bytes, 10);
    checks.storage = 'ok';
  } catch (err) {
    checks.storage = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({ status: ok ? 'ok' : 'degraded', checks });
}

Configure this as a /health/storage monitor with a lower check interval (every 5 minutes) since storage changes slowly.

Step 4: Heartbeat for Batch Jobs

RDS-backed batch processors and migration jobs run outside request/response cycles and are invisible to HTTP monitors. Use Vigilmon's Heartbeat monitor to detect when processing stops.

# batch/processor.py
import os, requests

def process_rds_batch():
    with get_db_connection() as conn:
        rows = fetch_pending_records(conn)
        for row in rows:
            process_record(conn, row)

    # Ping Vigilmon only after successful processing
    requests.post(os.environ['VIGILMON_HEARTBEAT_URL'], timeout=5)
    print(f'Processed {len(rows)} records — heartbeat sent')

Set the heartbeat grace period to 3× your expected batch interval. A missed heartbeat means the batch failed, the RDS instance is unreachable, or the job scheduler itself stopped.

Step 5: Alerting Strategy

| Alert channel | When to use | |---|---| | Email / PagerDuty | Service returns 503 (instance unavailable or in failover) | | Slack webhook | Connection utilization exceeds 80% | | Slack + Email | Read-only flag detected (failover in progress) | | PagerDuty | Heartbeat missed (batch job stopped) | | Status page | Any user-facing service backed by RDS |

Multi-AZ failover alert tuning: configure your Vigilmon monitor to require 2 consecutive failures before alerting. This prevents false alarms during the 60–120 second automatic failover window while ensuring genuine outages trigger promptly.

What Vigilmon Catches That CloudWatch Misses

| Scenario | CloudWatch | Vigilmon | |---|---|---| | Application can't connect (wrong password after rotation) | No application-layer check | HTTP monitor catches 503 immediately | | Instance in failover — DNS not yet propagated | MultiAZ failover event logged | Health endpoint catches read-only flag in real time | | Connection pool saturated — new requests timeout | No pool-level visibility | Health endpoint reports utilization — alert fires | | Batch processor stopped silently | No end-to-end check | Heartbeat grace period expires → alert | | CloudWatch alarm SNS delivery fails | Alerts silently lost | Vigilmon is external — unaffected | | RDS Proxy routing to wrong instance | No proxy-level check | Application health probe catches behavior change |

RDS automates much of database operations, but your application is still responsible for handling failover gracefully, managing connection pools, and detecting degraded connectivity. External health checks from Vigilmon give you the independent, application-layer signal that CloudWatch can't provide.

Start monitoring your RDS applications in under 5 minutes — register free at vigilmon.online.