tutorial

Monitoring AWS Aurora with Vigilmon: Writer/Reader Health, Failover Detection & Application-Layer Availability Checks

Practical guide to uptime and health monitoring for AWS Aurora — HTTP health endpoints for Aurora-connected services, writer/reader endpoint probing, failover detection, and independent availability checks with Vigilmon.

AWS Aurora delivers up to 5× the throughput of standard MySQL and 3× of PostgreSQL with built-in high availability, but your application's uptime depends on more than Aurora's infrastructure guarantees. Writer/reader endpoint confusion, failover lag, connection pool exhaustion, replica lag, and IAM authentication failures can all degrade your application while Aurora's own CloudWatch metrics remain green. Vigilmon adds an independent external layer: HTTP health checks on your Aurora-connected services that probe real database connectivity, detect replica lag, and alert on application-level failures before users notice.

This tutorial shows you how to build meaningful monitoring for Aurora-backed applications using Vigilmon.

What You'll Build

  • A health endpoint that probes real Aurora write and read availability
  • Writer and reader endpoint separation in your health checks
  • A Vigilmon HTTP monitor with appropriate timeout and assertion settings
  • Replica lag detection to catch replication falling behind
  • A failover-aware alerting strategy

Prerequisites

  • AWS account with an Aurora cluster (MySQL-compatible or PostgreSQL-compatible)
  • RDS credentials or IAM database authentication configured
  • An application connected to Aurora (any runtime)
  • A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your Aurora Service

Add a /health endpoint that performs a lightweight Aurora query to verify real connectivity against both the writer and reader endpoints.

Node.js (mysql2 — Aurora MySQL)

// routes/health.js
import mysql from 'mysql2/promise';

// Create separate pools for writer and reader
const writerPool = mysql.createPool({
  host: process.env.AURORA_WRITER_HOST,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
  connectionLimit: 2,
  connectTimeout: 5000,
});

const readerPool = mysql.createPool({
  host: process.env.AURORA_READER_HOST,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
  connectionLimit: 2,
  connectTimeout: 5000,
});

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  // Check 1: Writer endpoint — verify write path
  try {
    const [rows] = await writerPool.execute('SELECT @@aurora_server_id AS serverId, @@read_only AS readOnly');
    const latencyMs = Date.now() - start;
    checks.writer = 'ok';
    checks.writerServerId = rows[0].serverId;
    checks.writerReadOnly = rows[0].readOnly === 1;
    checks.writerLatencyMs = latencyMs;

    if (rows[0].readOnly === 1) {
      checks.writer = 'unexpectedly read-only — failover may be in progress';
      ok = false;
    }
  } catch (err) {
    checks.writer = `error: ${err.message}`;
    ok = false;
  }

  // Check 2: Reader endpoint — verify replica availability and lag
  try {
    const readerStart = Date.now();
    const [rows] = await readerPool.execute(
      'SELECT @@aurora_server_id AS serverId, (SELECT variable_value FROM performance_schema.global_status WHERE variable_name = "Aurora_Replica_Log_Buffer_Usage_Pct") AS replicaLagPct'
    );
    checks.reader = 'ok';
    checks.readerServerId = rows[0].serverId;
    checks.readerLatencyMs = Date.now() - readerStart;

    const lagPct = parseFloat(rows[0].replicaLagPct || '0');
    checks.replicaLagPct = lagPct;

    const maxLagPct = parseInt(process.env.MAX_REPLICA_LAG_PCT || '80', 10);
    if (lagPct > maxLagPct) {
      checks.reader = `replica lag high: ${lagPct}%`;
      ok = false;
    }
  } catch (err) {
    checks.reader = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'aurora-service',
    checks,
  });
}

Python (psycopg2 — Aurora PostgreSQL)

# routers/health.py
import os, time, psycopg2
from psycopg2 import pool as pg_pool
from fastapi import APIRouter

router = APIRouter()

writer_pool = pg_pool.SimpleConnectionPool(
    1, 2,
    host=os.environ['AURORA_WRITER_HOST'],
    user=os.environ['DB_USER'],
    password=os.environ['DB_PASSWORD'],
    dbname=os.environ['DB_NAME'],
    connect_timeout=5,
)

reader_pool = pg_pool.SimpleConnectionPool(
    1, 2,
    host=os.environ['AURORA_READER_HOST'],
    user=os.environ['DB_USER'],
    password=os.environ['DB_PASSWORD'],
    dbname=os.environ['DB_NAME'],
    connect_timeout=5,
)

@router.get("/health")
async def health():
    checks = {}
    ok = True
    start = time.monotonic()

    # Check 1: Writer endpoint
    conn = None
    try:
        conn = writer_pool.getconn()
        with conn.cursor() as cur:
            cur.execute("SELECT pg_is_in_recovery(), current_setting('server_version')")
            is_standby, version = cur.fetchone()
            latency_ms = int((time.monotonic() - start) * 1000)
            checks['writer'] = 'ok'
            checks['writerLatencyMs'] = latency_ms
            checks['postgresVersion'] = version

            if is_standby:
                checks['writer'] = 'unexpectedly in recovery — failover may be in progress'
                ok = False
    except Exception as e:
        checks['writer'] = f'error: {e}'
        ok = False
    finally:
        if conn:
            writer_pool.putconn(conn)

    # Check 2: Reader endpoint — replica lag
    conn = None
    try:
        reader_start = time.monotonic()
        conn = reader_pool.getconn()
        with conn.cursor() as cur:
            cur.execute("""
                SELECT pg_is_in_recovery(),
                       EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp())) AS replica_lag_seconds
            """)
            is_standby, lag_seconds = cur.fetchone()
            checks['reader'] = 'ok'
            checks['readerLatencyMs'] = int((time.monotonic() - reader_start) * 1000)
            checks['replicaLagSeconds'] = round(lag_seconds or 0, 2)

            max_lag = float(os.environ.get('MAX_REPLICA_LAG_SECONDS', '30'))
            if lag_seconds and lag_seconds > max_lag:
                checks['reader'] = f'replica lag high: {lag_seconds:.1f}s'
                ok = False
    except Exception as e:
        checks['reader'] = f'error: {e}'
        ok = False
    finally:
        if conn:
            reader_pool.putconn(conn)

    return {
        'status': 'ok' if ok else 'degraded',
        'service': 'aurora-service',
        'checks': checks,
    }

Go (pgx — Aurora PostgreSQL)

// internal/health/handler.go
package health

import (
    "context"
    "encoding/json"
    "net/http"
    "os"
    "time"

    "github.com/jackc/pgx/v5/pgxpool"
)

func Handler(writer *pgxpool.Pool, reader *pgxpool.Pool) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        checks := map[string]any{}
        ok := true
        ctx, cancel := context.WithTimeout(r.Context(), 8*time.Second)
        defer cancel()

        start := time.Now()
        var isRecovery bool
        err := writer.QueryRow(ctx, "SELECT pg_is_in_recovery()").Scan(&isRecovery)
        if err != nil {
            checks["writer"] = "error: " + err.Error()
            ok = false
        } else {
            checks["writerLatencyMs"] = time.Since(start).Milliseconds()
            if isRecovery {
                checks["writer"] = "unexpectedly in recovery"
                ok = false
            } else {
                checks["writer"] = "ok"
            }
        }

        readerStart := time.Now()
        var lagSeconds *float64
        err = reader.QueryRow(ctx,
            "SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))").Scan(&lagSeconds)
        if err != nil {
            checks["reader"] = "error: " + err.Error()
            ok = false
        } else {
            checks["readerLatencyMs"] = time.Since(readerStart).Milliseconds()
            if lagSeconds != nil && *lagSeconds > 30 {
                checks["reader"] = "replica lag high"
                checks["replicaLagSeconds"] = *lagSeconds
                ok = false
            } else {
                checks["reader"] = "ok"
            }
        }

        w.Header().Set("Content-Type", "application/json")
        if !ok {
            w.WriteHeader(http.StatusServiceUnavailable)
        }
        json.NewEncoder(w).Encode(map[string]any{
            "status":  map[bool]string{true: "ok", false: "degraded"}[ok],
            "service": "aurora-service",
            "checks":  checks,
        })
    }
}

Step 2: Configure the Vigilmon HTTP Monitor

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: your service's health endpoint (e.g. https://api.example.com/health).
  3. Check interval: 60 seconds.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. JSON assertion: path status, expected value ok.

Create a second monitor pointing to /health/writer for write-path isolation — this lets you distinguish total unavailability from read-only degradation during Aurora failover events.

Tip: Aurora failover typically completes in under 30 seconds for automatic failover, but your connection pool may cache stale DNS. Set connectTimeout to 5 seconds and ensure your pool recreates connections after errors rather than reusing failed ones.

Step 3: Failover Detection with Heartbeats

Aurora's automatic failover promotes a reader to writer without manual intervention, but your application may stall during DNS propagation. Use a Vigilmon Heartbeat monitor to detect when your application stops processing work.

// Heartbeat ping from a recurring job or health wrapper
async function runScheduledJob() {
  try {
    await processAuroraBatch();
    // Ping Vigilmon after each successful run
    await fetch(process.env.VIGILMON_HEARTBEAT_URL, { method: 'POST' });
  } catch (err) {
    console.error('Batch failed — heartbeat NOT sent:', err.message);
    // Vigilmon will alert when grace period expires
  }
}

Set the heartbeat grace period to 3× your expected job interval. If your batch runs every 60 seconds, set the grace period to 3 minutes.

Step 4: Replica Lag Alerting

Aurora replica lag signals that your reader endpoints are serving stale data. Surface this in your health endpoint and configure a separate Vigilmon monitor for /health/reader with a JSON assertion on replicaLagSeconds or replicaLagPct.

Key lag thresholds:

  • < 5 seconds: Normal for most workloads
  • 5–30 seconds: Warn — investigate write throughput or instance sizing
  • > 30 seconds: Critical — readers may serve significantly stale data

Step 5: Alerting Strategy

| Alert channel | When to use | |---|---| | Email / PagerDuty | Writer returns 503 (cluster unavailable or in failover) | | Slack webhook | Reader lag exceeds warning threshold | | Slack + Email | Writer is unexpectedly read-only (failover detected) | | PagerDuty | Heartbeat missed (application stopped processing) | | Status page | Any user-facing service backed by Aurora |

Configure alert escalation in Vigilmon: notify immediately on the first failure, re-notify after 5 minutes of continued downtime. Aurora failover is fast but DNS propagation can extend impact windows.

What Vigilmon Catches That CloudWatch Misses

| Scenario | CloudWatch | Vigilmon | |---|---|---| | Writer became read-only during failover | No application-layer check | /health/writer catches immediately | | Replica lag growing silently | AuroraReplicaLag metric exists but no HTTP check | Reader health endpoint surfaces lag | | Connection pool exhausted | No pool-level visibility | Health endpoint probe fails — 503 fires | | Application stalled after failover (DNS cache) | No end-to-end check | Heartbeat grace period expires → alert | | CloudWatch alarm SNS delivery fails | Alerts silently lost | Vigilmon is external — unaffected | | Cross-region Aurora Global Database lag | Regional metric only | External Vigilmon check from any region |

Aurora is one of the most reliable managed databases available, but application-layer failures during failover, replica lag, and connection pool issues can degrade your users' experience silently. External health checks from Vigilmon give you the independent signal you need — before your users report errors.

Start monitoring your Aurora applications in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles