tutorial

How to Monitor Amazon DocumentDB Uptime and Health with Vigilmon

Amazon DocumentDB TLS cert expiry, failover windows, read replica lag, and connection pool exhaustion from change streams can all impact your document database silently. Learn how to monitor DocumentDB externally with Vigilmon HTTP probes and heartbeat monitors for DMS migration jobs and batch aggregation pipelines.

Amazon DocumentDB is a fully managed, MongoDB-compatible document database — but "managed" does not mean "automatically monitored from your application's perspective." DocumentDB lives entirely inside your VPC, which means external uptime checkers cannot reach it directly. A cluster failover from primary to replica takes approximately 30 seconds; during that window your application's connection pool exhausts retries with timeout errors that CloudWatch surfaces only after the fact. A DMS migration job that stalls on a collection with deeply nested documents fails silently in a log no one is watching. A change stream consumer that leaks connections gradually starves every other operation sharing the pool. And because DocumentDB is not actually MongoDB — it is built on Aurora storage behind the MongoDB wire protocol — behaviour differences in aggregation pipelines and partial index support can cause application errors that look like database failures to monitors.

Vigilmon gives you external visibility into Amazon DocumentDB health through HTTP probe monitoring and heartbeat monitoring for DMS migration jobs and scheduled aggregation pipelines. This tutorial walks through both.

Why Amazon DocumentDB Monitoring Needs More Than Process Checks

AWS CloudWatch, the DocumentDB console, and VPC health dashboards tell you the cluster endpoint is reachable at the network layer. They cannot tell you:

  • Whether DocumentDB is reachable from your application servers across the VPC with a valid authenticated connection
  • Whether a failover is in progress and your primary endpoint is temporarily unavailable during the ~30-second promotion window
  • Whether the TLS certificate used to connect has expired or is approaching expiry — DocumentDB requires the AWS CA bundle (rds-combined-ca-bundle.pem) and will refuse connections without it
  • Whether a read replica your analytics tier queries has fallen behind the primary due to replication lag
  • Whether change stream consumers have exhausted the connection pool, blocking all other operations
  • Whether a DMS migration job has stalled or failed with partial data transfer
  • Whether a scheduled aggregation pipeline that powers your reporting tier has silently stopped running

These failure modes produce silent data errors or cascading timeouts without clean alerts. External monitoring through Vigilmon catches them by probing the actual connectivity and logic paths your application relies on.

Step 1: Build an Amazon DocumentDB Health Endpoint

Amazon DocumentDB is accessible only from within your VPC — port 27017 is not reachable from the public internet. Deploy a thin Lambda function or sidecar service inside the VPC that connects to DocumentDB, runs a lightweight probe, and exposes the result via API Gateway or an internal ALB.

Important: DocumentDB requires TLS for all connections. You must bundle the AWS DocumentDB CA certificate (rds-combined-ca-bundle.pem) with your Lambda deployment package or sidecar container image. Download it from the Amazon DocumentDB TLS documentation and include it at a known path in your deployment.

Node.js / Lambda Example (exposed via API Gateway)

// health/documentdb.js — deployed as a Lambda behind API Gateway (inside VPC)
const { MongoClient } = require('mongodb');
const path = require('path');

let cachedClient = null;

async function getClient() {
  if (cachedClient) return cachedClient;

  const connectionString =
    process.env.DOCDB_CONNECTION_STRING;
  // e.g. mongodb://user:pass@cluster.cluster-xxx.us-east-1.docdb.amazonaws.com:27017/
  //       ?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0

  cachedClient = new MongoClient(connectionString, {
    tls: true,
    tlsCAFile: path.join(__dirname, 'rds-combined-ca-bundle.pem'),
    serverSelectionTimeoutMS: 5000,
    connectTimeoutMS: 5000,
  });

  await cachedClient.connect();
  return cachedClient;
}

exports.handler = async (event) => {
  try {
    const client = await getClient();
    const db = client.db(process.env.DOCDB_DB_NAME || 'admin');

    // ping command — the lightest possible DocumentDB health probe
    const pingResult = await db.command({ ping: 1 });
    if (pingResult?.ok !== 1) {
      return {
        statusCode: 503,
        body: JSON.stringify({ status: 'degraded', reason: 'ping_failed', ping: pingResult }),
      };
    }

    // Optional: check replica role via isMaster (surfaces failover in progress)
    const role = event?.queryStringParameters?.role;
    if (role) {
      const isMaster = await db.command({ isMaster: 1 });
      const isWritable = isMaster?.ismaster === true;
      if (role === 'primary' && !isWritable) {
        return {
          statusCode: 503,
          body: JSON.stringify({ status: 'degraded', reason: 'not_primary', ismaster: isMaster?.ismaster }),
        };
      }
      if (role === 'replica' && isWritable) {
        return {
          statusCode: 503,
          body: JSON.stringify({ status: 'degraded', reason: 'not_replica', ismaster: isMaster?.ismaster }),
        };
      }
    }

    return {
      statusCode: 200,
      body: JSON.stringify({ status: 'ok', ping: pingResult?.ok }),
    };
  } catch (err) {
    // Reset cached client on connection error so next invocation retries
    cachedClient = null;
    return {
      statusCode: 503,
      body: JSON.stringify({ status: 'down', error: err.message }),
    };
  }
};

Python FastAPI Sidecar Example (ECS or EC2)

# health_documentdb.py — run as a sidecar container in ECS with VPC access
import os
import ssl
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from pymongo import MongoClient
from pymongo.errors import ConnectionFailure, ServerSelectionTimeoutError

app = FastAPI()

DOCDB_URI = os.environ["DOCDB_CONNECTION_STRING"]
# e.g. mongodb://user:pass@cluster.cluster-xxx.us-east-1.docdb.amazonaws.com:27017/
#       ?ssl=true&ssl_ca_certs=/app/rds-combined-ca-bundle.pem&replicaSet=rs0

CA_BUNDLE = os.environ.get("DOCDB_CA_BUNDLE", "/app/rds-combined-ca-bundle.pem")

_client: MongoClient | None = None

def get_client() -> MongoClient:
    global _client
    if _client is None:
        _client = MongoClient(
            DOCDB_URI,
            tls=True,
            tlsCAFile=CA_BUNDLE,
            serverSelectionTimeoutMS=5000,
        )
    return _client

@app.get("/health/documentdb")
async def documentdb_health(role: str | None = None):
    try:
        client = get_client()
        db = client["admin"]

        # ping — lightest DocumentDB health probe
        ping = db.command("ping")
        if ping.get("ok") != 1:
            return JSONResponse(
                status_code=503,
                content={"status": "degraded", "reason": "ping_failed", "ping": ping},
            )

        # role-specific check
        if role:
            is_master = db.command("isMaster")
            is_writable = is_master.get("ismaster", False)
            if role == "primary" and not is_writable:
                return JSONResponse(
                    status_code=503,
                    content={"status": "degraded", "reason": "not_primary"},
                )
            if role == "replica" and is_writable:
                return JSONResponse(
                    status_code=503,
                    content={"status": "degraded", "reason": "not_replica"},
                )

        return {"status": "ok", "ping": ping.get("ok")}

    except (ConnectionFailure, ServerSelectionTimeoutError) as e:
        global _client
        _client = None  # reset so next request retries
        return JSONResponse(
            status_code=503,
            content={"status": "down", "error": str(e)},
        )

Verify the endpoint before wiring up Vigilmon:

curl -i https://your-api.example.com/health/documentdb
# HTTP/1.1 200 OK
# {"status":"ok","ping":1}

Step 2: Configure Vigilmon HTTP Monitor for Amazon DocumentDB

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your health endpoint: https://your-api.example.com/health/documentdb
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: "status":"ok"
    • Response time threshold: 5000ms
  6. Under Alert channels, assign your Slack or PagerDuty channel
  7. Save the monitor

Vigilmon probes from multiple geographic regions simultaneously, requiring multi-region consensus before opening an incident. This eliminates false alarms from transient single-probe network hiccups.

Monitoring Primary and Read Replicas Separately

DocumentDB exposes a cluster endpoint (always resolves to the current primary) and a reader endpoint (load-balances across up to 15 read replicas). Create separate monitors for each role:

  • [documentdb-primary] /health/documentdb?role=primary — P1 page on failure; all writes stop and reads degrade
  • [documentdb-replica] /health/documentdb?role=replica — P2 Slack alert; analytics and reporting tiers degrade

The primary monitor should page on-call immediately — a failed primary health check during a failover means the ~30-second promotion window is actively impacting your application. The replica monitor warrants a Slack alert without immediate paging unless your application has no read-write fallback.

Use Vigilmon's status page grouping to surface both DocumentDB monitors in a single panel for on-call engineers.

Step 3: Heartbeat Monitoring for DocumentDB Migration Jobs and Batch Operations

DMS (Database Migration Service) jobs migrating collections from MongoDB (or another DocumentDB cluster) into your target DocumentDB instance are asynchronous and long-running. When a DMS task stalls due to a LOB column size mismatch, a change data capture (CDC) replication slot error, or an IAM permissions issue, it emits a CloudWatch event — but no push notification reaches your team unless you have SNS subscriptions configured. Similarly, scheduled aggregation pipelines that compute pre-rolled reporting collections can fail on DocumentDB's partial aggregation operator support without surfacing a user-visible error.

Vigilmon heartbeat monitors detect silent stalls: your job or pipeline pings Vigilmon on each successful completion. If pings stop arriving within the expected window, Vigilmon fires an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
  2. Set the name: documentdb-dms-migration (or documentdb-aggregation-pipeline)
  3. Set the expected interval to match your job schedule (e.g. 12 hours for a nightly migration sync)
  4. Set the grace period: 1 hour
  5. Save — copy the unique heartbeat URL, e.g. https://vigilmon.online/heartbeat/abc123xyz

Python Heartbeat Example for DMS Jobs and Aggregation Pipelines

# documentdb_jobs.py — runs as a Lambda, ECS task, or cron job inside the VPC
import os
import ssl
import time
import requests
from pymongo import MongoClient

DOCDB_URI = os.environ["DOCDB_CONNECTION_STRING"]
CA_BUNDLE = os.environ.get("DOCDB_CA_BUNDLE", "/app/rds-combined-ca-bundle.pem")
HEARTBEAT_URL = os.environ["VIGILMON_HEARTBEAT_URL"]

def get_client() -> MongoClient:
    return MongoClient(
        DOCDB_URI,
        tls=True,
        tlsCAFile=CA_BUNDLE,
        serverSelectionTimeoutMS=10000,
    )

def run_nightly_aggregation():
    """
    Example: roll up daily order totals into a reporting collection.
    DocumentDB supports $group, $sum, $match — but not all aggregation stages.
    Test operator support against your target DocumentDB version before deploying.
    """
    client = get_client()
    try:
        db = client[os.environ["DOCDB_DB_NAME"]]
        orders = db["orders"]
        daily_totals = db["daily_order_totals"]

        pipeline = [
            {"$match": {"status": "completed"}},
            {
                "$group": {
                    "_id": {
                        "year": {"$year": "$created_at"},
                        "month": {"$month": "$created_at"},
                        "day": {"$dayOfMonth": "$created_at"},
                    },
                    "total_revenue": {"$sum": "$amount"},
                    "order_count": {"$sum": 1},
                }
            },
            {"$sort": {"_id": -1}},
        ]

        results = list(orders.aggregate(pipeline))
        print(f"Aggregation produced {len(results)} daily buckets")

        # Upsert results into reporting collection
        for doc in results:
            daily_totals.replace_one({"_id": doc["_id"]}, doc, upsert=True)

        print("Aggregation pipeline complete")

        # Ping Vigilmon — job completed successfully
        requests.get(HEARTBEAT_URL, timeout=5)

    finally:
        client.close()


def poll_dms_task_until_complete(task_arn: str):
    """
    Poll a DMS replication task until it reaches Stopped or Failed status,
    then ping Vigilmon if successful.
    Requires boto3 and appropriate IAM role.
    """
    import boto3
    dms = boto3.client("dms", region_name=os.environ.get("AWS_REGION", "us-east-1"))

    print(f"Polling DMS task: {task_arn}")
    while True:
        response = dms.describe_replication_tasks(
            Filters=[{"Name": "replication-task-arn", "Values": [task_arn]}]
        )
        tasks = response.get("ReplicationTasks", [])
        if not tasks:
            raise RuntimeError(f"DMS task not found: {task_arn}")

        status = tasks[0]["Status"]
        print(f"DMS task status: {status}")

        if status == "stopped":
            stats = tasks[0].get("ReplicationTaskStats", {})
            if stats.get("TablesErrored", 0) > 0:
                raise RuntimeError(
                    f"DMS task completed with {stats['TablesErrored']} tables in error"
                )
            # All tables loaded successfully — ping Vigilmon
            requests.get(HEARTBEAT_URL, timeout=5)
            print("DMS migration complete — heartbeat sent")
            return

        if status in ("failed", "deleting"):
            raise RuntimeError(f"DMS task entered terminal state: {status}")

        time.sleep(60)  # poll every 60 seconds


if __name__ == "__main__":
    import sys
    if sys.argv[1] == "aggregate":
        run_nightly_aggregation()
    elif sys.argv[1] == "dms":
        poll_dms_task_until_complete(os.environ["DMS_TASK_ARN"])

If the aggregation pipeline raises an exception or the DMS poll detects failed tables, no heartbeat ping is sent. Vigilmon fires an alert after the grace period expires.

Step 4: Alert Routing and Response Time Thresholds

Configure alert routing in Vigilmon to match the severity of each DocumentDB failure mode:

| Monitor | Alert Channel | Priority | |---|---|---| | DocumentDB primary /health/documentdb?role=primary | Slack + PagerDuty | P1 | | DocumentDB replica /health/documentdb?role=replica | Slack | P2 | | Heartbeat: DMS migration job | Slack + email | P2 | | Heartbeat: aggregation pipeline | Email | P3 |

Set response time thresholds as early warnings before full availability failures:

  • Alert at 2000ms for the health endpoint — a db.command({ ping: 1 }) round-trip inside a VPC should complete in under 100ms; latency above 2 seconds indicates connection pool saturation or a network issue
  • Alert at 8000ms for any endpoint that runs a lightweight query against a collection — signals read replica lag, insufficient indexes, or change stream consumers starving the connection pool
  • Set a separate P1 threshold at 15000ms to page on-call before Vigilmon marks the monitor fully down — gives the team a head-start on failover investigation

Summary

Amazon DocumentDB failures surface in subtle ways — silent failovers, TLS cert expiry, DMS stalls, read replica lag, and change stream connection pool exhaustion — long before your users see errors. Vigilmon gives you external visibility across the full failure surface:

| Monitor Type | What It Covers | |---|---| | HTTP monitor on /health/documentdb | Authenticated ping, TLS validity, connection pool availability | | HTTP monitor per role (primary/replica) | Role-specific availability during the ~30s failover window | | Heartbeat monitor: DMS migration | Silent stalls, partial table loads, CDC replication errors | | Heartbeat monitor: aggregation pipeline | Nightly batch failures, aggregation operator incompatibilities |

Get started free at vigilmon.online — your first DocumentDB monitor is running in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles