tutorial

Monitoring Azure Cosmos DB with Vigilmon: Account Health, RU/s Throttling & Application-Layer Availability Checks

Practical guide to uptime and health monitoring for Azure Cosmos DB — HTTP health endpoints for Cosmos DB-connected services, request unit throttling detection, latency-aware health checks, and independent availability monitoring with Vigilmon.

Azure Cosmos DB promises 99.999% availability with single-digit millisecond latency, but your application's experience depends on more than the database SLA. Request unit (RU/s) exhaustion, partition key hot-spotting, stale SDK connections, and RBAC misconfigurations all degrade your application silently while Cosmos DB's own availability metrics stay green. Azure Monitor can alert on server-side RU consumption and throttling rates, but it can't tell you whether your application can actually execute a query. Vigilmon adds an independent external layer: HTTP health checks on your Cosmos DB-connected services that probe real connectivity, detect 429 throttling, and alert on application-level failures.

This tutorial shows you how to build meaningful monitoring for Azure Cosmos DB-backed applications using Vigilmon.

What You'll Build

  • A health endpoint that probes real Cosmos DB read availability and checks for throttling
  • A Vigilmon HTTP monitor with appropriate timeout and assertion settings
  • A heartbeat pattern for Cosmos DB-backed batch jobs and change feed processors
  • An RU throttling-aware health check that surfaces capacity issues early
  • An alerting strategy for Cosmos DB unavailability and throughput exhaustion

Prerequisites

  • Azure subscription with a Cosmos DB account (any API — Core SQL, MongoDB, Cassandra)
  • Managed Identity or connection string with Cosmos DB Built-in Data Reader role
  • An application connected to Cosmos DB (Node.js, Python, Go, or Java)
  • A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your Cosmos DB Service

Add a /health endpoint that executes a lightweight Cosmos DB read to verify real connectivity, SDK health, and that RU budget is not exhausted.

Node.js (@azure/cosmos)

// routes/health.js
import { CosmosClient } from '@azure/cosmos';

const cosmos = new CosmosClient({
  endpoint: process.env.COSMOS_ENDPOINT,
  key: process.env.COSMOS_KEY,
  connectionPolicy: {
    requestTimeout: 5000,
    retryOptions: { maxRetryAttemptCount: 0 }, // Fail fast in health checks
  },
});

const container = cosmos
  .database(process.env.COSMOS_DATABASE_ID)
  .container(process.env.COSMOS_CONTAINER_ID);

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  // Check 1: Read account properties — verifies endpoint reachability and auth
  try {
    await cosmos.getDatabaseAccount();
    checks.account = 'ok';
  } catch (err) {
    checks.account = `error: ${err.code || err.message}`;
    ok = false;
  }

  // Check 2: Execute a lightweight point read for latency and RU health
  if (ok && process.env.COSMOS_PROBE_ID && process.env.COSMOS_PROBE_PARTITION_KEY) {
    try {
      const { resource, requestCharge, statusCode } = await container.item(
        process.env.COSMOS_PROBE_ID,
        process.env.COSMOS_PROBE_PARTITION_KEY
      ).read();

      const latencyMs = Date.now() - start;
      checks.read = 'ok';
      checks.latencyMs = latencyMs;
      checks.requestChargeRU = requestCharge;

      const warnMs = parseInt(process.env.COSMOS_LATENCY_WARN_MS || '1000', 10);
      if (latencyMs > warnMs) {
        checks.read = 'slow';
        ok = false;
      }
    } catch (err) {
      if (err.code === 429) {
        checks.read = 'throttled (429 — RU/s exhausted)';
        ok = false;
      } else if (err.code === 404) {
        // 404 means the probe item was deleted — connectivity is fine
        checks.read = 'ok (probe item not found — consider recreating)';
      } else {
        checks.read = `error: ${err.code} - ${err.message}`;
        ok = false;
      }
    }
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'cosmos-service',
    checks,
  });
}

Python (azure-cosmos)

# routers/health.py
import os, time
from azure.cosmos import CosmosClient, exceptions
from fastapi import APIRouter

router = APIRouter()

_client = CosmosClient(
    url=os.environ['COSMOS_ENDPOINT'],
    credential=os.environ['COSMOS_KEY'],
)
_container = _client.get_database_client(os.environ['COSMOS_DATABASE_ID'])\
                    .get_container_client(os.environ['COSMOS_CONTAINER_ID'])

@router.get("/health")
async def health():
    checks = {}
    ok = True
    start = time.monotonic()

    try:
        list(_client.list_databases(max_item_count=1))
        checks['account'] = 'ok'
    except exceptions.CosmosHttpResponseError as e:
        checks['account'] = f'error: {e.status_code} {e.message}'
        ok = False

    if ok and os.environ.get('COSMOS_PROBE_ID'):
        try:
            item = _container.read_item(
                item=os.environ['COSMOS_PROBE_ID'],
                partition_key=os.environ['COSMOS_PROBE_PARTITION_KEY'],
            )
            latency_ms = int((time.monotonic() - start) * 1000)
            checks['read'] = 'ok'
            checks['latencyMs'] = latency_ms
        except exceptions.CosmosHttpResponseError as e:
            if e.status_code == 429:
                checks['read'] = 'throttled (429 — RU/s exhausted)'
                ok = False
            elif e.status_code == 404:
                checks['read'] = 'ok (probe item not found)'
            else:
                checks['read'] = f'error: {e.status_code}'
                ok = False

    return {
        'status': 'ok' if ok else 'degraded',
        'service': 'cosmos-service',
        'checks': checks,
    }

Go (azcosmos)

// internal/health/handler.go
package health

import (
    "context"
    "encoding/json"
    "net/http"
    "os"
    "time"

    "github.com/Azure/azure-sdk-for-go/sdk/data/azcosmos"
)

func Handler(client *azcosmos.Client) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        checks := map[string]any{}
        ok := true
        start := time.Now()

        ctx, cancel := context.WithTimeout(r.Context(), 8*time.Second)
        defer cancel()

        dbClient, err := client.NewDatabase(os.Getenv("COSMOS_DATABASE_ID"))
        if err != nil {
            checks["account"] = "error: " + err.Error()
            ok = false
        } else {
            _, err = dbClient.Read(ctx, nil)
            if err != nil {
                checks["account"] = "error: " + err.Error()
                ok = false
            } else {
                checks["account"] = "ok"
                checks["latencyMs"] = time.Since(start).Milliseconds()
            }
        }

        w.Header().Set("Content-Type", "application/json")
        if !ok {
            w.WriteHeader(http.StatusServiceUnavailable)
        }
        json.NewEncoder(w).Encode(map[string]any{
            "status":  map[bool]string{true: "ok", false: "degraded"}[ok],
            "service": "cosmos-service",
            "checks":  checks,
        })
    }
}

Step 2: Configure the Vigilmon HTTP Monitor

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: your service's health endpoint (e.g. https://api.example.com/health).
  3. Check interval: 60 seconds.
  4. Response timeout: 10 seconds — Cosmos DB SDK initialization on a cold container can take 2–3 s.
  5. Expected status: 200.
  6. JSON assertion: path status, expected value ok.

Tip: Create a dedicated probe document in your container with a fixed id and partition key. Write it once during provisioning. Your health endpoint can attempt a point read against it — a 404 is treated as connectivity success (the SDK reached Cosmos), while a 429 or 503 signals real trouble.

Step 3: Heartbeat Monitoring for Change Feed Processors

The Cosmos DB Change Feed drives real-time event pipelines, materialized views, and cross-service data sync. Change feed processors are long-running and invisible to HTTP monitors. Use Vigilmon's Heartbeat monitor to detect when processing silently stalls.

# change_feed_processor.py
import os, asyncio, requests
from azure.cosmos.aio import CosmosClient
from azure.cosmos import PartitionKey

async def process_changes(items, context):
    for item in items:
        await handle_change(item)

    # Ping heartbeat after each successful lease checkpoint
    requests.post(os.environ['VIGILMON_HEARTBEAT_URL'], timeout=5)
    print(f"Processed {len(items)} change feed items — heartbeat sent")

async def run():
    async with CosmosClient(
        os.environ['COSMOS_ENDPOINT'],
        credential=os.environ['COSMOS_KEY']
    ) as client:
        monitored_container = client \
            .get_database_client(os.environ['COSMOS_DATABASE_ID']) \
            .get_container_client(os.environ['COSMOS_CONTAINER_ID'])

        lease_container = client \
            .get_database_client(os.environ['COSMOS_DATABASE_ID']) \
            .get_container_client('leases')

        change_feed_processor = monitored_container \
            .create_change_feed_processor(
                processor_name='health-probe-processor',
                lease_container=lease_container,
                on_change=process_changes,
            )

        await change_feed_processor.start()
        await asyncio.Event().wait()

asyncio.run(run())

In Vigilmon, create a Heartbeat monitor:

  • Set the grace period to 3× your expected change feed polling interval (default is 5 seconds, so 15–30 second grace period).
  • A missed heartbeat means the processor crashed, the lease container is unavailable, or the source container stopped receiving writes.

Step 4: RU Throttling Awareness

Cosmos DB returns HTTP 429 when RU/s is exhausted. The SDK retries automatically by default, which hides throttling from your application — until the retry budget runs out and requests start failing. Make throttling visible:

// Middleware to track 429s and expose them in health
let throttleCount = 0;
let lastThrottleTime = null;

cosmos.clientContext.pipeline.addPolicy({
  name: 'throttle-tracker',
  async sendRequest(request, next) {
    const response = await next(request);
    if (response.status === 429) {
      throttleCount++;
      lastThrottleTime = new Date().toISOString();
    }
    return response;
  },
});

// Include in health response
export async function healthHandler(req, res) {
  // ... existing checks ...
  checks.throttleCount = throttleCount;
  checks.lastThrottle = lastThrottleTime;

  const recentThrottleThreshold = parseInt(process.env.THROTTLE_WARN_COUNT || '10', 10);
  if (throttleCount > recentThrottleThreshold) {
    checks.throttling = 'elevated';
    ok = false;
  }

  // Reset counter each health check cycle
  throttleCount = 0;
  res.status(ok ? 200 : 503).json({ status: ok ? 'ok' : 'degraded', checks });
}

Step 5: Alerting Strategy

| Alert channel | When to use | |---|---| | Email / PagerDuty | Service returns 503 (Cosmos DB unreachable or auth failed) | | Slack webhook | Heartbeat missed (change feed processor stalled) | | Slack + Email | 429 throttling elevated — RU/s budget exhausted | | Status page | Public API backed by Azure Cosmos DB |

Configure alert escalation in Vigilmon: notify immediately on first failure, re-notify after 5 minutes. Cosmos DB 429 throttling during peak traffic can escalate into full outages within minutes.

What Vigilmon Catches That Azure Monitor Misses

| Scenario | Azure Monitor | Vigilmon | |---|---|---| | Application can't connect (RBAC error) | No built-in application-layer check | HTTP monitor catches 503 immediately | | SDK uses stale connection after failover | Server-side metrics look fine | Health endpoint detects latency spike | | Change feed processor silently stops | Needs custom metric + alert rule | Heartbeat grace period fires alert | | 429 throttling masked by SDK retries | Server-side metric requires metric alert | Throttle tracker in health endpoint | | Azure Monitor itself is degraded | Alert delivery may also fail | Vigilmon is external — unaffected | | Geo-replication lag causes stale reads | Not surfaced in availability metrics | Latency health check catches it |

Azure Cosmos DB's globally distributed design makes your database elastic and resilient, but RU exhaustion, SDK connection issues, and change feed processing failures can quietly break your application. External health checks from Vigilmon give you the independent signal you need — before your users encounter errors.

Start monitoring your Cosmos DB applications in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles