tutorial

How to Monitor RavenDB Uptime and Health with Vigilmon

RavenDB replication conflicts, indexing errors, and cluster leader elections can degrade your database silently. Learn how to monitor RavenDB uptime externally with Vigilmon HTTP probes, heartbeat monitors for subscriptions, and alert routing for cluster health.

RavenDB is designed for zero-downtime resilience, but its multi-master replication model, auto-indexing, and subscription workers mean there are many moving parts that can fail silently. A cluster node that loses connectivity from the leader will continue serving reads from a stale in-memory state. An indexing error on a newly deployed map-reduce index silently returns incomplete results. A document subscription worker that throws an unhandled exception stops processing without alerting your on-call team.

Vigilmon gives you external visibility into RavenDB health through HTTP probe monitoring and heartbeat monitoring for subscription workers and background jobs. This tutorial walks through both.

Why RavenDB Monitoring Needs More Than Process Checks

systemd, Docker health checks, and the RavenDB Studio tell you the process is running. They cannot tell you:

  • Whether RavenDB is reachable from your application servers across the network
  • Whether a cluster leader election is in progress and writes are temporarily unavailable
  • Whether index errors are silently returning partial or stale query results
  • Whether a document subscription worker has stopped processing
  • Whether replication conflicts are accumulating and require manual resolution
  • Whether license limits (document count, indexing throughput) have been reached

These are the failure modes that produce incorrect, slow, or unavailable responses without clean error signals. External monitoring through Vigilmon catches them by probing the actual connectivity and logic paths your application relies on.

Step 1: Build a RavenDB Health Endpoint

RavenDB exposes a /health endpoint on each node and a management Studio UI, but these only confirm the process is alive — not that your application's database and indexes are healthy. Add a thin health route to your application server.

Node.js / Express Example

// health/ravendb.js
const express = require('express');
const { DocumentStore } = require('ravendb');

const app = express();
const store = new DocumentStore(
  [process.env.RAVENDB_URL],
  process.env.RAVENDB_DATABASE
);
store.initialize();

app.get('/health/ravendb', async (req, res) => {
  const session = store.openSession();
  try {
    // 1. Write + read probe document
    const probeId = 'HealthProbes/1';
    await session.store({ type: 'health_probe', ts: Date.now() }, probeId);
    await session.saveChanges();

    const readSession = store.openSession();
    const probe = await readSession.load(probeId);
    if (!probe || !probe.ts) {
      return res.status(503).json({ status: 'degraded', reason: 'probe_read_mismatch' });
    }

    // 2. Check cluster topology for leader availability
    const topology = await store.maintenance.server.send(
      new (require('ravendb').GetDatabaseTopologyCommand)(process.env.RAVENDB_DATABASE)
    ).catch(() => null);

    if (topology) {
      const hasLeader = topology.Leader !== null && topology.Leader !== '';
      if (!hasLeader) {
        return res.status(503).json({ status: 'degraded', reason: 'no_cluster_leader' });
      }
    }

    // 3. Check for index errors
    const stats = await store.maintenance.send(
      new (require('ravendb').GetStatisticsOperation)()
    );
    const indexErrors = stats.Indexes?.filter(i => i.State === 'Error').length ?? 0;
    if (indexErrors > 0) {
      return res.status(503).json({
        status: 'degraded',
        reason: 'index_errors',
        errored_indexes: indexErrors,
      });
    }

    return res.status(200).json({ status: 'ok' });
  } catch (err) {
    return res.status(503).json({ status: 'down', error: err.message });
  } finally {
    session.dispose?.();
  }
});

app.listen(3002);

Python (FastAPI) Example

# health_ravendb.py
from fastapi import FastAPI
from fastapi.responses import JSONResponse
import httpx
import os
import time

app = FastAPI()

RAVENDB_URL = os.environ["RAVENDB_URL"]       # e.g. http://localhost:8080
RAVENDB_DB  = os.environ["RAVENDB_DATABASE"]  # e.g. NorthWind

@app.get("/health/ravendb")
async def ravendb_health():
    try:
        async with httpx.AsyncClient(timeout=5) as client:
            # 1. Write probe document
            put_resp = await client.put(
                f"{RAVENDB_URL}/databases/{RAVENDB_DB}/docs?id=HealthProbes%2F1",
                json={"type": "health_probe", "ts": int(time.time() * 1000)},
                headers={"Content-Type": "application/json"},
            )
            if put_resp.status_code not in (200, 201):
                return JSONResponse(status_code=503, content={"status": "degraded", "reason": "write_failed"})

            # 2. Read probe document
            get_resp = await client.get(
                f"{RAVENDB_URL}/databases/{RAVENDB_DB}/docs?id=HealthProbes%2F1"
            )
            if get_resp.status_code != 200:
                return JSONResponse(status_code=503, content={"status": "degraded", "reason": "read_failed"})

            # 3. Check cluster leader
            cluster_resp = await client.get(f"{RAVENDB_URL}/cluster/topology")
            if cluster_resp.status_code == 200:
                topology = cluster_resp.json()
                leader = topology.get("Leader", "")
                if not leader:
                    return JSONResponse(status_code=503, content={
                        "status": "degraded",
                        "reason": "no_cluster_leader",
                    })

            # 4. Check index health
            stats_resp = await client.get(f"{RAVENDB_URL}/databases/{RAVENDB_DB}/stats")
            if stats_resp.status_code == 200:
                stats = stats_resp.json()
                errored = [i for i in stats.get("Indexes", []) if i.get("State") == "Error"]
                if errored:
                    return JSONResponse(status_code=503, content={
                        "status": "degraded",
                        "reason": "index_errors",
                        "errored_indexes": len(errored),
                    })

        return {"status": "ok"}
    except Exception as e:
        return JSONResponse(status_code=503, content={"status": "down", "error": str(e)})

Verify the endpoint before wiring up Vigilmon:

curl -i https://your-app.example.com/health/ravendb
# HTTP/1.1 200 OK
# {"status":"ok"}

Step 2: Configure Vigilmon HTTP Monitor for RavenDB

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your health endpoint: https://your-app.example.com/health/ravendb
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: "status":"ok"
    • Response time threshold: 3000ms
  6. Under Alert channels, assign your Slack or PagerDuty channel
  7. Save the monitor

Vigilmon probes from multiple geographic regions simultaneously, requiring multi-region consensus before opening an incident. This eliminates false alarms from transient single-probe network hiccups.

Monitoring Cluster Nodes Individually

For RavenDB clusters, monitor each node separately to detect partial failures during rolling restarts or leader elections:

  • [ravendb-node-A] /health/ravendb — P1 alert on node failure
  • [ravendb-node-B] /health/ravendb — P1 alert on node failure
  • [ravendb-node-C] /health/ravendb — P1 alert on node failure

Use Vigilmon's status page grouping to collect all RavenDB monitors in a single pane for on-call engineers.

Step 3: Heartbeat Monitoring for Subscription Workers and Background Jobs

RavenDB document subscriptions deliver batches of documents to long-running workers. When a subscription worker crashes or disconnects, RavenDB will attempt reconnection — but your processing logic stops until it reconnects. Batch processing scripts, ETL tasks, and periodic maintenance operations can also stall silently.

Vigilmon heartbeat monitors detect silent stalls: your worker pings Vigilmon after each successfully processed batch. If pings stop arriving within the expected window, Vigilmon fires an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
  2. Set the name: ravendb-subscription-worker
  3. Set the expected interval: 5 minutes (adjust to your batch throughput)
  4. Set the grace period: 10 minutes
  5. Save — copy the unique heartbeat URL, e.g. https://vigilmon.online/heartbeat/abc123xyz

Wire It Into a Subscription Worker

// subscription-worker.js
const { DocumentStore } = require('ravendb');
const axios = require('axios');

const store = new DocumentStore([process.env.RAVENDB_URL], process.env.RAVENDB_DATABASE);
store.initialize();

async function run() {
  const subscriptionName = await store.subscriptions.ensureExists({
    query: 'from Orders where Status = "Pending"',
  }).catch(() => 'Orders/Pending'); // Use existing if already created

  const worker = store.subscriptions.getSubscriptionWorker({ subscriptionName });

  worker.on('batch', async (batch, callback) => {
    try {
      for (const item of batch.items) {
        await processOrder(item.result);
      }
      // Ping Vigilmon after each successful batch
      await axios.get(process.env.VIGILMON_HEARTBEAT_URL, { timeout: 5000 }).catch(() => {});
      callback();
    } catch (err) {
      console.error('Subscription batch error:', err);
      callback(err); // Heartbeat stops — Vigilmon alerts within grace period
    }
  });

  worker.on('error', err => {
    console.error('Subscription worker error:', err);
  });
}

run();

For periodic maintenance tasks such as document expiration cleanup or ETL exports:

# etl_export.py
import requests
import os
import httpx

def run_etl():
    ravendb_url = os.environ["RAVENDB_URL"]
    db = os.environ["RAVENDB_DATABASE"]

    # Fetch documents modified in the last 24 hours
    with httpx.Client(timeout=30) as client:
        resp = client.post(
            f"{ravendb_url}/databases/{db}/queries",
            json={"Query": "from Orders where LastModified > '2024-01-01' select *"},
        )
        resp.raise_for_status()
        docs = resp.json().get("Results", [])

    # ... export logic ...
    print(f"Exported {len(docs)} documents")

    # Ping Vigilmon on successful completion
    requests.get(os.environ["VIGILMON_HEARTBEAT_URL"], timeout=5)

if __name__ == "__main__":
    run_etl()

Step 4: Replication Conflict and Alert Routing

RavenDB's multi-master model can produce document conflicts when the same document is written concurrently on multiple nodes during a network partition. These conflicts accumulate in the conflict queue and require resolution (either automatic via a resolution script or manual). Expose conflict counts in your health endpoint:

// Add to your health endpoint:
const conflictsResp = await fetch(
  `${process.env.RAVENDB_URL}/databases/${process.env.RAVENDB_DATABASE}/replication/conflicts/count`,
  { signal: AbortSignal.timeout(2000) }
);
if (conflictsResp.ok) {
  const data = await conflictsResp.json();
  const conflictCount = data.TotalResults ?? 0;
  if (conflictCount > 10) {
    return res.status(503).json({
      status: 'degraded',
      reason: 'replication_conflicts',
      conflict_count: conflictCount,
    });
  }
}

Configure alert routing in Vigilmon:

| Monitor | Alert Channel | Priority | |---|---|---| | RavenDB node /health/ravendb | Slack + PagerDuty | P1 | | Per-cluster-node monitors | Slack | P1 | | Heartbeat: subscription worker | Slack + email | P2 | | Heartbeat: ETL export job | Email | P3 |

Set response time thresholds as early warnings:

  • Alert at 1000ms for the health endpoint (document fetches should be fast)
  • Alert at 4000ms for application query endpoints (signals index staleness or cluster pressure)

Summary

RavenDB failures surface in subtle ways — silent index errors, leader election stalls, subscription worker disconnections, replication conflicts — long before your users see errors. Vigilmon gives you external visibility across the full failure surface:

| Monitor Type | What It Covers | |---|---| | HTTP monitor on /health/ravendb | Read/write round-trip, leader availability, index health | | HTTP monitor per cluster node | Per-node availability during rolling restarts | | Heartbeat monitor | Subscription worker liveness, batch job completion |

Get started free at vigilmon.online — your first RavenDB monitor is running in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles