tutorial

Monitoring VoltDB with Vigilmon: In-Memory NewSQL Availability, Partition Health, and External Uptime Checks

Practical guide to uptime and health monitoring for VoltDB — HTTP health endpoints that probe real partition connectivity, transaction throughput, and independent external monitoring with Vigilmon.

VoltDB is a purpose-built in-memory NewSQL database engineered for high-throughput OLTP workloads — telco billing, financial trading systems, and real-time fraud detection where sub-millisecond transaction latency and strict ACID guarantees are non-negotiable. Its shared-nothing partition architecture serialises all work per partition; a partition failure, a k-safety violation, or a network split between partition replicas can silently degrade throughput or halt writes on affected rows. VoltDB's internal alerting covers process crashes, but it cannot tell you whether your application can actually complete transactions from its network vantage point. Vigilmon fills that gap — independent HTTP health checks that probe real partition connectivity from outside the cluster and alert before your users notice.

This tutorial shows you how to build meaningful monitoring for VoltDB-backed applications using Vigilmon.

What You'll Build

  • A health endpoint that performs a live VoltDB system procedure call to verify connectivity and cluster health
  • Partition health and k-safety violation detection
  • A Vigilmon HTTP monitor with appropriate timeout and assertion settings
  • A heartbeat monitor for VoltDB-backed background workers
  • An alerting strategy tuned for in-memory database failure modes

Prerequisites

  • A running VoltDB cluster (community or enterprise edition)
  • An application connected to VoltDB via its JDBC or HTTP JSON API
  • A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your VoltDB Service

VoltDB exposes a built-in HTTP/JSON API on port 8080 by default. The @SystemInformation and @Statistics system procedures return real-time cluster state — use them in a /health route to give Vigilmon something meaningful to probe.

Node.js (VoltDB HTTP JSON API)

// routes/health.js
import fetch from 'node-fetch';

const VOLTDB_HTTP = process.env.VOLTDB_HTTP_URL || 'http://localhost:8080';

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  // Check 1: Cluster availability via @SystemInformation
  try {
    const resp = await fetch(
      `${VOLTDB_HTTP}/api/1.0/?Procedure=@SystemInformation&Parameters=["OVERVIEW"]`,
      { signal: AbortSignal.timeout(5000) }
    );
    const body = await resp.json();
    checks.pingLatencyMs = Date.now() - start;

    if (body.status !== 1) {
      checks.connectivity = `error: status ${body.status} — ${body.statusstring}`;
      ok = false;
    } else {
      checks.connectivity = 'ok';
    }
  } catch (err) {
    checks.connectivity = `error: ${err.message}`;
    ok = false;
  }

  // Check 2: Partition health via @Statistics PARTITIONCOUNT
  try {
    const statsResp = await fetch(
      `${VOLTDB_HTTP}/api/1.0/?Procedure=@Statistics&Parameters=["PARTITIONCOUNT",0]`,
      { signal: AbortSignal.timeout(5000) }
    );
    const stats = await statsResp.json();

    if (stats.status !== 1) {
      checks.partitionHealth = `error: ${stats.statusstring}`;
      ok = false;
    } else {
      const results = stats.results?.[0]?.data || [];
      const partitionCount = results.reduce((sum, row) => sum + (row[2] || 0), 0);
      checks.partitionCount = partitionCount;
      checks.partitionHealth = partitionCount > 0 ? 'ok' : 'no partitions active';
      if (partitionCount === 0) ok = false;
    }
  } catch (err) {
    checks.partitionHealth = `error: ${err.message}`;
    ok = false;
  }

  // Check 3: Simple write probe via a stored procedure or ad-hoc query
  try {
    const probeStart = Date.now();
    const probeResp = await fetch(
      `${VOLTDB_HTTP}/api/1.0/?Procedure=@AdHoc&Parameters=["SELECT NOW;"]`,
      { signal: AbortSignal.timeout(5000) }
    );
    const probe = await probeResp.json();
    checks.queryLatencyMs = Date.now() - probeStart;
    checks.queryProbe = probe.status === 1 ? 'ok' : `error: ${probe.statusstring}`;
    if (probe.status !== 1) ok = false;
  } catch (err) {
    checks.queryProbe = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'voltdb-service',
    checks,
  });
}

Python (VoltDB HTTP JSON API)

# routers/health.py
import os, time
import httpx
from fastapi import APIRouter
from fastapi.responses import JSONResponse

router = APIRouter()

VOLTDB_HTTP = os.environ.get('VOLTDB_HTTP_URL', 'http://localhost:8080')

@router.get("/health")
async def health():
    checks = {}
    ok = True
    start = time.monotonic()

    async with httpx.AsyncClient(timeout=5.0) as client:
        # Check 1: Cluster availability
        try:
            resp = await client.get(
                f'{VOLTDB_HTTP}/api/1.0/',
                params={'Procedure': '@SystemInformation', 'Parameters': '["OVERVIEW"]'}
            )
            body = resp.json()
            checks['pingLatencyMs'] = int((time.monotonic() - start) * 1000)

            if body.get('status') != 1:
                checks['connectivity'] = f"error: status {body.get('status')} — {body.get('statusstring')}"
                ok = False
            else:
                checks['connectivity'] = 'ok'
        except Exception as e:
            checks['connectivity'] = f'error: {e}'
            ok = False

        # Check 2: Partition health
        try:
            stats_resp = await client.get(
                f'{VOLTDB_HTTP}/api/1.0/',
                params={'Procedure': '@Statistics', 'Parameters': '["PARTITIONCOUNT",0]'}
            )
            stats = stats_resp.json()

            if stats.get('status') != 1:
                checks['partitionHealth'] = f"error: {stats.get('statusstring')}"
                ok = False
            else:
                results = stats.get('results', [{}])[0].get('data', [])
                partition_count = sum(row[2] for row in results if len(row) > 2)
                checks['partitionCount'] = partition_count
                checks['partitionHealth'] = 'ok' if partition_count > 0 else 'no partitions active'
                if partition_count == 0:
                    ok = False
        except Exception as e:
            checks['partitionHealth'] = f'error: {e}'
            ok = False

        # Check 3: Ad-hoc query probe
        try:
            probe_start = time.monotonic()
            probe_resp = await client.get(
                f'{VOLTDB_HTTP}/api/1.0/',
                params={'Procedure': '@AdHoc', 'Parameters': '["SELECT NOW;"]'}
            )
            probe = probe_resp.json()
            checks['queryLatencyMs'] = int((time.monotonic() - probe_start) * 1000)
            checks['queryProbe'] = 'ok' if probe.get('status') == 1 else f"error: {probe.get('statusstring')}"
            if probe.get('status') != 1:
                ok = False
        except Exception as e:
            checks['queryProbe'] = f'error: {e}'
            ok = False

    return JSONResponse(
        status_code=200 if ok else 503,
        content={'status': 'ok' if ok else 'degraded', 'service': 'voltdb-service', 'checks': checks}
    )

Java (VoltDB JDBC)

// HealthController.java
import org.springframework.web.bind.annotation.*;
import org.springframework.http.*;
import org.voltdb.client.*;
import java.util.*;

@RestController
public class HealthController {

    private final Client voltClient;

    public HealthController(Client voltClient) {
        this.voltClient = voltClient;
    }

    @GetMapping("/health")
    public ResponseEntity<Map<String, Object>> health() {
        Map<String, Object> checks = new LinkedHashMap<>();
        boolean ok = true;
        long start = System.currentTimeMillis();

        // Check 1: Connectivity via @SystemInformation
        try {
            ClientResponse resp = voltClient.callProcedure("@SystemInformation", "OVERVIEW");
            checks.put("pingLatencyMs", System.currentTimeMillis() - start);
            if (resp.getStatus() == ClientResponse.SUCCESS) {
                checks.put("connectivity", "ok");
            } else {
                checks.put("connectivity", "error: " + resp.getStatusString());
                ok = false;
            }
        } catch (Exception e) {
            checks.put("connectivity", "error: " + e.getMessage());
            ok = false;
        }

        // Check 2: Query probe
        try {
            long probeStart = System.currentTimeMillis();
            ClientResponse probe = voltClient.callProcedure("@AdHoc", "SELECT NOW;");
            checks.put("queryLatencyMs", System.currentTimeMillis() - probeStart);
            if (probe.getStatus() == ClientResponse.SUCCESS) {
                checks.put("queryProbe", "ok");
            } else {
                checks.put("queryProbe", "error: " + probe.getStatusString());
                ok = false;
            }
        } catch (Exception e) {
            checks.put("queryProbe", "error: " + e.getMessage());
            ok = false;
        }

        Map<String, Object> body = new LinkedHashMap<>();
        body.put("status", ok ? "ok" : "degraded");
        body.put("service", "voltdb-service");
        body.put("checks", checks);
        return ResponseEntity.status(ok ? 200 : 503).body(body);
    }
}

Deploy and verify manually before wiring it into Vigilmon:

curl -i https://your-app.example.com/health
# HTTP/1.1 200 OK
# {"status":"ok","service":"voltdb-service","checks":{"pingLatencyMs":2,"connectivity":"ok","partitionCount":8,"partitionHealth":"ok","queryLatencyMs":1,"queryProbe":"ok"}}

Step 2: Configure the Vigilmon HTTP Monitor

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: your service's health endpoint (e.g. https://api.example.com/health).
  3. Check interval: 60 seconds.
  4. Response timeout: 10 seconds (VoltDB in-memory operations are fast; anything over 5 seconds signals cluster stress).
  5. Expected status: 200.
  6. JSON assertion: path status, expected value ok.
  7. Save the monitor.

Vigilmon probes from multiple geographic regions simultaneously, so transient single-region blips won't wake your on-call team. This matters for VoltDB because brief latency spikes during re-election or partition rebalancing resolve in seconds — you want durable alerts, not noise.

Tip: Add a second JSON assertion on checks.partitionHealth equals ok. This fires specifically when VoltDB is running but has lost active partitions — a failure mode that wouldn't trigger a plain HTTP status check.

Step 3: Heartbeat Monitoring for VoltDB-Backed Workers

VoltDB is often the transaction engine for high-frequency event pipelines — billing systems, game session managers, IoT telemetry ingestion. Workers that depend on VoltDB can stall silently when k-safety falls below threshold or when snapshot compaction pauses the cluster. Vigilmon's heartbeat monitors detect this: your worker pings Vigilmon after each successful cycle, and missed pings trigger an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat.
  2. Set the name: voltdb-worker.
  3. Set the expected interval: 2 minutes (adjust to your job frequency).
  4. Set the grace period: 5 minutes.
  5. Save — copy the unique heartbeat URL.

Wire It Into Your Worker

Node.js:

import fetch from 'node-fetch';

const VOLTDB_HTTP = process.env.VOLTDB_HTTP_URL || 'http://localhost:8080';
const HEARTBEAT_URL = process.env.VIGILMON_HEARTBEAT_URL;

async function runWorkerCycle() {
  // Execute transactional batch via stored procedure
  const resp = await fetch(
    `${VOLTDB_HTTP}/api/1.0/?Procedure=ProcessEventBatch&Parameters=[50]`,
    { signal: AbortSignal.timeout(8000) }
  );
  const result = await resp.json();
  if (result.status !== 1) {
    throw new Error(`Batch failed: ${result.statusstring}`);
  }

  // Only ping after successful commit
  await fetch(HEARTBEAT_URL).catch(() => {});
}

setInterval(async () => {
  try {
    await runWorkerCycle();
  } catch (err) {
    console.error(`Worker cycle failed — heartbeat NOT sent: ${err.message}`);
  }
}, 60_000);

Python:

import os, time, requests

VOLTDB_HTTP = os.environ.get('VOLTDB_HTTP_URL', 'http://localhost:8080')
HEARTBEAT_URL = os.environ['VIGILMON_HEARTBEAT_URL']

def run_worker_cycle():
    resp = requests.get(
        f'{VOLTDB_HTTP}/api/1.0/',
        params={'Procedure': 'ProcessEventBatch', 'Parameters': '[50]'},
        timeout=8
    )
    result = resp.json()
    if result.get('status') != 1:
        raise RuntimeError(f"Batch failed: {result.get('statusstring')}")

    # Only ping after successful commit
    requests.get(HEARTBEAT_URL, timeout=5)

while True:
    try:
        run_worker_cycle()
    except Exception as e:
        print(f'Worker cycle failed — heartbeat NOT sent: {e}')
    time.sleep(60)

Step 4: VoltDB System Procedures for Direct Cluster Health Checks

For infrastructure-level insight alongside Vigilmon's application-layer checks, VoltDB's system procedures provide the authoritative cluster view:

# Check cluster health via HTTP JSON API
curl 'http://localhost:8080/api/1.0/?Procedure=@SystemInformation&Parameters=["OVERVIEW"]' | python3 -m json.tool

# Check partition statistics
curl 'http://localhost:8080/api/1.0/?Procedure=@Statistics&Parameters=["PARTITIONCOUNT",0]' | python3 -m json.tool

# Check k-safety status
curl 'http://localhost:8080/api/1.0/?Procedure=@Statistics&Parameters=["REBALANCE",0]' | python3 -m json.tool

# Key indicators to monitor:
# status == 1 — procedures are executing
# PARTITIONCOUNT.data[*][2] — active partition count (should match configured value)
# @SystemInformation OVERVIEW — node count, cluster mode, build version

You can wrap these checks in a cron job or dedicated probe that pushes metrics to Prometheus or Grafana — and use Vigilmon for the independent external HTTP check that confirms application-layer availability.

Step 5: Alerting Strategy

| Alert channel | When to use | |---|---| | PagerDuty | Connectivity probe fails (cluster unreachable) | | PagerDuty | partitionHealth not ok (active partition loss) | | PagerDuty | Transaction query probe fails or returns non-success status | | Slack webhook | Query latency > 5ms (elevated for in-memory system) | | PagerDuty | Heartbeat missed (event pipeline stopped) | | Status page | Any user-facing service backed by VoltDB |

Set response time thresholds as an early warning:

  • Alert at 5ms for query probe latency (in-memory transactions should complete in under 1ms)
  • Alert at 3000ms for health endpoint response time (anything over 2s on an in-memory database signals serious cluster distress)

What Vigilmon Catches That Internal Monitoring Misses

| Scenario | VoltDB system procedures / internal metrics | Vigilmon | |---|---|---| | Cluster reachable from ops host but not from app servers | No | HTTP probe from app's perspective | | K-safety violation (below fault tolerance threshold) | @Statistics only | partitionHealth check fires | | Partition leader re-election (brief write stall) | Cluster log only | Query probe fails — 503 fires | | Worker pipeline silently stopped | Process appears running | Heartbeat grace period expires → alert | | Query latency spike (GC pressure or snapshot I/O) | Internal metric only | Latency threshold fires | | Monitoring pipeline (Prometheus/Grafana) fails | Alerts silently lost | Vigilmon is external — unaffected |

VoltDB's in-memory speed is only as reliable as the partition topology serving your application. K-safety violations and partition leader transitions happen faster than any human can respond — you need automated external monitoring that sees failures from your application's perspective. Vigilmon provides that independent layer in minutes.

Start monitoring your VoltDB applications in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles