tutorial

How to Monitor OrientDB Uptime and Health with Vigilmon

OrientDB heap exhaustion, cluster split-brain, and record version conflicts can degrade your multi-model graph database silently. Learn how to monitor OrientDB externally with Vigilmon HTTP probes, heartbeat monitors for ETL pipelines and batch queries, and alert routing for distributed mode.

OrientDB is a multi-model database that handles Graph, Document, Object, and Key-Value data in a single engine — but that versatility comes with a complex failure surface. A JVM heap exhaustion event will kill the OrientDB process entirely, but only after a period of severe GC pauses where queries appear to complete successfully while response times spiral. A distributed cluster running in Hazelcast-based consensus can enter a split-brain state where two partitions each believe they hold the authoritative copy of a record, causing silent data divergence. Record version conflicts from concurrent writes are logged at the driver level but do not surface as HTTP errors unless your health probe is designed to catch them.

Vigilmon gives you external visibility into OrientDB health through HTTP probe monitoring and heartbeat monitoring for ETL pipelines and scheduled SQL queries. This tutorial walks through both.

Why OrientDB Monitoring Needs More Than Process Checks

systemd, Docker health checks, and JVM process monitors tell you OrientDB is running. They cannot tell you:

  • Whether OrientDB is reachable from your application servers across the network — the Studio UI and HTTP API share port 2480, so a firewall change can kill both simultaneously
  • Whether the JVM heap is under pressure and causing stop-the-world GC pauses that stall queries before an OOM crash
  • Whether a distributed cluster has lost quorum and a REPLICA node is serving stale data because it can no longer reach the MASTER
  • Whether a split-brain partition has occurred and two nodes each believe they hold the canonical record version
  • Whether an OrientDB ETL pipeline has stalled mid-import because of a schema mismatch or malformed record
  • Whether a scheduled SQL query used for materialized views or reporting has stopped running without error
  • Whether record version conflicts from concurrent multi-model writes are accumulating and causing silent data loss

These are the failure modes that produce degraded experiences and data inconsistency without clean error signals. External monitoring through Vigilmon catches them by probing the actual connectivity and logic paths your application relies on.

Step 1: Build an OrientDB Health Endpoint

OrientDB exposes an HTTP API on port 2480 (the same port as the Studio web UI). The /api/server endpoint returns server statistics including storage engines, cluster membership, and connection counts. Use this endpoint as the foundation of your health probe.

Python (FastAPI) Example

# health_orientdb.py
from fastapi import FastAPI
from fastapi.responses import JSONResponse
import httpx
import os

app = FastAPI()

# OrientDB HTTP API — default port 2480
ORIENTDB_HTTP = os.environ.get("ORIENTDB_HTTP", "http://localhost:2480")
ORIENTDB_USER = os.environ.get("ORIENTDB_USER", "root")
ORIENTDB_PASS = os.environ.get("ORIENTDB_PASS", "")

@app.get("/health/orientdb")
async def orientdb_health():
    try:
        auth = (ORIENTDB_USER, ORIENTDB_PASS)

        async with httpx.AsyncClient(timeout=5) as client:
            # 1. Check /api/server for storage engines and cluster state
            server_resp = await client.get(
                f"{ORIENTDB_HTTP}/api/server",
                auth=auth,
            )
            if server_resp.status_code != 200:
                return JSONResponse(status_code=503, content={
                    "status": "down",
                    "reason": "api_server_unreachable",
                    "http_status": server_resp.status_code,
                })

            server_data = server_resp.json()

            # 2. Verify at least one storage engine is listed
            storages = server_data.get("storages", [])
            if not storages:
                return JSONResponse(status_code=503, content={
                    "status": "degraded",
                    "reason": "no_storage_engines_reported",
                })

            # 3. Check for any storage in error state
            error_storages = [
                s["name"] for s in storages
                if s.get("status", "").upper() not in ("ONLINE", "OPEN", "")
            ]
            if error_storages:
                return JSONResponse(status_code=503, content={
                    "status": "degraded",
                    "reason": "storage_error",
                    "affected_storages": error_storages,
                })

            connections = server_data.get("connections", 0)

        return {
            "status": "ok",
            "storages": len(storages),
            "connections": connections,
        }

    except httpx.ConnectError as e:
        return JSONResponse(status_code=503, content={
            "status": "down",
            "reason": "connection_refused",
            "error": str(e),
        })
    except Exception as e:
        return JSONResponse(status_code=503, content={
            "status": "down",
            "error": str(e),
        })

Java / Vert.x Example

If your OrientDB service is embedded inside a Java application, expose the health check using Vert.x's HTTP server alongside your main application. This keeps the health probe in-process so it reflects the actual OrientDB client connection state.

// OrientDbHealthVerticle.java
import io.vertx.core.AbstractVerticle;
import io.vertx.core.json.JsonObject;
import io.vertx.ext.web.Router;
import io.vertx.ext.web.client.WebClient;
import io.vertx.ext.web.client.WebClientOptions;

public class OrientDbHealthVerticle extends AbstractVerticle {

    private static final String ORIENTDB_HOST = System.getenv().getOrDefault("ORIENTDB_HOST", "localhost");
    private static final int ORIENTDB_PORT = 2480;
    private static final String ORIENTDB_USER = System.getenv().getOrDefault("ORIENTDB_USER", "root");
    private static final String ORIENTDB_PASS = System.getenv().getOrDefault("ORIENTDB_PASS", "");

    @Override
    public void start() {
        Router router = Router.router(vertx);
        WebClient httpClient = WebClient.create(vertx, new WebClientOptions().setConnectTimeout(5000));

        router.get("/health/orientdb").handler(ctx -> {
            httpClient.get(ORIENTDB_PORT, ORIENTDB_HOST, "/api/server")
                .basicAuthentication(ORIENTDB_USER, ORIENTDB_PASS)
                .timeout(5000)
                .send(ar -> {
                    if (ar.failed()) {
                        ctx.response()
                            .setStatusCode(503)
                            .end(new JsonObject()
                                .put("status", "down")
                                .put("reason", "connection_failed")
                                .put("error", ar.cause().getMessage())
                                .encode());
                        return;
                    }

                    var resp = ar.result();
                    if (resp.statusCode() != 200) {
                        ctx.response()
                            .setStatusCode(503)
                            .end(new JsonObject()
                                .put("status", "down")
                                .put("http_status", resp.statusCode())
                                .encode());
                        return;
                    }

                    JsonObject body = resp.bodyAsJsonObject();
                    var storages = body.getJsonArray("storages");
                    if (storages == null || storages.isEmpty()) {
                        ctx.response()
                            .setStatusCode(503)
                            .end(new JsonObject()
                                .put("status", "degraded")
                                .put("reason", "no_storage_engines_reported")
                                .encode());
                        return;
                    }

                    ctx.response()
                        .setStatusCode(200)
                        .end(new JsonObject()
                            .put("status", "ok")
                            .put("storages", storages.size())
                            .put("connections", body.getInteger("connections", 0))
                            .encode());
                });
        });

        vertx.createHttpServer().requestHandler(router).listen(8080);
    }
}

Verify the endpoint before wiring up Vigilmon:

curl -i http://your-app.example.com:8080/health/orientdb
# HTTP/1.1 200 OK
# {"status":"ok","storages":2,"connections":4}

Step 2: Configure Vigilmon HTTP Monitor for OrientDB

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your health endpoint: https://your-app.example.com/health/orientdb
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: "status":"ok"
    • Response time threshold: 3000ms
  6. Under Alert channels, assign your Slack or PagerDuty channel
  7. Save the monitor

Vigilmon probes from multiple geographic regions simultaneously, requiring multi-region consensus before opening an incident. This eliminates false alarms from transient single-probe network hiccups.

Distributed Mode vs. Single-Node

OrientDB can run as a standalone single-node server or in distributed mode using Hazelcast-based consensus. The monitoring strategy differs:

Single-node: Create one HTTP monitor pointing at your health endpoint. Any failure is immediately P1 — there is no replica to fail over to.

Distributed mode (MASTER + REPLICA cluster): Create separate monitors for each node role:

  • [orientdb-master] /health/orientdb?node=master — immediate P1 page on failure; writes stop
  • [orientdb-replica-1] /health/orientdb?node=replica1 — P2 Slack alert; read traffic degrades
  • [orientdb-replica-2] /health/orientdb?node=replica2 — P2 Slack alert

Pass a node query parameter to your health handler to direct the probe at the correct OrientDB host. When the MASTER becomes unreachable and the cluster cannot achieve quorum, the remaining nodes will refuse writes — your monitor will catch this as a 503 before your application logs fill with errors.

Use Vigilmon's status page grouping to surface all OrientDB node monitors in a single pane for on-call engineers.

Step 3: Heartbeat Monitoring for OrientDB ETL Jobs and Batch Queries

OrientDB's ETL framework imports data from CSV, JSON, JDBC, and other sources into the multi-model engine using a pipeline of extractor, transformer, and loader components. When an ETL job fails mid-pipeline — due to a schema mismatch, a record version conflict, or a heap pressure event that triggers an OOM — the job terminates but does not notify your team. Similarly, OrientDB supports SQL-like queries for scheduled batch operations; these run silently and their failure is only discoverable by inspecting logs.

Vigilmon heartbeat monitors detect silent stalls: your pipeline or batch job pings Vigilmon after each successful completion. If pings stop arriving within the expected window, Vigilmon fires an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
  2. Set the name: orientdb-etl-pipeline
  3. Set the expected interval: 4 hours (adjust to your ETL schedule)
  4. Set the grace period: 30 minutes
  5. Save — copy the unique heartbeat URL, e.g. https://vigilmon.online/heartbeat/abc123xyz

Wire It Into an OrientDB ETL Job

OrientDB ETL jobs are defined in a JSON configuration file and run via the oetl.sh script. Wrap the execution in a Python or shell script that pings Vigilmon on successful completion:

# run_orientdb_etl.py
import subprocess
import requests
import sys
import os

OETL_SCRIPT = os.environ.get("OETL_SCRIPT", "/opt/orientdb/bin/oetl.sh")
ETL_CONFIG = os.environ.get("ETL_CONFIG", "/etc/orientdb/etl/import-products.json")
VIGILMON_HEARTBEAT_URL = os.environ["VIGILMON_HEARTBEAT_URL"]

def run_etl():
    result = subprocess.run(
        [OETL_SCRIPT, ETL_CONFIG],
        capture_output=True,
        text=True,
        timeout=3600,  # 1 hour max
    )

    if result.returncode != 0:
        print(f"ETL job failed (exit {result.returncode}):", file=sys.stderr)
        print(result.stderr, file=sys.stderr)
        sys.exit(result.returncode)  # Heartbeat stops — Vigilmon alerts within grace period

    print("ETL job completed successfully")
    print(result.stdout)

    # Ping Vigilmon only on clean exit
    try:
        requests.get(VIGILMON_HEARTBEAT_URL, timeout=5)
    except Exception as e:
        print(f"Warning: failed to ping Vigilmon heartbeat: {e}", file=sys.stderr)

if __name__ == "__main__":
    run_etl()

Wire It Into a Scheduled OrientDB SQL Batch Query

OrientDB SQL queries can be run via the HTTP API. For batch operations that populate reporting tables or refresh materialized graph views, wrap the query in a Python scheduler:

# orientdb_batch_query.py
import httpx
import requests
import os
import sys
from apscheduler.schedulers.blocking import BlockingScheduler

ORIENTDB_HTTP = os.environ.get("ORIENTDB_HTTP", "http://localhost:2480")
ORIENTDB_USER = os.environ.get("ORIENTDB_USER", "root")
ORIENTDB_PASS = os.environ.get("ORIENTDB_PASS", "")
ORIENTDB_DB = os.environ.get("ORIENTDB_DB", "GratefulDeadConcerts")
VIGILMON_HEARTBEAT_URL = os.environ["VIGILMON_HEARTBEAT_URL"]

def run_batch_query():
    # OrientDB SQL query via HTTP API — POST to /command/{db}/sql
    sql = "SELECT FROM V WHERE @class = 'Product' AND updated_at < sysdate() - 7"
    try:
        resp = httpx.post(
            f"{ORIENTDB_HTTP}/command/{ORIENTDB_DB}/sql",
            auth=(ORIENTDB_USER, ORIENTDB_PASS),
            json={"command": sql},
            timeout=120,
        )
        resp.raise_for_status()
        result = resp.json()
        record_count = len(result.get("result", []))
        print(f"Batch query returned {record_count} records")

        # Ping Vigilmon on successful completion
        requests.get(VIGILMON_HEARTBEAT_URL, timeout=5)
    except Exception as e:
        print(f"Batch query failed: {e}", file=sys.stderr)
        # Do NOT ping heartbeat — Vigilmon alerts within grace period

scheduler = BlockingScheduler()
scheduler.add_job(run_batch_query, "cron", hour=2, minute=0)  # nightly at 02:00
scheduler.start()

Step 4: Alert Routing and Response Time Thresholds

OrientDB response times are a leading indicator of JVM heap pressure. Under normal operation, an /api/server call completes in under 100ms. When the JVM enters frequent minor GC cycles before a heap exhaustion event, that latency climbs to 500ms, then 2000ms, then the process dies. Set tight response time thresholds in Vigilmon to catch heap pressure before the OOM crash.

Configure alert routing in Vigilmon:

| Monitor | Alert Channel | Priority | |---|---|---| | OrientDB MASTER /health/orientdb | Slack + PagerDuty | P1 | | OrientDB REPLICA nodes /health/orientdb | Slack | P2 | | Heartbeat: ETL pipeline | Slack + email | P2 | | Heartbeat: nightly batch SQL query | Email | P3 |

Set response time thresholds as early warnings of JVM heap pressure:

  • Alert at 500ms for the health endpoint — OrientDB /api/server should respond in under 100ms under normal load; 500ms signals GC pause activity
  • Alert at 2000ms for the health endpoint — at this threshold a heap exhaustion crash is likely imminent; page immediately
  • Alert at 5000ms for application endpoints backed by OrientDB graph traversals (signals missing index or deep traversal on a large graph)

Summary

OrientDB failures surface in subtle ways — JVM heap exhaustion with its preceding GC pressure, split-brain partitions in distributed clusters, silent ETL job failures, record version conflicts from concurrent multi-model writes — long before your users see errors. Vigilmon gives you external visibility across the full failure surface:

| Monitor Type | What It Covers | |---|---| | HTTP monitor on /health/orientdb | HTTP API reachability, storage engine status, connection count | | HTTP monitor per cluster node | MASTER availability, REPLICA sync, distributed quorum failures | | Heartbeat monitor (ETL pipeline) | OrientDB ETL job completion, import liveness | | Heartbeat monitor (batch SQL) | Scheduled SQL query execution, reporting pipeline health |

Get started free at vigilmon.online — your first OrientDB monitor is running in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles