tutorial

Monitoring DGraph with Vigilmon: Graph Database Availability, Cluster Health, and External Uptime Checks

Practical guide to uptime and health monitoring for DGraph — HTTP health endpoints that probe real cluster connectivity, Alpha and Zero node availability, and independent external monitoring with Vigilmon.

DGraph is a native graph database built for high-performance graph traversals, distributed across a cluster of Alpha nodes coordinated by Zero nodes. Its distributed architecture means there are multiple independent failure surfaces: a Zero node losing quorum, an Alpha node falling behind replication, a schema mutation leaving the cluster in a partially applied state, and client-side gRPC connection pool exhaustion can all cause silent degradation while DGraph's internal processes appear healthy. Vigilmon adds an independent external layer — HTTP health checks on your DGraph-connected services that verify real Alpha connectivity, surface cluster health signals, and alert on application-level failures before your users notice.

This tutorial shows you how to build meaningful monitoring for DGraph-backed applications using Vigilmon.

What You'll Build

  • A health endpoint that probes real DGraph Alpha and Zero connectivity
  • Cluster state and schema health detection
  • A Vigilmon HTTP monitor with appropriate timeout and assertion settings
  • A heartbeat monitor for DGraph-backed background workers
  • A cluster-aware alerting strategy for multi-Alpha deployments

Prerequisites

  • A running DGraph cluster (single-node or multi-Alpha production)
  • An application connected to DGraph (any runtime)
  • A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your DGraph Service

Add a /health endpoint that performs a lightweight DGraph query to verify real Alpha connectivity and checks cluster state via the admin API.

Node.js (dgraph-js-http)

// routes/health.js
import fetch from 'node-fetch';

const DGRAPH_ALPHA = process.env.DGRAPH_ALPHA_URL || 'http://localhost:8080';
const DGRAPH_ZERO = process.env.DGRAPH_ZERO_URL || 'http://localhost:6080';

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  // Check 1: Alpha health via built-in /health endpoint
  try {
    const alphaRes = await fetch(`${DGRAPH_ALPHA}/health`, {
      signal: AbortSignal.timeout(5000),
    });
    const alphaBody = await alphaRes.json();
    checks.alphaStatus = alphaRes.ok ? 'ok' : 'degraded';
    checks.alphaLatencyMs = Date.now() - start;

    // DGraph /health returns an array of node health objects
    if (Array.isArray(alphaBody)) {
      const unhealthy = alphaBody.filter(n => n.status !== 'healthy');
      checks.alphaNodeCount = alphaBody.length;
      checks.unhealthyAlphaNodes = unhealthy.length;
      if (unhealthy.length > 0) {
        checks.alphaStatus = `${unhealthy.length} unhealthy nodes`;
        ok = false;
      }
    } else if (!alphaRes.ok) {
      ok = false;
    }
  } catch (err) {
    checks.alphaStatus = `error: ${err.message}`;
    ok = false;
  }

  // Check 2: Zero state — cluster membership and leader election
  try {
    const zeroRes = await fetch(`${DGRAPH_ZERO}/state`, {
      signal: AbortSignal.timeout(5000),
    });
    if (zeroRes.ok) {
      const state = await zeroRes.json();
      const groups = state.groups ? Object.keys(state.groups).length : 0;
      checks.groupCount = groups;
      checks.zeroLeader = state.leader ?? 'unknown';
      checks.zeroStatus = 'ok';
    } else {
      checks.zeroStatus = `http ${zeroRes.status}`;
      ok = false;
    }
  } catch (err) {
    checks.zeroStatus = `error: ${err.message}`;
    ok = false;
  }

  // Check 3: Probe with a minimal DQL query
  try {
    const queryStart = Date.now();
    const queryRes = await fetch(`${DGRAPH_ALPHA}/query`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/dql' },
      body: '{ vigilmon(func: has(dgraph.type), first: 1) { uid } }',
      signal: AbortSignal.timeout(5000),
    });
    if (queryRes.ok) {
      checks.queryStatus = 'ok';
      checks.queryLatencyMs = Date.now() - queryStart;
    } else {
      const body = await queryRes.text();
      checks.queryStatus = `error: ${body.slice(0, 100)}`;
      ok = false;
    }
  } catch (err) {
    checks.queryStatus = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'dgraph-service',
    checks,
  });
}

Python (pydgraph)

# routers/health.py
import os, time
import requests
from fastapi import APIRouter
from fastapi.responses import JSONResponse

router = APIRouter()

DGRAPH_ALPHA = os.environ.get('DGRAPH_ALPHA_URL', 'http://localhost:8080')
DGRAPH_ZERO = os.environ.get('DGRAPH_ZERO_URL', 'http://localhost:6080')

@router.get("/health")
async def health():
    checks = {}
    ok = True
    start = time.monotonic()

    # Check 1: Alpha health
    try:
        r = requests.get(f'{DGRAPH_ALPHA}/health', timeout=5)
        checks['alphaLatencyMs'] = int((time.monotonic() - start) * 1000)
        if r.ok:
            body = r.json()
            if isinstance(body, list):
                unhealthy = [n for n in body if n.get('status') != 'healthy']
                checks['alphaNodeCount'] = len(body)
                checks['unhealthyAlphaNodes'] = len(unhealthy)
                checks['alphaStatus'] = f'{len(unhealthy)} unhealthy' if unhealthy else 'ok'
                if unhealthy:
                    ok = False
            else:
                checks['alphaStatus'] = 'ok'
        else:
            checks['alphaStatus'] = f'http {r.status_code}'
            ok = False
    except Exception as e:
        checks['alphaStatus'] = f'error: {e}'
        ok = False

    # Check 2: Zero state
    try:
        r = requests.get(f'{DGRAPH_ZERO}/state', timeout=5)
        if r.ok:
            state = r.json()
            checks['groupCount'] = len(state.get('groups', {}))
            checks['zeroLeader'] = state.get('leader', 'unknown')
            checks['zeroStatus'] = 'ok'
        else:
            checks['zeroStatus'] = f'http {r.status_code}'
            ok = False
    except Exception as e:
        checks['zeroStatus'] = f'error: {e}'
        ok = False

    # Check 3: Minimal DQL query
    try:
        q_start = time.monotonic()
        r = requests.post(
            f'{DGRAPH_ALPHA}/query',
            data='{ vigilmon(func: has(dgraph.type), first: 1) { uid } }',
            headers={'Content-Type': 'application/dql'},
            timeout=5,
        )
        if r.ok:
            checks['queryStatus'] = 'ok'
            checks['queryLatencyMs'] = int((time.monotonic() - q_start) * 1000)
        else:
            checks['queryStatus'] = f'error: {r.text[:100]}'
            ok = False
    except Exception as e:
        checks['queryStatus'] = f'error: {e}'
        ok = False

    return JSONResponse(
        status_code=200 if ok else 503,
        content={'status': 'ok' if ok else 'degraded', 'service': 'dgraph-service', 'checks': checks},
    )

Go (dgo)

// internal/health/handler.go
package health

import (
    "context"
    "encoding/json"
    "net/http"
    "os"
    "time"
)

func Handler() http.HandlerFunc {
    alphaURL := os.Getenv("DGRAPH_ALPHA_URL")
    if alphaURL == "" {
        alphaURL = "http://localhost:8080"
    }
    zeroURL := os.Getenv("DGRAPH_ZERO_URL")
    if zeroURL == "" {
        zeroURL = "http://localhost:6080"
    }
    httpClient := &http.Client{Timeout: 6 * time.Second}

    return func(w http.ResponseWriter, r *http.Request) {
        ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
        defer cancel()
        _ = ctx

        checks := map[string]any{}
        ok := true

        // Check 1: Alpha health endpoint
        start := time.Now()
        alphaResp, err := httpClient.Get(alphaURL + "/health")
        if err != nil {
            checks["alphaStatus"] = "error: " + err.Error()
            ok = false
        } else {
            checks["alphaLatencyMs"] = time.Since(start).Milliseconds()
            if alphaResp.StatusCode == 200 {
                checks["alphaStatus"] = "ok"
            } else {
                checks["alphaStatus"] = "degraded"
                ok = false
            }
            alphaResp.Body.Close()
        }

        // Check 2: Zero state
        zeroResp, err := httpClient.Get(zeroURL + "/state")
        if err != nil {
            checks["zeroStatus"] = "error: " + err.Error()
            ok = false
        } else {
            if zeroResp.StatusCode == 200 {
                checks["zeroStatus"] = "ok"
            } else {
                checks["zeroStatus"] = "degraded"
                ok = false
            }
            zeroResp.Body.Close()
        }

        // Check 3: Minimal DQL query
        qStart := time.Now()
        body := []byte(`{ vigilmon(func: has(dgraph.type), first: 1) { uid } }`)
        qReq, _ := http.NewRequest("POST", alphaURL+"/query", bytesReader(body))
        qReq.Header.Set("Content-Type", "application/dql")
        qResp, err := httpClient.Do(qReq)
        if err != nil {
            checks["queryStatus"] = "error: " + err.Error()
            ok = false
        } else {
            checks["queryLatencyMs"] = time.Since(qStart).Milliseconds()
            checks["queryStatus"] = map[bool]string{true: "ok", false: "error"}[qResp.StatusCode == 200]
            if qResp.StatusCode != 200 {
                ok = false
            }
            qResp.Body.Close()
        }

        w.Header().Set("Content-Type", "application/json")
        if !ok {
            w.WriteHeader(http.StatusServiceUnavailable)
        }
        _ = json.NewEncoder(w).Encode(map[string]any{
            "status":  map[bool]string{true: "ok", false: "degraded"}[ok],
            "service": "dgraph-service",
            "checks":  checks,
        })
    }
}

func bytesReader(b []byte) *bytesReaderWrapper {
    return &bytesReaderWrapper{data: b, pos: 0}
}

type bytesReaderWrapper struct {
    data []byte
    pos  int
}

func (br *bytesReaderWrapper) Read(p []byte) (n int, err error) {
    if br.pos >= len(br.data) {
        return 0, io.EOF
    }
    n = copy(p, br.data[br.pos:])
    br.pos += n
    return
}

Deploy and verify manually before wiring it into Vigilmon:

curl -i https://your-app.example.com/health
# HTTP/1.1 200 OK
# {"status":"ok","service":"dgraph-service","checks":{"alphaStatus":"ok","zeroStatus":"ok","queryStatus":"ok","queryLatencyMs":12}}

Step 2: Configure the Vigilmon HTTP Monitor

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: your service's health endpoint (e.g. https://api.example.com/health).
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds (DQL queries can take slightly longer than key-value lookups).
  5. Expected status: 200.
  6. JSON assertion: path status, expected value ok.
  7. Save the monitor.

Vigilmon probes from multiple geographic regions simultaneously. A single-region blip will not page you; multi-region consensus is required before an incident opens. This is especially valuable for DGraph because Alpha node failures may leave the cluster degraded but not fully down, causing partial failures that single-region checks miss.

Tip: Create a second monitor that asserts on checks.unhealthyAlphaNodes equals 0. This alerts on partial cluster degradation before your primary connectivity check fails.

Step 3: Heartbeat Monitoring for DGraph-Backed Workers

DGraph is commonly used for graph traversal pipelines, recommendation engines, and fraud detection jobs that run on a schedule or event trigger. These workers will stall silently when the Alpha cluster becomes unavailable or when a schema mutation leaves the database in an inconsistent state. Vigilmon's heartbeat monitors detect silent worker death: your worker pings Vigilmon after each successful cycle, and if pings stop arriving, Vigilmon fires an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat.
  2. Set the name: dgraph-worker.
  3. Set the expected interval: 5 minutes (adjust to your pipeline frequency).
  4. Set the grace period: 10 minutes.
  5. Save — copy the unique heartbeat URL.

Wire It Into Your Worker

Node.js:

import fetch from 'node-fetch';
import { DgraphClient, DgraphClientStub, Mutation } from 'dgraph-js-http';

const stub = new DgraphClientStub(process.env.DGRAPH_ALPHA_URL);
const client = new DgraphClient(stub);

async function runGraphPipeline() {
  const txn = client.newTxn();
  try {
    const query = `{ nodes(func: eq(status, "pending"), first: 50) { uid status } }`;
    const result = await txn.query(query);
    const nodes = result.data.nodes;

    // Process each node
    for (const node of nodes) {
      const mu = new Mutation();
      mu.setSetJson({ uid: node.uid, status: 'processed' });
      await txn.mutate(mu);
    }
    await txn.commit();

    // Only ping Vigilmon after successful commit
    await fetch(process.env.VIGILMON_HEARTBEAT_URL).catch(() => {});
  } finally {
    await txn.discard();
  }
}

setInterval(runGraphPipeline, 5 * 60 * 1000);

Python:

import os, time, requests
import pydgraph

stub = pydgraph.DgraphClientStub(os.environ['DGRAPH_ALPHA_GRPC'])
client = pydgraph.DgraphClient(stub)

def run_graph_pipeline():
    txn = client.txn()
    try:
        query = '{ nodes(func: eq(status, "pending"), first: 50) { uid status } }'
        result = txn.query(query)
        nodes = result.json.get('nodes', [])

        for node in nodes:
            mutation = {'uid': node['uid'], 'status': 'processed'}
            txn.mutate(set_obj=mutation)

        txn.commit()

        # Only ping Vigilmon after successful commit
        requests.get(os.environ['VIGILMON_HEARTBEAT_URL'], timeout=5)
    except Exception as e:
        print(f"Pipeline failed — heartbeat NOT sent: {e}")
    finally:
        txn.discard()

while True:
    run_graph_pipeline()
    time.sleep(300)

Step 4: Multi-Alpha Cluster Monitoring

In production DGraph deployments, each Alpha group serves a shard of the data. Monitor each group independently:

  • Create one Vigilmon monitor per Alpha group if you expose per-group health endpoints
  • Name monitors clearly: [dgraph-alpha-1] graph-api /health, [dgraph-alpha-2] graph-api /health
  • Monitor Zero separately: Zero handles cluster membership and schema operations; a Zero outage blocks all schema mutations even if Alpha nodes remain queryable

DGraph exposes a built-in /health endpoint on each Alpha node at port 8080. Wire Vigilmon directly to this endpoint for the most direct signal:

# Monitor DGraph Alpha directly
# URL: http://alpha1.internal:8080/health
# Expected: status 200, body contains "healthy"

Group all DGraph monitors into a single Status Page so your team has a single pane of glass for the graph tier.

Step 5: Alerting Strategy

| Alert channel | When to use | |---|---| | PagerDuty | Alpha connectivity fails (cluster unreachable) | | PagerDuty | Zero unreachable (schema mutations blocked) | | Slack webhook | Unhealthy Alpha node count > 0 (partial degradation) | | Slack + Email | Query latency > 500ms sustained (cluster under load) | | PagerDuty | Heartbeat missed (graph pipeline stopped) | | Status page | Any user-facing service backed by DGraph |

Set response time thresholds as an early warning:

  • Alert at 100ms for DQL probe query latency (simple traversals should complete in < 50ms)
  • Alert at 2000ms for health endpoint response time

What Vigilmon Catches That Internal Monitoring Misses

| Scenario | DGraph internal metrics | Vigilmon | |---|---|---| | Alpha node reachable internally but not from app servers | No | HTTP probe from app's perspective | | Zero loses quorum (schema mutations silently blocked) | Cluster log only | Zero /state check returns degraded | | DQL query timeout (cluster overloaded) | Latency metric only | Query check fails — 503 fires | | Worker pipeline silently stopped | Process appears running | Heartbeat grace period expires → alert | | Schema mutation left database inconsistent | No | Query probe returns error | | Monitoring pipeline (Prometheus/Grafana) fails | Alerts silently lost | Vigilmon is external — unaffected |

DGraph's distributed architecture is its strength for graph workloads and its challenge for operations — multiple failure surfaces that process-level checks miss. External health checks with Vigilmon give you the independent signal you need — before graph traversal latency reaches your users.

Start monitoring your DGraph applications in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles