tutorial

How to Monitor Apache Directory Server (ApacheDS) Health with Vigilmon

Apache Directory Server LDAP port unavailability, replication lag, and Kerberos KDC failures are invisible until users report authentication errors across all systems. Learn how to monitor ApacheDS availability, partition health, and replication job liveness with Vigilmon HTTP probes and heartbeat monitors.

Apache Directory Server (ApacheDS) is a fully Java-based LDAP v3 server that also provides Kerberos authentication, DNS, and change password services. When ApacheDS's LDAP port becomes unavailable due to a JVM out-of-memory crash — a known failure mode under high concurrent bind volume — every application in your infrastructure that uses LDAP for authentication simultaneously loses access, with no external alert unless you have active monitoring on port 389 or 636; when ApacheDS's JDBM partition reaches its configured size limit, write operations throw LdapOperationErrorException with UNWILLING_TO_PERFORM while read operations and binds continue normally — the error is returned to the writing client but nothing alerts operations teams that the directory is in a read-only failure state; when ApacheDS replication falls behind in a consumer-provider topology, the consumer continues serving stale directory entries for reads while the replication thread silently queues an unbounded backlog. These are infrastructure-wide authentication failures disguised as isolated application errors.

Vigilmon gives you external visibility into ApacheDS's directory service health through HTTP probe monitoring and heartbeat monitors. This tutorial covers both.

Why ApacheDS Needs External Monitoring

ApacheDS's failure modes have infrastructure-wide blast radius:

  • JVM memory exhaustion crash: ApacheDS runs in a JVM and is sensitive to heap pressure under concurrent bind storms; when the JVM OOMs, the process exits and the LDAP port becomes unreachable — all authentication across every application using this directory server fails simultaneously until the process is restarted
  • Partition size limit reached: ApacheDS partitions have a configurable size limit; when the JDBM partition on the consumer fills up, writes silently fail with UNWILLING_TO_PERFORM — new user registrations, password changes, and group membership updates are rejected while authentication for existing accounts continues to work
  • Replication consumer lag: ApacheDS multi-master or consumer-provider replication can fall behind when the consumer is processing a large import; during lag, the consumer serves stale entries for password changes and new group assignments, causing "correct password fails" errors for users whose passwords were recently reset
  • Kerberos KDC port failure: ApacheDS provides an embedded Kerberos KDC on port 88; when the KDC thread crashes due to an unhandled exception in ticket processing, Kerberos-dependent services (SSH with GSSAPI, HDFS with Kerberos auth) fail while LDAP authentication continues — the failure is in a different port and process thread, so LDAP health checks miss it
  • Schema violation on entry load: during startup after a schema change, ApacheDS validates all stored entries against the new schema; if any entry violates a new must-attribute constraint, ApacheDS fails to load that partition and the affected directory subtree becomes inaccessible — other partitions continue working
  • TLS certificate expiry on LDAPS: ApacheDS's LDAPS certificate is configured in config.ldif; when the certificate expires, new TLS connections fail with a handshake error while existing connections and plain LDAP on port 389 continue — monitoring that only checks port 389 misses the LDAPS failure

External monitoring with Vigilmon adds:

  • Proactive alerting when ApacheDS LDAP ports stop accepting connections
  • Partition health visibility through the embedded HTTP administration endpoint
  • Heartbeat monitoring so you know when scheduled replication health checks and partition maintenance tasks stop completing
  • Multi-region probe consensus that filters transient GC-pause bind timeouts from genuine ApacheDS crashes

Step 1: Build an ApacheDS Health Endpoint

ApacheDS exposes an HTTP administration endpoint on port 8080 by default. Use this alongside a custom LDAP probe sidecar for comprehensive health monitoring.

ApacheDS HTTP Admin Endpoint

ApacheDS ships with an embedded HTTP server that exposes /system/info and partition statistics. Enable it in conf/server.xml (or config.ldif for embedded deployments):

<!-- conf/server.xml — enable HTTP admin endpoint -->
<httpServer port="8080">
  <webApp contextPath="/admin" warFile="apacheds-admin.war"/>
</httpServer>

Probe the HTTP endpoint from Vigilmon directly at:

http://apacheds.your-org.example.com:8080/admin/alive

For ApacheDS 2.x, the REST API is minimal. Build a dedicated sidecar that performs an LDAP bind probe and exposes it over HTTP.

Node.js LDAP Health Sidecar

// health/apacheds.js
const express = require('express');
const ldap = require('ldapjs');

const app = express();
const LDAP_URL = process.env.LDAP_URL || 'ldap://localhost:389';
const LDAPS_URL = process.env.LDAPS_URL || 'ldaps://localhost:636';
const BIND_DN = process.env.LDAP_BIND_DN || 'uid=admin,ou=system';
const BIND_PW = process.env.LDAP_BIND_PW || 'secret';
const BASE_DN = process.env.LDAP_BASE || 'dc=example,dc=com';

function bindProbe(url) {
  return new Promise((resolve, reject) => {
    const client = ldap.createClient({
      url,
      connectTimeout: 4000,
      tlsOptions: { rejectUnauthorized: false }, // for self-signed LDAPS
    });
    client.on('error', err => { client.destroy(); reject(err); });
    client.bind(BIND_DN, BIND_PW, err => {
      if (err) { client.destroy(); return reject(err); }
      client.search(BASE_DN, { scope: 'base', filter: '(objectClass=*)' }, (serr, res) => {
        if (serr) { client.destroy(); return reject(serr); }
        res.on('searchEntry', () => {});
        res.on('error', e => { client.destroy(); reject(e); });
        res.on('end', result => {
          client.unbind();
          if (result.status !== 0) return reject(new Error(`Search status ${result.status}`));
          resolve(true);
        });
      });
    });
  });
}

app.get('/health/apacheds', async (req, res) => {
  const checks = {};
  let healthy = true;

  try {
    await bindProbe(LDAP_URL);
    checks.ldap_389 = 'ok';
  } catch (err) {
    checks.ldap_389 = `down: ${err.message}`;
    healthy = false;
  }

  try {
    await bindProbe(LDAPS_URL);
    checks.ldaps_636 = 'ok';
  } catch (err) {
    checks.ldaps_636 = `down: ${err.message}`;
    // LDAPS failure is P1 — alert but don't mark overall as down if LDAP works
    if (!healthy) {
      checks.ldaps_636 = `down: ${err.message}`;
    }
  }

  return res.status(healthy ? 200 : 503).json({
    status: healthy ? 'ok' : 'down',
    checks,
  });
});

app.listen(3022);

Python Health Sidecar

# health_apacheds.py
import os
from ldap3 import Server, Connection, ALL, Tls
from flask import Flask, jsonify
import ssl

app = Flask(__name__)

LDAP_HOST = os.environ.get('LDAP_HOST', 'localhost')
LDAP_PORT = int(os.environ.get('LDAP_PORT', '389'))
LDAPS_PORT = int(os.environ.get('LDAPS_PORT', '636'))
BIND_DN = os.environ.get('LDAP_BIND_DN', 'uid=admin,ou=system')
BIND_PW = os.environ.get('LDAP_BIND_PW', 'secret')
BASE_DN = os.environ.get('LDAP_BASE', 'dc=example,dc=com')

def probe_ldap(port, use_ssl=False):
    tls = Tls(validate=ssl.CERT_NONE) if use_ssl else None
    server = Server(LDAP_HOST, port=port, use_ssl=use_ssl, tls=tls,
                    connect_timeout=4)
    conn = Connection(server, BIND_DN, BIND_PW, auto_bind=True,
                      receive_timeout=4)
    conn.search(BASE_DN, '(objectClass=*)', search_scope='BASE')
    conn.unbind()
    return True

@app.route('/health/apacheds')
def health():
    checks = {}
    healthy = True

    try:
        probe_ldap(LDAP_PORT)
        checks['ldap_389'] = 'ok'
    except Exception as e:
        checks['ldap_389'] = f'down: {e}'
        healthy = False

    try:
        probe_ldap(LDAPS_PORT, use_ssl=True)
        checks['ldaps_636'] = 'ok'
    except Exception as e:
        checks['ldaps_636'] = f'down: {e}'

    return jsonify({'status': 'ok' if healthy else 'down', 'checks': checks}), \
           200 if healthy else 503

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=3022)

Java Spring Boot Sidecar

// health/ApacheDsHealthIndicator.java
package com.yourorg.health;

import com.unboundid.ldap.sdk.*;
import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

@Component
public class ApacheDsHealthIndicator implements HealthIndicator {

    @Value("${ldap.host:localhost}")
    private String ldapHost;

    @Value("${ldap.port:389}")
    private int ldapPort;

    @Value("${ldap.bind-dn:uid=admin,ou=system}")
    private String bindDn;

    @Value("${ldap.bind-password:secret}")
    private String bindPw;

    @Value("${ldap.base-dn:dc=example,dc=com}")
    private String baseDn;

    @Override
    public Health health() {
        try (LDAPConnection conn = new LDAPConnection(ldapHost, ldapPort)) {
            conn.bind(bindDn, bindPw);
            SearchResult result = conn.search(
                baseDn, SearchScope.BASE, "(objectClass=*)"
            );
            return Health.up()
                .withDetail("apacheds_ldap", "connected")
                .withDetail("host", ldapHost + ":" + ldapPort)
                .withDetail("base_entries", result.getEntryCount())
                .build();
        } catch (LDAPException e) {
            return Health.down(e)
                .withDetail("apacheds_ldap", "failed")
                .withDetail("ldap_result_code", e.getResultCode().toString())
                .build();
        }
    }
}

Step 2: Configure Vigilmon HTTP Monitor for ApacheDS

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your ApacheDS health sidecar: http://apacheds.your-org.example.com:3022/health/apacheds
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: "status":"ok"
    • Response time threshold: 4000ms
  6. Under Alert channels, assign your Slack or PagerDuty channel — set P1 priority (LDAP down = all authentication failing)
  7. Save the monitor

Add monitors for LDAPS and Kerberos KDC:

| Endpoint / Port | What it catches | |---|---| | HTTP sidecar /health/apacheds | LDAP bind and search probe on ports 389 and 636 | | TCP check port 389 | Raw LDAP port availability (JVM crash detection) | | TCP check port 636 | LDAPS port availability and TLS handshake | | TCP check port 88 | Kerberos KDC port availability | | HTTP /admin/alive port 8080 | ApacheDS admin HTTP server liveness |

For TCP port monitors:

  1. In Vigilmon, choose TCP monitor type
  2. Enter apacheds.your-org.example.com and port 389
  3. Set interval to 1 minute
  4. Set alert priority to P1 with immediate PagerDuty notification

This catches JVM crashes and process exits faster than any HTTP probe, since TCP connect failures are near-instantaneous.

Vigilmon's multi-region probing confirms a genuine ApacheDS outage across multiple vantage points before paging the on-call engineer — filtering single-region network blips from actual crashes.

Step 3: Heartbeat Monitoring for Replication and Maintenance Jobs

HTTP and TCP monitors verify ApacheDS port availability but not the health of replication and background maintenance jobs. Common scheduled tasks in ApacheDS deployments include:

  • Replication consumer lag checks: periodic scripts that compare CSN (Change Sequence Number) between provider and consumer to detect replication lag
  • Partition compaction: periodic JDBM or MVCC store compaction to prevent partition fragmentation
  • Password policy enforcement audit: periodic scan that verifies pwdPolicy references are valid on all user entries
  • Backup export: ldapsearch-based export of the directory to LDIF for disaster recovery

Vigilmon heartbeat monitors detect when these jobs stop completing.

Set Up Heartbeat Monitors

For replication lag check liveness:

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
  2. Set the name: apacheds-replication-check
  3. Set the expected interval: 70 minutes
  4. Set the grace period: 90 minutes
  5. Save — copy the heartbeat URL

For LDIF backup liveness:

  1. Create another Heartbeat monitor named apacheds-ldif-backup
  2. Set the expected interval: 25 hours
  3. Set the grace period: 27 hours
  4. Save — copy the heartbeat URL

Wire Heartbeats Into Maintenance Scripts

#!/bin/bash
# scripts/check-apacheds-replication.sh
# Compares CSN between provider and consumer and pings heartbeat only if in sync

set -e

PROVIDER_HOST="${LDAP_PROVIDER_HOST:-ldap-provider.internal}"
CONSUMER_HOST="${LDAP_CONSUMER_HOST:-ldap-consumer.internal}"
BIND_DN="${LDAP_BIND_DN:-uid=admin,ou=system}"
BIND_PW="${LDAP_BIND_PW:-secret}"
BASE_DN="${LDAP_BASE:-dc=example,dc=com}"
MAX_LAG_SECONDS="${REPLICATION_MAX_LAG_SECONDS:-300}"

get_csn() {
  ldapsearch -H "ldap://$1" \
    -D "$BIND_DN" -w "$BIND_PW" \
    -b "cn=changelog" -s base "(objectClass=*)" contextCSN 2>/dev/null \
    | grep contextCSN | awk '{print $2}' | head -1
}

PROVIDER_CSN=$(get_csn "$PROVIDER_HOST")
CONSUMER_CSN=$(get_csn "$CONSUMER_HOST")

if [ -z "$PROVIDER_CSN" ] || [ -z "$CONSUMER_CSN" ]; then
  echo "ERROR: Could not retrieve CSN from one or both hosts"
  exit 1
fi

if [ "$PROVIDER_CSN" = "$CONSUMER_CSN" ]; then
  echo "Replication in sync: CSN $PROVIDER_CSN"
  if [ -n "$VIGILMON_REPLICATION_HEARTBEAT_URL" ]; then
    curl -s "$VIGILMON_REPLICATION_HEARTBEAT_URL" > /dev/null 2>&1 || true
  fi
else
  echo "WARN: Replication lag detected. Provider=$PROVIDER_CSN Consumer=$CONSUMER_CSN"
  # Do not ping heartbeat — lag means Vigilmon fires the alert
  exit 1
fi

#!/bin/bash
# scripts/apacheds-ldif-backup.sh
set -e

BACKUP_DIR="/opt/apacheds/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="$BACKUP_DIR/apacheds_${TIMESTAMP}.ldif"

mkdir -p "$BACKUP_DIR"

ldapsearch -H "ldap://${LDAP_HOST:-localhost}" \
  -D "${LDAP_BIND_DN:-uid=admin,ou=system}" \
  -w "${LDAP_BIND_PW:-secret}" \
  -b "${LDAP_BASE:-dc=example,dc=com}" \
  "(objectClass=*)" > "$BACKUP_FILE"

gzip "$BACKUP_FILE"
echo "Backup written to ${BACKUP_FILE}.gz"

if [ -n "$VIGILMON_LDIF_BACKUP_HEARTBEAT_URL" ]; then
  curl -s "$VIGILMON_LDIF_BACKUP_HEARTBEAT_URL" > /dev/null 2>&1 || true
fi

For Java-based partition maintenance jobs running inside ApacheDS:

// maintenance/PartitionCompactionJob.java
package com.yourorg.apacheds.maintenance;

import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import java.net.URI;
import java.net.http.*;

@Service
public class PartitionCompactionJob {

    private final String heartbeatUrl = System.getenv("VIGILMON_COMPACTION_HEARTBEAT_URL");
    private final HttpClient http = HttpClient.newHttpClient();

    @Scheduled(cron = "0 0 3 * * SUN") // weekly at 3 AM Sunday
    public void compactPartitions() {
        try {
            performCompaction();
            pingHeartbeat();
        } catch (Exception e) {
            throw new RuntimeException("Partition compaction failed", e);
        }
    }

    private void performCompaction() {
        // ApacheDS partition maintenance via DirectoryService API
    }

    private void pingHeartbeat() {
        if (heartbeatUrl == null || heartbeatUrl.isBlank()) return;
        try {
            http.sendAsync(
                HttpRequest.newBuilder(URI.create(heartbeatUrl)).GET().build(),
                HttpResponse.BodyHandlers.discarding()
            );
        } catch (Exception ignored) {}
    }
}

Step 4: Alert Routing for ApacheDS Failures

ApacheDS failures are infrastructure-wide by nature — every application using LDAP authentication is affected when the directory server is down. Route alerts by blast radius:

| Monitor | Alert Channel | Priority | |---|---|---| | TCP: port 389 LDAP | Slack + PagerDuty | P1 (immediate) | | TCP: port 636 LDAPS | Slack + PagerDuty | P1 | | TCP: port 88 Kerberos KDC | Slack + PagerDuty | P1 | | HTTP: LDAP bind probe sidecar | Slack + PagerDuty | P1 | | HTTP: admin endpoint port 8080 | Slack | P2 | | Heartbeat: replication lag check | Slack + PagerDuty | P1 | | Heartbeat: LDIF backup | Slack + email | P2 | | Heartbeat: partition compaction | Slack + email | P3 |

Set response time thresholds as early warning signals:

  • Alert at 2000ms for the LDAP bind probe (slow bind responses indicate connection pool pressure or JVM GC pressure before a crash)
  • Alert at 1000ms for TCP port 389 (near-instant connect should always succeed)
  • Alert at 90 minutes gap in replication heartbeat (replication lag may be causing stale reads)
  • Alert at 27 hours gap in LDIF backup heartbeat (disaster recovery gap accumulating)

For production ApacheDS deployments, set up a status page in Vigilmon to give all application teams visibility into directory server health — instead of every team filing tickets when authentication fails, they check the status page first and understand whether the incident is infrastructure-wide or application-specific. This significantly reduces mean time to diagnosis during LDAP outages.

Summary

ApacheDS failures cascade instantly across all LDAP-dependent applications, but different failure modes are caught by different monitor types. A TCP probe catches JVM crashes fastest; an LDAP bind probe confirms authentication is functional; heartbeat monitors catch replication drift before users notice stale credentials:

| Monitor Type | What It Covers | |---|---| | TCP monitor on port 389/636/88 | LDAP, LDAPS, and Kerberos KDC port availability | | HTTP monitor on LDAP probe sidecar | Bind and search functionality inside the directory | | HTTP monitor on admin endpoint | ApacheDS HTTP administration service availability | | Heartbeat monitor (replication check) | Consumer-provider replication lag and CSN sync | | Heartbeat monitor (LDIF backup) | Scheduled directory backup job liveness |

Get started free at vigilmon.online — your first ApacheDS directory monitor is running in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles