tutorial

Monitoring Apache Solr with Vigilmon: Core Health, Query Latency, ZooKeeper Connectivity & Search Service SLAs

How to monitor Apache Solr with Vigilmon — core/collection health, indexing throughput, query handler latency, replica sync status, ZooKeeper connectivity, and external checks for search service SLAs.

Apache Solr powers search for enterprise applications, e-commerce platforms, and content management systems where query latency and index freshness are directly tied to revenue. When a Solr core goes down, a replica falls out of sync, or ZooKeeper connectivity is lost in SolrCloud mode, search results degrade or disappear — often silently, with application-layer fallbacks masking the failure until users complain. Vigilmon gives you external monitoring of Solr's admin API, core health, and ZooKeeper connectivity to catch search service degradation before it becomes an outage.

What You'll Build

  • A liveness monitor on Solr's admin ping endpoint
  • Core and collection health checks via the Solr admin API
  • ZooKeeper connectivity monitoring for SolrCloud deployments
  • Query handler latency sentinel via the Solr metrics API
  • Replica sync status monitoring
  • SSL certificate monitoring for your Solr endpoint

Prerequisites

  • A running Apache Solr 8+ or 9+ instance (standalone or SolrCloud mode)
  • An accessible admin endpoint (e.g., https://solr.example.com)
  • At least one configured core or collection
  • A free account at vigilmon.online

Step 1: Verify Solr's Admin Ping Endpoint

Solr provides a built-in ping handler at /solr/<core>/admin/ping:

curl "https://solr.example.com/solr/my-collection/admin/ping?wt=json"

A healthy Solr core returns:

{
  "responseHeader": {
    "status": 0,
    "QTime": 2
  },
  "status": "OK"
}

The status: OK string and responseHeader.status: 0 confirm the core is loaded, the request handler is responsive, and the schema is intact.

Authentication: Solr supports Basic Auth and LDAP via the security plugin. Pass credentials with: curl -u solr:SolrRocks https://solr.example.com/solr/my-collection/admin/ping?wt=json. Create a read-only monitoring user in security.json rather than using admin credentials.

Step 2: Create a Vigilmon Core Ping Monitor

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://solr.example.com/solr/my-collection/admin/ping?wt=json.
  3. Check interval: 60 seconds.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: "status":"OK".
  7. Click Save.

This monitor catches the full failure cascade: JVM crash (no connection), core unloaded (404), index corruption (non-OK status), and request handler failure (non-zero responseHeader.status).

Step 3: Monitor Core Status via Admin API

For standalone Solr deployments, monitor all cores via the admin cores endpoint:

curl "https://solr.example.com/solr/admin/cores?action=STATUS&wt=json"

A healthy multi-core response:

{
  "responseHeader": { "status": 0, "QTime": 5 },
  "status": {
    "my-collection": {
      "name": "my-collection",
      "isLoaded": true,
      "isLoading": false,
      "uptime": 3600000,
      "index": {
        "numDocs": 2500000,
        "size": "8.5 GB"
      }
    }
  }
}

Add a monitor for core availability:

  1. Add Monitor → HTTP.
  2. URL: https://solr.example.com/solr/admin/cores?action=STATUS&wt=json.
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: "isLoaded":true.
  6. Label: Solr core status.

A missing "isLoaded":true means a core is absent or still loading — queries against an unloaded core return 404 errors to search applications.

Step 4: Monitor SolrCloud Collection Health

For SolrCloud deployments, use the collections API to check collection and replica health:

curl "https://solr.example.com/solr/admin/collections?action=CLUSTERSTATUS&wt=json"

This returns the full cluster state including replica states for each shard:

{
  "cluster": {
    "collections": {
      "my-collection": {
        "shards": {
          "shard1": {
            "state": "active",
            "replicas": {
              "core_node1": { "state": "active", "leader": "true" },
              "core_node2": { "state": "active" }
            }
          }
        }
      }
    }
  }
}

Replica states:

  • active: Replica is synced and serving queries
  • recovering: Replica is syncing from the leader (normal after restart)
  • recovery_failed: Replica couldn't sync; requires manual intervention
  • down: Replica is offline

Add a monitor that alerts on replica failures:

  1. Add Monitor → HTTP.
  2. URL: https://solr.example.com/solr/admin/collections?action=CLUSTERSTATUS&wt=json.
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: "recovery_failed" — alert when present.
  6. Label: SolrCloud replica health.

Step 5: Monitor ZooKeeper Connectivity

SolrCloud depends entirely on ZooKeeper for cluster coordination, leader election, and configuration management. Lost ZooKeeper connectivity causes entire shards to become unavailable.

Check ZooKeeper status via the Solr admin API:

curl "https://solr.example.com/solr/admin/zookeeper?detail=true&path=/&wt=json"

This endpoint errors if Solr has lost its ZooKeeper session. Add a Vigilmon monitor:

  1. Add Monitor → HTTP.
  2. URL: https://solr.example.com/solr/admin/zookeeper?path=/&wt=json.
  3. Check interval: 2 minutes.
  4. Expected status: 200.
  5. Keyword: "stat" (present in the ZooKeeper node stat response).
  6. Label: SolrCloud ZooKeeper connectivity.

A 500 response or timeout from this endpoint indicates lost ZooKeeper connectivity — the entire SolrCloud cluster may be degraded.

Step 6: Monitor Query Handler Latency via Metrics API

Solr exposes a comprehensive metrics API that includes query handler request times and error rates:

curl "https://solr.example.com/solr/my-collection/admin/mbeans?cat=QUERYHANDLER&stats=true&wt=json"

Look for requestTimes under the /select handler:

{
  "responseHeader": { "status": 0 },
  "beans": [{
    "/select": {
      "stats": {
        "requestTimes": {
          "avgRequestsPerSecond": 45.2,
          "avgTimePerRequest": 12.3,
          "errors": 0
        }
      }
    }
  }]
}

Add a monitor that checks the metrics endpoint responds quickly (a proxy for query handler health):

  1. Add Monitor → HTTP.
  2. URL: https://solr.example.com/solr/my-collection/admin/mbeans?cat=QUERYHANDLER&stats=true&wt=json.
  3. Check interval: 2 minutes.
  4. Response timeout: 5 seconds — a slow metrics response indicates overloaded query handlers.
  5. Expected status: 200.
  6. Keyword: "requestTimes".
  7. Label: Solr query handler metrics.

Step 7: Monitor Indexing Throughput

For Solr deployments with continuous indexing (SolrJ clients, DataImportHandler, or Kafka sink connectors), track the document count to detect indexing stalls:

curl "https://solr.example.com/solr/admin/cores?action=STATUS&wt=json"

Extract index.numDocs from the response. A stagnant numDocs when documents should be arriving indicates indexing pipeline failure.

Add a monitor:

  1. Add Monitor → HTTP.
  2. URL: https://solr.example.com/solr/my-collection/admin/ping?wt=json.
  3. Check interval: 5 minutes.
  4. Keyword: "QTime" — a low QTime in the ping response indicates healthy indexing throughput; a high value indicates I/O saturation.
  5. Label: Solr indexing health: my-collection.

Step 8: SSL Certificate Monitoring

  1. Add Monitor → SSL Certificate.
  2. Domain: solr.example.com.
  3. Alert when expiry is within: 30 days.
  4. Click Save.

Solr is frequently accessed by internal services (CMS platforms, e-commerce backends, recommendation engines) that validate TLS strictly. A certificate expiry cuts off all search functionality silently, with applications receiving SSLHandshakeException errors that don't appear in Solr's own logs.

Step 9: Configure Alert Routing

In Vigilmon under Settings → Notifications:

| Monitor | Trigger | Action | |---|---|---| | Core ping | Status not OK | Core unloaded or corrupt; check Solr logs | | Core status | isLoaded false | Core failed to load; check schema and data directory | | Replica health | recovery_failed present | Manual re-sync required; check disk space | | ZooKeeper connectivity | 500 or timeout | SolrCloud coordination lost; check ZK ensemble | | Query handler metrics | Timeout or slow | Check JVM heap, merge activity, OS I/O | | SSL certificate | < 30 days to expiry | Renew certificate |

Alert after: 1 failure for core ping and ZooKeeper monitors. 2 failures for replica health (brief recovering states during rolling restarts are normal).

Common Solr Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Solr JVM crash or OOM | Core ping monitor; alert in < 60 s | | Core unloaded after config error | Core status isLoaded false | | Shard leader election stall | ZooKeeper connectivity monitor fires | | Replica sync failure after restart | Replica health monitor; recovery_failed | | Disk full stops indexing | Ping QTime spikes; indexing health fires | | ZooKeeper ensemble lost | ZooKeeper admin endpoint returns 500 | | Query routing fails (NRT lag) | Query handler latency sentinel fires | | SSL certificate expires | SSL monitor alerts at 30-day threshold | | Network partition splits cluster | Multiple monitors fire simultaneously |

Solr failures are often masked by application-layer caching and fallback logic — a downed shard may serve stale cached results while the search engine itself is broken. Vigilmon's external monitoring of Solr's admin API, ZooKeeper connectivity, and replica health endpoints gives you the ground-truth view of your search infrastructure, catching degradation before users experience it.

Start monitoring Apache Solr in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles