tutorial

Monitoring Quickwit with Vigilmon

Quickwit is a cloud-native log analytics engine built for scale. When ingestion stalls or the API goes dark, logs disappear silently. Here's how to monitor Quickwit's health endpoints, gRPC port, SSL certificates, and indexing jobs with Vigilmon.

Quickwit is a cloud-native open-source full-text search and log analytics engine written in Rust, backed by S3-native storage and an Elasticsearch-compatible API. Designed for cost-efficient log ingestion at scale, it's an emerging alternative to Elasticsearch and OpenSearch for teams who want petabyte-scale search without the operational overhead. Quickwit's distributed architecture means that when a node's API goes dark or an indexing pipeline stalls, logs are silently dropped — often with no immediate user-visible error. Vigilmon lets you catch these failures at the HTTP, TCP, and heartbeat layer before they turn into blind spots in your observability stack.

What You'll Set Up

  • Liveness monitor for the /api/v1/_health/livez endpoint
  • Readiness monitor for the /api/v1/_health/readyz endpoint
  • Index management API availability check
  • TCP monitor for the Quickwit gRPC port (7280)
  • SSL certificate expiry alerts for HTTPS-proxied deployments
  • Heartbeat monitor for scheduled indexing pipelines

Prerequisites

  • Quickwit running and accessible (default REST API port: 7280)
  • At least one index configured
  • A free Vigilmon account

Step 1: Monitor the Liveness Endpoint

Quickwit exposes /api/v1/_health/livez as its Kubernetes-style liveness probe. A 200 OK response means the Quickwit process is alive and the HTTP server is responding; a failure means the process has crashed or the HTTP layer is broken.

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: http://YOUR_HOST:7280/api/v1/_health/livez
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

If Quickwit is deployed behind a reverse proxy (nginx, Caddy, Traefik), use the proxy's URL with HTTPS instead. The liveness endpoint has no authentication — it's intentionally public for health checks.

Step 2: Monitor the Readiness Endpoint

The readiness endpoint /api/v1/_health/readyz goes further than liveness — it checks whether Quickwit's internal services (metastore, storage, indexing) are ready to serve traffic. A node can be alive but not ready if object storage is unreachable or the metastore connection has failed.

  1. Add Monitor → HTTP / HTTPS.
  2. URL: http://YOUR_HOST:7280/api/v1/_health/readyz
  3. Set Expected HTTP status to 200.
  4. Set Check interval to 2 minutes.
  5. Click Save.

When readiness fails but liveness succeeds, the Quickwit process is running but cannot serve queries or accept ingestion — logs are being dropped silently. This is the most important monitor in this stack.

Step 3: Check the Index Management API

The /api/v1/indexes endpoint lists all configured indexes. A successful JSON response confirms that the REST API is fully functional and the metastore is reachable.

  1. Add Monitor → HTTP / HTTPS.
  2. URL: http://YOUR_HOST:7280/api/v1/indexes
  3. Set Expected HTTP status to 200.
  4. Under Advanced, set Expected body contains to "indexes" (the JSON response key present in a healthy reply).
  5. Set Check interval to 2 minutes.
  6. Click Save.

Step 4: TCP Monitor for the gRPC Port

Quickwit uses port 7280 for both its REST API and gRPC inter-node communication. A TCP check confirms the port is open and accepting connections independently of the HTTP layer.

  1. Add Monitor → TCP Port.
  2. Host: YOUR_HOST, Port: 7280.
  3. Set Check interval to 2 minutes.
  4. Click Save.

In multi-node Quickwit clusters, monitor the gRPC port on each indexer and searcher node separately to detect partial cluster failures where one node goes dark while the rest remain healthy.

Step 5: SSL Certificate Expiry Alerts

Quickwit itself does not terminate TLS — HTTPS is typically handled by a reverse proxy (nginx, Caddy, Traefik) in front of Quickwit. Monitor the proxy's certificate to catch renewal failures before they interrupt log ingestion pipelines.

  1. Open the HTTP / HTTPS monitor for your Quickwit deployment (from Step 1 or 2, using the HTTPS URL).
  2. Enable Monitor SSL certificate.
  3. Set Alert when certificate expires in less than 14 days.
  4. Click Save.

If you use multiple ingestion endpoints (separate subdomains for different log sources), add a certificate monitor for each one — a single expired certificate on an ingestion endpoint silently drops all logs from that source.

Step 6: Heartbeat Monitor for Indexing Jobs

Quickwit's indexing pipelines run as internal scheduled tasks — source ingestion jobs pull from Kafka, Kinesis, or other sources on a configurable schedule. If an indexing pipeline stalls (source connection lost, storage write failure, metastore error), new logs stop appearing in search results with no external signal.

Set up a heartbeat that your indexing health check script pings:

  1. In Vigilmon, Add Monitor → Heartbeat.
  2. Name it Quickwit Indexing Pipeline.
  3. Set Expected interval to match your indexing pipeline frequency (e.g. 5 minutes).
  4. Copy the generated heartbeat ping URL.
  5. Create a monitoring script that checks pipeline status and pings the heartbeat:
#!/bin/bash
# Check that Quickwit indexing pipeline is processing documents
RESPONSE=$(curl -sf http://YOUR_HOST:7280/api/v1/_health/readyz)
if [ $? -eq 0 ]; then
  curl -fsS YOUR_HEARTBEAT_URL > /dev/null 2>&1
fi
  1. Schedule the script via cron:
*/5 * * * * /usr/local/bin/quickwit-healthcheck.sh

When the readiness check fails or the cron stops running, the heartbeat goes silent and Vigilmon alerts you — before your log retention gaps become visible in dashboards.

Step 7: Alert Channels and Grouping

  1. Go to Alert Channels in Vigilmon and configure your preferred channel (Slack, PagerDuty, or webhook).
  2. Group Quickwit monitors under a Quickwit service group for consolidated alerting.
  3. Set escalation priority higher for the readiness monitor than the liveness monitor — readiness failure means active log loss, which is more urgent than a process restart.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Liveness | /api/v1/_health/livez | Process crash, HTTP server failure | | Readiness | /api/v1/_health/readyz | Storage unreachable, metastore failure | | Index API | /api/v1/indexes | REST API failure, metastore query error | | TCP gRPC | Port 7280 | Port-level connection failure | | SSL certificate | HTTPS proxy | Certificate renewal failure | | Heartbeat | Indexing pipeline | Pipeline stall, silent log drops |

Quickwit's cloud-native design means failures often surface as silent gaps in your log data rather than loud errors. Vigilmon's external checks catch these gaps at the protocol level — ensuring that your observability stack stays observable.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles