Monitoring Striim with Vigilmon

Striim is a real-time data integration and streaming platform that moves data between databases, message queues, and cloud services — often for CDC (Change Data Capture) pipelines feeding analytics or replication targets. When a Striim pipeline stalls or loses its source connection, downstream systems silently fall behind. Vigilmon gives you an external health check on top of Striim's internal monitoring so you know before your data consumers do.

What You'll Set Up

• HTTP uptime monitor for the Striim web console
• Cron heartbeat monitors to detect stalled pipelines via Striim's REST API
• TCP port monitor for the Striim server port
• Alert channels for pipeline lag and connectivity failures

Prerequisites

• Striim 4.x or later installed (standalone, cluster, or Striim Cloud)
• Striim REST API accessible from your monitoring environment
• A free Vigilmon account

Step 1: Monitor the Striim Web Console

Striim exposes a web console (default port 9080) and an HTTPS port (9081). Add an HTTP monitor to confirm the console is reachable:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter the Striim console URL: https://striim.yourdomain.com:9081.
4. Set Expected HTTP status to 200.
5. Set Check interval to 1 minute.
6. Click Save.

If Striim's web server goes down (JVM crash, OOM kill, disk full), this monitor catches it within a minute.

Step 2: Monitor the Striim Server TCP Port

The Striim server listens on a configurable port for agent-to-server communication (default 9999). Add a TCP monitor:

1. Click Add Monitor → TCP Port.
2. Enter the Striim server hostname and port: striim.yourdomain.com:9999.
3. Set Check interval to 1 minute.
4. Click Save.

A TCP check catches server-level failures faster than an HTTP probe, since the web layer may still serve cached responses while the Striim core is unhealthy.

Step 3: Poll Striim Pipeline Status via REST API

Striim exposes a REST API at /api/v2.0/pipeline for querying pipeline state. Set up a polling script that checks pipeline status and pings a Vigilmon heartbeat only when all critical pipelines are running.

Get a Striim auth token

curl -s -X POST https://striim.yourdomain.com:9081/api/login \
  -H "Content-Type: application/json" \
  -d '{"userid": "admin", "password": "YOUR_PASSWORD"}'
# Returns: {"token": "..."}

Check pipeline status and ping Vigilmon

#!/bin/bash
STRIIM_HOST="https://striim.yourdomain.com:9081"
STRIIM_TOKEN="your-token-here"
HEARTBEAT_URL="https://vigilmon.online/heartbeat/abc123"
PIPELINES=("orders-cdc-pipeline" "inventory-sync-pipeline")

ALL_RUNNING=true

for pipeline in "${PIPELINES[@]}"; do
  STATUS=$(curl -s \
    -H "Authorization: $STRIIM_TOKEN" \
    "$STRIIM_HOST/api/v2.0/pipeline/$pipeline/status" \
    | jq -r '.status')

  if [ "$STATUS" != "RUNNING" ]; then
    echo "Pipeline $pipeline is in status: $STATUS"
    ALL_RUNNING=false
  fi
done

if [ "$ALL_RUNNING" = true ]; then
  curl -s "$HEARTBEAT_URL"
fi

Run this script as a cron job at an interval matching your Vigilmon heartbeat expectation:

*/5 * * * * /opt/scripts/check-striim-pipelines.sh >> /var/log/striim-check.log 2>&1

In Vigilmon, set the heartbeat monitor's expected interval to 10 minutes (giving two missed checks before an alert fires).

Step 4: Monitor Pipeline Lag with a Custom Endpoint

Striim tracks metrics like events-per-second and source-to-target lag. Deploy a lightweight HTTP endpoint (on the same host or a Lambda function) that queries Striim metrics and returns 503 when lag exceeds your threshold:

import requests
import os
from flask import Flask, jsonify

app = Flask(__name__)

STRIIM_HOST = os.environ["STRIIM_HOST"]
STRIIM_TOKEN = os.environ["STRIIM_TOKEN"]
LAG_THRESHOLD_SECONDS = 60

@app.route("/health/striim")
def striim_health():
    try:
        resp = requests.get(
            f"{STRIIM_HOST}/api/v2.0/metrics/pipeline",
            headers={"Authorization": STRIIM_TOKEN},
            timeout=5,
        )
        metrics = resp.json()
        max_lag = max(
            m.get("lagSeconds", 0) for m in metrics.get("pipelines", [])
        )
        if max_lag > LAG_THRESHOLD_SECONDS:
            return jsonify({"status": "lagging", "lag_seconds": max_lag}), 503
        return jsonify({"status": "ok", "max_lag_seconds": max_lag}), 200
    except Exception as e:
        return jsonify({"status": "error", "error": str(e)}), 503

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Add a Vigilmon HTTP monitor pointing to https://yourhost.com/health/striim with expected status 200. Any lag over 60 seconds returns a 503 and Vigilmon fires an alert.

Step 5: Alert on Source Connectivity Loss

CDC pipelines depend on the source database connection. Add a separate TCP monitor for each source database:

1. Click Add Monitor → TCP Port.
2. Enter the source database host and port (e.g., postgres.internal:5432 for PostgreSQL).
3. Set Check interval to 1 minute.
4. Name it clearly: striim-source-postgres.
5. Click Save.

If the source DB becomes unreachable, Striim's pipelines will stall within seconds. The TCP monitor catches the database outage independently, so you know whether to fix the source or the pipeline.

Step 6: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, PagerDuty, or email.
2. For CDC pipelines feeding production reports, set Consecutive failures before alert to 1 — even one missed heartbeat means data lag is already growing.
3. Use Maintenance windows during planned Striim upgrades or source database maintenance:

curl -X POST https://vigilmon.online/api/maintenance \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{"monitor_id": "abc123", "duration_minutes": 30}'

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP uptime | Striim web console (:9081) | JVM crash, OOM kill, process down | | TCP port | Striim server port (:9999) | Core server failure | | Cron heartbeat | Pipeline status cron script | Stalled or failed pipeline | | HTTP endpoint | Lag health API | Pipeline lag exceeding threshold | | TCP port | Source database host | Source connectivity loss |

Striim's internal dashboard shows you everything when you're looking at it. Vigilmon makes sure you hear about it when you're not — covering the Striim server, each critical pipeline, and the source databases that feed them all from a single external vantage point.