Monitoring Self-Hosted Harness CI with Vigilmon

Harness CI Community Edition brings enterprise-grade continuous integration to your own servers — but self-hosting means you're responsible for knowing when the platform itself goes down, not just your builds. A silent Harness outage can stall every pipeline in your organization without a single notification. Vigilmon watches your Harness instance continuously, alerting you the moment a service degrades so developers keep shipping rather than troubleshooting infrastructure.

What You'll Set Up

• HTTP health monitor for the Harness CI manager service on port 3000
• Pipeline execution API probe to verify the CI engine is processing jobs
• Build delegate (runner) uptime checks via heartbeat monitoring
• SSL certificate expiry alerts for the Harness web UI
• Alerting channel configuration with noise suppression

Prerequisites

• Harness CI Community Edition installed (Docker Compose or Kubernetes)
• Harness accessible on port 3000 (default) or your configured port
• A free Vigilmon account

Step 1: Monitor the Harness Manager Health Endpoint

The Harness CI manager exposes a /api/health endpoint that returns service status. This is your primary liveness check.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter the URL: http://your-harness-host:3000/api/health
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Under Keyword match, enter "status":"running" to validate the response body, not just the status code.
7. Click Save.

The health endpoint returns a JSON payload when the manager is operational:

{
  "status": "running",
  "metaData": {},
  "correlationId": "..."
}

If the keyword check fails — for example, if the service returns 200 but with "status":"degraded" — Vigilmon alerts you even though the HTTP status appeared healthy.

Step 2: Verify Pipeline Execution API Availability

The pipeline execution service runs separately from the manager. Probe the execution API to confirm pipelines can actually be triggered:

1. Click Add Monitor → HTTP / HTTPS.
2. Enter: http://your-harness-host:3000/pipeline/api/health
3. Set Check interval to 2 minutes.
4. Set Expected HTTP status to 200.
5. Click Save.

You can also probe the gateway service if you have API gateway routing in front of Harness:

http://your-harness-host:7143/api/health

Add a separate monitor for each service endpoint you want to track independently. Isolating monitors by service lets you pinpoint which component failed when an alert fires.

Step 3: Monitor Build Delegate Uptime with Heartbeats

Harness delegates (build agents) are the workers that execute pipeline steps. If every delegate goes offline, pipelines queue indefinitely without failing — which means no error to catch with a simple HTTP probe. Use Vigilmon's cron heartbeat to detect silent delegate outages.

Set up the heartbeat monitor

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set the expected ping interval to 5 minutes.
3. Copy the heartbeat URL (e.g. https://vigilmon.online/heartbeat/abc123).

Send the ping from your delegate host

Add a cron job or a shell script on the machine running the Harness delegate that pings Vigilmon after confirming the delegate process is alive:

#!/bin/bash
# /usr/local/bin/harness-delegate-healthcheck.sh

# Check if the delegate Java process is running
if pgrep -f "delegate.jar" > /dev/null 2>&1; then
  curl -s --max-time 10 https://vigilmon.online/heartbeat/abc123
fi

Schedule this script every 5 minutes:

*/5 * * * * /usr/local/bin/harness-delegate-healthcheck.sh

If the delegate process dies between pings, the heartbeat stops arriving and Vigilmon alerts after the expected interval passes.

For Docker-based delegates

#!/bin/bash
# Check delegate container health
if docker inspect harness-delegate --format='{{.State.Status}}' 2>/dev/null | grep -q "running"; then
  curl -s --max-time 10 https://vigilmon.online/heartbeat/abc123
fi

Step 4: Monitor the Harness Database Connection

Harness CI stores pipeline state in a MongoDB or TimescaleDB backend. A database connectivity failure causes pipelines to fail non-obviously. Probe the Harness database status through the admin API:

1. Click Add Monitor → HTTP / HTTPS.
2. Enter: http://your-harness-host:3000/api/version
3. Set Check interval to 2 minutes.
4. Set Expected HTTP status to 200.
5. Click Save.

This endpoint only returns 200 when Harness can reach its backend storage. A database connectivity failure surfaces here as a degraded or error response.

Step 5: SSL Certificate Monitoring

If you've placed Harness behind an nginx or Caddy reverse proxy with TLS, SSL certificate expiry is a silent risk.

1. Open the Harness manager monitor created in Step 1.
2. Switch the URL to https://harness.yourdomain.com (your TLS-terminated domain).
3. Enable Monitor SSL certificate under the SSL section.
4. Set Alert when certificate expires in less than 21 days.
5. Click Save.

For Let's Encrypt certificates managed by Certbot, a 21-day window gives you three renewal attempts before expiry.

Step 6: Configure Alerting

Add an alert channel

1. Go to Alert Channels in Vigilmon.
2. Add Slack, email, PagerDuty, or a webhook endpoint.
3. Assign the channel to each Harness monitor you created.

Tune alert thresholds

Harness services occasionally restart during updates. Set Consecutive failures before alert to 2 on the manager and pipeline monitors to avoid single-probe false positives during rolling restarts.

Maintenance windows during upgrades

Suppress alerts automatically when upgrading Harness using the Vigilmon API:

# Open a maintenance window before upgrading
curl -X POST https://vigilmon.online/api/maintenance \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "monitor_id": "harness-manager-monitor-id",
    "duration_minutes": 30
  }'

# Run your Harness upgrade
docker compose pull && docker compose up -d

# The maintenance window expires automatically after 30 minutes

Summary

| Monitor | Target | What It Catches | |---|---|---| | Manager health | :3000/api/health | Service crash, startup failure | | Pipeline API | :3000/pipeline/api/health | Execution engine degraded | | Delegate heartbeat | Cron ping every 5 min | Build agent process death | | Version/DB endpoint | :3000/api/version | Database connectivity loss | | SSL certificate | https://harness.yourdomain.com | Certificate expiry |

Harness CI Community Edition gives your team enterprise-grade pipelines without the SaaS price tag — but that control comes with monitoring responsibility. With Vigilmon watching the manager, execution engine, delegates, and TLS certificates, your on-call team hears about Harness infrastructure problems before developers do.