tutorial

Monitoring ReviewPad with Vigilmon

ReviewPad automates PR labeling, reviewer assignment, and merge rule enforcement — but a silent webhook failure means none of that automation runs. Here's how to monitor your self-hosted ReviewPad instance with Vigilmon.

ReviewPad is an open-source pull request automation and AI-powered code review tool. You define workflows in a .reviewpad.yml file inside each repository — automatically labeling PRs by size, requesting the right reviewers based on changed files, blocking merges until review requirements are met, and running AI code review via OpenAI or Claude. ReviewPad runs as a self-hosted GitHub App or GitLab integration: a Go binary webhook server on port 3000.

When ReviewPad is healthy, it's invisible — PRs flow through automation and your team never thinks about it. When it breaks, the failure mode is equally invisible: PRs open, nothing happens, and reviewers wonder why the bot didn't respond. Without monitoring, you discover the outage hours later when a developer asks why their PR is missing its size label.

Vigilmon gives you uptime monitoring, heartbeat checks, and alerting for every critical layer of a self-hosted ReviewPad deployment.

What You'll Set Up

  • HTTP uptime monitor for the ReviewPad webhook server (port 3000)
  • GitHub/GitLab API connectivity check
  • LLM provider connectivity check (for AI code review)
  • PR processing queue heartbeat
  • TLS certificate expiry alert for the webhook endpoint

Prerequisites

  • ReviewPad running as a Go binary webhook server on port 3000
  • GitHub App or GitLab integration configured with a webhook pointing to your ReviewPad server
  • .reviewpad.yml committed to repositories you want automated
  • LLM provider configured (OpenAI or Anthropic Claude) if using AI code review
  • A free Vigilmon account

Step 1: Monitor the Webhook Server

The webhook server on port 3000 is the entry point for every GitHub and GitLab event. If it goes down, all PR automation stops: labels don't get applied, reviewers don't get assigned, merge rules don't get enforced. GitHub and GitLab retry failed webhooks for a few hours, then give up — events are permanently lost.

ReviewPad exposes a health endpoint at /health by default. Configure the Vigilmon monitor:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter https://reviewpad.yourdomain.com/health.
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

If your ReviewPad instance doesn't expose a health route, monitor the root path and accept any response (including 401 or 404 — any HTTP response means the process is alive).

Alert immediately on 2 consecutive failures — ReviewPad downtime during business hours means all PR activity proceeds without automation for the duration of the outage.

Step 2: Check GitHub and GitLab API Connectivity

ReviewPad reads PR metadata and diffs, posts review comments, assigns reviewers, and applies labels via the GitHub or GitLab API. A network partition, token expiry, or API outage means ReviewPad receives webhook events, processes them, and then silently fails to post the result.

GitHub API

  1. Add Monitor → HTTP / HTTPS.
  2. URL: https://api.github.com.
  3. Expected status: 200.
  4. Interval: 5 minutes.

GitLab API (self-hosted)

  1. Add Monitor → HTTP / HTTPS.
  2. URL: https://gitlab.yourdomain.com/api/v4/version.
  3. Expected status: 200.
  4. Interval: 5 minutes.

Also probe your GitHub App or GitLab token validity regularly. An expired token produces 401 errors on every API call — ReviewPad may log these internally without surfacing them visibly:

#!/bin/bash
# probe-github-token.sh
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: token $REVIEWPAD_GITHUB_TOKEN" \
  https://api.github.com/user)

[ "$STATUS" = "200" ] && curl -s "$VIGILMON_GITHUB_TOKEN_HEARTBEAT_URL"

Schedule every hour and create a 2-hour heartbeat window.

Step 3: Check LLM Provider Connectivity (AI Code Review)

If you've enabled ReviewPad's AI code review feature, every PR triggers a call to OpenAI or Claude to generate review comments. An expired API key or rate limit breach means AI reviews stop silently — the automation runs but produces no output.

OpenAI

#!/bin/bash
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  https://api.openai.com/v1/models)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_LLM_HEARTBEAT_URL"

Anthropic Claude

#!/bin/bash
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  https://api.anthropic.com/v1/models)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_LLM_HEARTBEAT_URL"

Schedule every 5 minutes and create a 10-minute heartbeat window. Alert on miss via Slack — AI review failure is less urgent than a webhook server outage, but still impacts PR quality.

Step 4: Track PR Processing Health with a Heartbeat

ReviewPad's processing queue — the internal buffer of received webhook events awaiting processing — can stall under high PR volume, resource contention, or internal errors. Stalled processing means events sit in the queue unprocessed while ReviewPad reports itself as healthy.

Instrument ReviewPad to push a Vigilmon heartbeat each time it successfully completes processing a PR event:

If you have access to ReviewPad's source or can wrap the binary, add a heartbeat call after each processed event. Alternatively, parse the ReviewPad log for success entries:

#!/bin/bash
# probe-reviewpad-processing.sh
# Push heartbeat if a PR event was processed in the last 5 minutes
RECENT=$(journalctl -u reviewpad --since "5 minutes ago" | grep -c "PR processed successfully")
[ "$RECENT" -gt 0 ] && curl -s "$VIGILMON_PROCESSING_HEARTBEAT_URL"

Schedule every 5 minutes during business hours. Set the heartbeat window to 30 minutes — on a quiet codebase, a 30-minute gap without a processed PR event is a reasonable silence. On an active team, reduce to 10 minutes.

This heartbeat catches queue stalls that the health endpoint can't detect: ReviewPad is running and accepting webhooks, but nothing is making it through to completion.

Step 5: Validate .reviewpad.yml Configuration Health

ReviewPad reads .reviewpad.yml from each repository on every PR event. A syntax error introduced in a recent commit causes ReviewPad to silently skip workflow execution for that repository — no labels, no reviewer assignments, no merge rules enforced.

Add a configuration validation step to your CI pipeline for any repository using ReviewPad:

# .github/workflows/validate-reviewpad.yml
name: Validate ReviewPad Config
on:
  push:
    paths:
      - '.reviewpad.yml'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Validate reviewpad.yml
        run: |
          # Use the ReviewPad CLI to validate config
          reviewpad validate .reviewpad.yml
      - name: Notify Vigilmon on success
        if: success()
        run: curl -s "${{ secrets.VIGILMON_CONFIG_HEARTBEAT_URL }}"

Create a Vigilmon heartbeat with a 7-day window — config validation should run on every .reviewpad.yml change, and at minimum weekly on main branch commits.

Step 6: Monitor Reviewer Assignment Accuracy

ReviewPad's automatic reviewer assignment is a core workflow feature. Silent "no reviewer found" errors — caused by misconfigured team mappings, empty codeowner files, or inactive GitHub users — mean PRs sit without reviewers and review SLAs are silently violated.

Parse ReviewPad logs for assignment errors:

#!/bin/bash
# check-reviewer-errors.sh
# Count reviewer assignment failures in the last hour
FAILURES=$(journalctl -u reviewpad --since "1 hour ago" | grep -c "no reviewer found\|reviewer assignment failed")

if [ "$FAILURES" -eq 0 ]; then
  curl -s "$VIGILMON_REVIEWER_HEARTBEAT_URL"
fi

Schedule hourly. A missed heartbeat means reviewer assignment failed at least once in the last hour. On large teams with high PR velocity, set the threshold higher (e.g., fewer than 3 failures per hour) to avoid noise from edge cases.

Step 7: Monitor Merge Rule Enforcement

ReviewPad enforces merge blockers — rules that prevent merging until review requirements, CI checks, or approval thresholds are met. Rule evaluation failures can produce false positive merge blocks (blocking valid PRs) or false negatives (allowing PRs through that should be blocked).

Track rule evaluation health from the log:

#!/bin/bash
# probe-merge-rules.sh
EVAL_ERRORS=$(journalctl -u reviewpad --since "1 hour ago" | grep -c "rule evaluation error\|merge rule failed")
[ "$EVAL_ERRORS" -eq 0 ] && curl -s "$VIGILMON_MERGERULES_HEARTBEAT_URL"

Schedule hourly and set a 2-hour heartbeat window.

Step 8: Alert on TLS Certificate Expiry

ReviewPad must run over HTTPS — GitHub and GitLab require all webhook endpoints to have a valid TLS certificate. A lapsed certificate breaks all webhook delivery immediately and silently: no errors in ReviewPad, just GitHub quietly failing to deliver events.

  1. Open your existing HTTP monitor for https://reviewpad.yourdomain.com/health.
  2. Enable SSL certificate expiry alerting.
  3. Set alert threshold: 14 days before expiry.

Fourteen days gives you two weekly check-ins to renew before the deadline.

Alerting Configuration

| Monitor | Alert condition | Suggested channel | |---|---|---| | Webhook server | Down 2+ minutes | PagerDuty / Slack #eng | | GitHub/GitLab API | Down 10+ minutes | Slack #eng | | GitHub token validity | Heartbeat missed 2 hr | Slack #eng | | LLM provider | Heartbeat missed 10 min | Slack #eng | | PR processing | Heartbeat missed window | Slack #eng | | Config validation | Heartbeat missed 7 days | Email | | Reviewer assignment | Heartbeat missed 2 hr | Slack #eng | | Merge rule enforcement | Heartbeat missed 2 hr | Slack #eng | | TLS certificate | 14 days to expiry | Email |

The webhook server and PR processing heartbeats are the two most critical monitors. Route both to PagerDuty or a high-visibility Slack channel — both represent complete automation breakdowns.

Conclusion

A self-hosted ReviewPad instance has eight independently-failable layers between a developer opening a PR and the automation firing correctly. Vigilmon monitors each one: the webhook server, the Git provider API, the GitHub token, the LLM backend, the processing queue, the config validity, the reviewer assignment logic, and the TLS certificate. With all monitors in place, you catch ReviewPad failures within minutes rather than when a developer asks why their PR never got labeled.

Get started at vigilmon.online — the free plan covers the core monitors in this tutorial.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles