tutorial

Monitoring Ellipsis AI with Vigilmon

Ellipsis AI automates your PR reviews and changelogs — but a silent webhook failure means your team never gets those reviews. Here's how to monitor every critical layer of your self-hosted Ellipsis instance with Vigilmon.

Ellipsis AI is an AI-powered code review and developer workflow automation tool that plugs into GitHub and GitLab. When a pull request opens, Ellipsis receives a webhook, calls an LLM (GPT-4, Claude, or a local Ollama model), reads the diff, and posts a structured review comment — all in seconds. Self-hosting Ellipsis gives you full control over cost and data privacy, but it also means you're responsible for keeping the webhook receiver up, the LLM connection healthy, and the review latency inside developer expectations.

Vigilmon gives you uptime monitoring, latency tracking, and alerting across every layer of a self-hosted Ellipsis stack.

What You'll Set Up

  • HTTP uptime monitor for the Ellipsis webhook receiver (port 3000)
  • LLM provider API connectivity check (OpenAI, Claude, or Ollama)
  • GitHub/GitLab API reachability monitor
  • PR review latency heartbeat
  • TLS certificate expiry alert for the webhook endpoint

Prerequisites

  • Ellipsis AI running as a self-hosted webhook service (Node.js or Python, port 3000)
  • LLM provider configured (OpenAI GPT-4, Anthropic Claude, or local Ollama)
  • GitHub App or GitLab integration webhook pointing to your Ellipsis instance
  • A free Vigilmon account

Step 1: Monitor the Webhook Receiver

The Ellipsis webhook receiver on port 3000 is the entry point for every GitHub and GitLab event. If it goes down, pull request and push events pile up unprocessed — or are silently dropped after GitHub's retry window closes.

Add a health check endpoint to your Ellipsis service if one doesn't exist yet. For a Node.js service:

app.get('/health', (req, res) => {
  res.json({ status: 'ok', uptime: process.uptime() });
});

Then configure the Vigilmon monitor:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter https://ellipsis.yourdomain.com/health.
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

If Ellipsis doesn't expose a health route, monitor the root endpoint or a known no-auth route and expect a 200 or 401 — any response proves the process is alive.

Step 2: Check LLM Provider Connectivity

Every PR review depends on the LLM backend. An expired API key, a rate limit breach, or an Ollama container crash silently breaks all reviews — Ellipsis may accept the webhook and then fail internally.

OpenAI or Claude (external API)

Create a lightweight probe script that calls the provider's health or models endpoint:

#!/bin/bash
# probe-llm.sh — run via cron, push heartbeat on success
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  https://api.openai.com/v1/models)

if [ "$STATUS" = "200" ]; then
  curl -s "$VIGILMON_HEARTBEAT_URL" > /dev/null
fi

Schedule this every 5 minutes:

*/5 * * * * /opt/ellipsis/probe-llm.sh

Local Ollama

If you run Ollama locally, monitor its REST API directly:

  1. In Vigilmon, click Add Monitor → HTTP / HTTPS.
  2. URL: http://localhost:11434/api/tags (or the public URL if Ollama is network-accessible).
  3. Interval: 2 minutes.
  4. Expected status: 200.

Alert threshold: if the LLM endpoint misses 2 consecutive checks, page the on-call engineer — PR reviews are blocked for the entire team.

Step 3: Monitor GitHub and GitLab API Reachability

Ellipsis reads PR diffs and posts review comments via the GitHub or GitLab API. A network partition or API outage means reviews are generated but never delivered.

Add a reachability check for your Git provider:

  1. Add Monitor → HTTP / HTTPS.
  2. For GitHub: URL https://api.github.com → Expected status 200.
  3. For GitLab (self-hosted): URL https://gitlab.yourdomain.com/api/v4/version → Expected status 200.
  4. Interval: 5 minutes.

This detects external outages and internal network misrouting before your developers notice missing reviews.

Step 4: Track PR Review Latency with a Heartbeat

Developer satisfaction depends on fast reviews. If the time from webhook receipt to posted comment exceeds 5 minutes, teams start assuming Ellipsis is broken — even if it's technically up.

Instrument Ellipsis to push a Vigilmon heartbeat each time a review is successfully posted:

// After posting review comment to GitHub/GitLab
await axios.get(process.env.VIGILMON_REVIEW_HEARTBEAT_URL);

Configure the heartbeat monitor in Vigilmon:

  1. Add Monitor → Heartbeat / Cron.
  2. Name: Ellipsis Review Posted.
  3. Set Expected interval to 60 minutes (adjust to your team's PR velocity — if you open at least one PR per hour, this catches stalled processing).
  4. Copy the heartbeat URL into your VIGILMON_REVIEW_HEARTBEAT_URL environment variable.

If no review heartbeat arrives within the expected window, Vigilmon alerts you that the review pipeline is stalled — even though the webhook receiver is still responding 200.

Step 5: Monitor Webhook Delivery Reliability

GitHub and GitLab retry failed webhooks up to a point, then give up. If Ellipsis is down during a surge of PR activity, those events may be lost permanently.

Track this with a second heartbeat that fires when Ellipsis successfully processes any webhook event (not just reviews):

// On any successful webhook processing
await axios.get(process.env.VIGILMON_WEBHOOK_HEARTBEAT_URL);

Set the expected interval to your team's typical PR cadence. During business hours, if no webhook is processed for 30 minutes, investigate whether GitHub is delivering events and whether the Ellipsis queue is draining.

Step 6: Monitor the Database (If Ellipsis Persists Review History)

Some Ellipsis configurations store review history, feedback, and PR metadata in a database. Monitor write health from the application side:

// Probe script: probe-db.sh
pg_isready -h localhost -U ellipsis -d ellipsis_db && \
  curl -s "$VIGILMON_DB_HEARTBEAT_URL" > /dev/null

Schedule every 5 minutes via cron and configure a 10-minute heartbeat window in Vigilmon.

Step 7: Alert on TLS Certificate Expiry

Ellipsis must run over HTTPS — GitHub and GitLab require webhook endpoints to have a valid TLS certificate. A lapsed certificate breaks all webhook delivery instantly.

  1. In Vigilmon, open your existing HTTP monitor for https://ellipsis.yourdomain.com/health.
  2. Enable SSL certificate expiry alerting.
  3. Set alert threshold: 14 days before expiry.

This gives you two weeks to renew before GitHub starts rejecting your webhook endpoint.

Alerting Configuration

Configure Vigilmon alerts for each monitor:

| Monitor | Alert condition | Suggested channel | |---|---|---| | Webhook receiver | Down 2+ minutes | PagerDuty / Slack | | LLM provider | Heartbeat missed | Slack #engineering | | GitHub/GitLab API | Down 5+ minutes | Slack #engineering | | Review latency heartbeat | Missed window | PagerDuty | | Webhook delivery heartbeat | Missed 30 min | Slack #engineering | | TLS certificate | 14 days to expiry | Email |

For the webhook receiver and review latency heartbeat, use PagerDuty or phone alerts — a missed PR review is immediately visible to the team and erodes confidence in the tool.

Conclusion

A self-hosted Ellipsis instance has five independently-failable layers: the webhook receiver, the LLM provider, the Git provider API, the review posting pipeline, and the TLS certificate. Vigilmon monitors each one and alerts you before developers notice the gap. With all five monitors in place, you catch outages in minutes rather than discovering them when a developer complains that no review arrived on their PR.

Get started at vigilmon.online — the free plan covers all five monitors described in this tutorial.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles