Monitoring Aviator with Vigilmon

Aviator solves the "mergeability problem" by serializing PRs through a smart merge queue — batching compatible changes, running CI on the combined set, and merging only when everything passes. But if Aviator's API server goes down, the queue stalls, or GitHub connectivity is interrupted, your entire merge pipeline freezes. Vigilmon monitors every critical component of your self-hosted Aviator deployment so you know the moment the merge queue stops moving.

What You'll Set Up

• HTTP uptime monitor for the Aviator API server (port 3000)
• SSL certificate expiry alert for the API endpoint
• TCP monitor for PostgreSQL connectivity
• Cron heartbeat for queue throughput health
• HTTP monitor for GitHub API connectivity
• Alert routing for merge pipeline failures

Prerequisites

• Self-hosted Aviator running Python/FastAPI + PostgreSQL
• Aviator API server accessible on port 3000
• A free Vigilmon account

Step 1: Monitor the Aviator API Server

The Aviator API server on port 3000 is the brain of the merge queue — it processes GitHub webhook events, manages queue state, batches PRs, and triggers merges. If this service is unreachable, all queued PRs are stuck until it recovers.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your Aviator API URL: https://aviator.yourdomain.com (or http://your-server:3000).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

Aviator exposes a health endpoint at /health — use it for a more meaningful check:

https://aviator.yourdomain.com/health

The health endpoint reports internal state including database connectivity, so a single monitor covers both the API server and its dependencies.

Step 2: SSL Certificate Expiry Alert

Aviator's API endpoint receives webhooks from GitHub — those webhook deliveries require a valid TLS certificate. An expired certificate causes GitHub to stop delivering events, silently draining your merge queue.

1. Open the HTTP monitor you created in Step 1.
2. Enable Monitor SSL certificate under the SSL section.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

GitHub webhook delivery failures due to TLS errors appear in your repository's webhook delivery log under Settings → Webhooks, but there's no push notification — Vigilmon's certificate alert is your proactive warning.

Step 3: Monitor PostgreSQL Connectivity

Aviator stores all queue state, PR metadata, batch assignments, and CI results in PostgreSQL. If the database is unavailable, Aviator cannot update queue state or record merge outcomes — the queue appears to run but makes no progress.

Add a TCP monitor for PostgreSQL:

1. Click Add Monitor → TCP Port.
2. Enter your database host and port: your-server:5432.
3. Set Check interval to 1 minute.
4. Click Save.

For deeper visibility, monitor disk space on your PostgreSQL host. Aviator stores CI results and merge history for every batch — on a busy repository this can grow quickly and fill the disk, causing silent database write failures.

Step 4: Monitor GitHub API Connectivity

Aviator makes intensive use of the GitHub API: reading PR status, checking CI results, posting merge status comments, and performing the actual merge operation. GitHub API rate limits are a real operational concern — Aviator makes many calls per merge cycle, and a busy repository can exhaust rate limits during peak hours.

Add an HTTP monitor for GitHub API availability:

1. Click Add Monitor → HTTP / HTTPS.
2. Enter https://api.github.com/meta.
3. Set Check interval to 5 minutes.
4. Click Save.

To monitor your specific rate limit headroom, add a lightweight probe that checks the X-RateLimit-Remaining header from GitHub's API. You can expose this via a small endpoint in your Aviator deployment:

import requests
from fastapi import APIRouter

router = APIRouter()

@router.get("/probes/github-rate-limit")
def github_rate_limit():
    r = requests.get(
        "https://api.github.com/rate_limit",
        headers={"Authorization": f"token {GITHUB_TOKEN}"}
    )
    data = r.json()
    remaining = data["rate"]["remaining"]
    return {"remaining": remaining, "healthy": remaining > 500}

Add a Vigilmon HTTP monitor on this endpoint that alerts when "healthy": false — giving you advance warning before rate limit exhaustion stalls the queue.

Step 5: Monitor CI System Connectivity

Aviator's merge queue depends on CI results to decide when to merge batches. If your CI system (GitHub Actions, CircleCI, or Jenkins) stops delivering status checks, Aviator's batches get stuck waiting for a result that never comes.

Add a monitor for your CI system's webhook or status endpoint:

• GitHub Actions: Monitor https://www.githubstatus.com/api/v2/status.json and alert on non-green status.
• CircleCI: Monitor https://status.circleci.com/api/v2/status.json.
• Jenkins self-hosted: Add a TCP monitor for your Jenkins port and an HTTP monitor for /login.

For detecting stuck batches specifically, add a probe to Aviator's queue status API:

# Check for batches stuck in CI for more than 60 minutes
curl -s https://aviator.yourdomain.com/api/queue/status | \
  jq '.batches[] | select(.age_minutes > 60 and .status == "pending_ci")'

Expose this check as a dedicated health endpoint and add a Vigilmon HTTP monitor that alerts when any batch is stuck beyond your SLA.

Step 6: Heartbeat for Queue Throughput

A stalled merge queue can look healthy from a service availability standpoint — the API server responds to health checks, but no PRs are actually being merged. Use a Vigilmon cron heartbeat to confirm the queue is making forward progress.

1. Click Add Monitor → Cron Heartbeat.
2. Set the expected interval to your normal queue throughput cycle — if your team merges at least one PR per hour, use 60 minutes.
3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/YOUR_ID.
4. Add a ping to Aviator's post-merge webhook handler:

import httpx

async def on_merge_success(pr_number: int):
    # Record the successful merge
    await record_merge(pr_number)
    # Ping Vigilmon heartbeat
    async with httpx.AsyncClient() as client:
        await client.get("https://vigilmon.online/heartbeat/YOUR_ID")

If no PRs are merged within the expected window, Vigilmon alerts — surfacing a stalled queue even when the API server itself appears healthy.

Step 7: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
2. For the Aviator API monitor, set Consecutive failures before alert to 2.
3. For PostgreSQL TCP, alert on 1 failure — queue state corruption starts immediately.
4. For the queue throughput heartbeat, alert on 1 missed beat — a stalled queue needs immediate attention.

Recommended Slack routing for a development team:

| Monitor | Channel | Threshold | |---|---|---| | API server (port 3000) | #dev-alerts | 2 failures | | SSL certificate | #dev-alerts | 21 days | | PostgreSQL TCP | #on-call | 1 failure | | GitHub API | #dev-alerts | 2 failures | | Queue throughput heartbeat | #dev-alerts | 1 missed beat |

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP API server | https://aviator.yourdomain.com/health | Service outage, crashes | | SSL certificate | API endpoint domain | Certificate expiry blocking webhooks | | TCP | PostgreSQL port 5432 | Queue state database outage | | HTTP | GitHub API | Rate limit exhaustion, API outage | | HTTP | CI system status | Stuck batches waiting on CI | | Cron heartbeat | Heartbeat URL | Stalled queue with no actual merges |

Aviator's value comes from keeping PRs moving — the moment the queue stalls, your engineering team loses the automation that prevents merge conflicts from piling up. With Vigilmon watching every layer of your Aviator deployment, you'll know immediately when something stops the flow.