Monitoring Release Drafter with Vigilmon

Release Drafter automatically maintains a draft release in your GitHub repository, updating it every time a pull request is merged — categorizing changes by label, building a structured changelog, and keeping your release notes current without any manual effort. When you self-host Release Drafter to avoid GitHub API rate limits and keep release metadata private, you own the webhook receiver that processes every PR merge event. If that webhook service goes down, draft releases silently stop updating — and your next release goes out with incomplete or stale notes. Vigilmon keeps your Release Drafter webhook server observable and alerts you the moment it stops processing PR events.

What You'll Set Up

• Webhook server availability monitor (port 3000)
• GitHub API connectivity and rate limit monitoring
• Draft release update health via Cron Heartbeat
• Webhook signature verification health
• PR label compliance tracking
• GitHub App installation health checks
• TLS certificate expiry alerts for the webhook endpoint

Prerequisites

• Release Drafter self-hosted GitHub App running as a Node.js service
• GitHub App installed on at least one repository
• .github/release-drafter.yml configured in each target repository
• A free Vigilmon account

Step 1: Monitor the Webhook Server

Release Drafter's self-hosted mode runs a Node.js webhook receiver (default port 3000). All GitHub PR merge events are delivered to this endpoint. When it's unreachable, draft releases freeze at the last successful update.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your webhook server URL: https://release-drafter.yourdomain.com/ or http://your-server:3000/.
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

If your Release Drafter instance exposes a dedicated health path, use it. Otherwise, the root path typically returns 200 with a brief status payload.

Add a second monitor for the webhook path itself:

1. Add Monitor → HTTP / HTTPS.
2. URL: https://release-drafter.yourdomain.com/api/github/webhooks.
3. Expected status: 200 (GET on the webhook path returns OK; POST delivers events).
4. Check interval: 1 minute.

Step 2: Add a Health Endpoint to Your Release Drafter Instance

If you're running a custom Release Drafter deployment, add a /health route for Vigilmon to probe:

// server.js — Release Drafter with custom health endpoint
const { createNodeMiddleware, createProbot } = require('probot');
const releasedrafter = require('release-drafter');
const http = require('http');
const express = require('express');

const app = express();
const probot = createProbot();

// Health check endpoint
app.get('/health', (req, res) => {
  res.json({
    status: 'ok',
    uptime: process.uptime(),
    timestamp: new Date().toISOString(),
  });
});

// Mount Probot webhook handler
app.use('/api/github', createNodeMiddleware(releasedrafter, { probot, webhooksPath: '/webhooks' }));

http.createServer(app).listen(3000, () => {
  console.log('Release Drafter webhook server running on port 3000');
});

Restart your service and point Vigilmon at https://release-drafter.yourdomain.com/health.

Step 3: Monitor GitHub API Connectivity and Rate Limits

Release Drafter calls the GitHub API on every PR merge: it reads PR metadata, labels, and author info, then creates or updates the draft release. Under a heavy PR volume, the GitHub API rate limit is the first thing to hit.

Add an HTTP monitor for the GitHub API:

1. Add Monitor → HTTP / HTTPS.
2. URL: https://api.github.com.
3. Check interval: 5 minutes.
4. Expected status: 200.

Add a rate limit check to your alerting workflow. In Release Drafter's logs, filter for the X-RateLimit-Remaining header and emit a metric:

// In your Release Drafter app handler (probot context)
app.on('pull_request.closed', async (context) => {
  if (!context.payload.pull_request.merged) return;

  // Check rate limit before processing
  const { data: rateLimit } = await context.octokit.rateLimit.get();
  const remaining = rateLimit.rate.remaining;

  if (remaining < 100) {
    console.warn(`GitHub API rate limit low: ${remaining} requests remaining`);
    // Optionally: emit to your metrics system
  }

  // Continue with draft release update...
});

Step 4: Monitor Draft Release Update Health via Heartbeat

The most important signal is whether draft releases are actually being updated. Set up a Cron Heartbeat that fires after every successful draft release update.

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Name it Release Drafter Update Heartbeat.
3. Set Expected ping interval to a window slightly longer than your busiest PR merge cadence (e.g., 1h for active repos, 24h for slower projects).
4. Copy the generated heartbeat URL.

In your Release Drafter app handler, ping the heartbeat after each successful update:

const https = require('https');

async function pingVigilmon(url) {
  return new Promise((resolve) => {
    https.get(url, resolve).on('error', () => resolve(null));
  });
}

app.on('pull_request.closed', async (context) => {
  if (!context.payload.pull_request.merged) return;

  try {
    // ... existing draft release update logic ...
    await context.octokit.repos.updateRelease({ /* ... */ });

    // Ping Vigilmon after successful update
    await pingVigilmon(process.env.VIGILMON_DRAFT_HB_URL);
    context.log.info('Draft release updated and heartbeat sent');
  } catch (error) {
    context.log.error('Failed to update draft release:', error);
    // Do NOT ping heartbeat on failure — Vigilmon will alert on timeout
  }
});

Set VIGILMON_DRAFT_HB_URL in your service environment. If a draft release update fails (GitHub API error, config parse error, or any other exception), the heartbeat silently stops and Vigilmon pages your team.

Step 5: Monitor Webhook Signature Verification

Release Drafter verifies GitHub's HMAC-SHA256 webhook signature on every incoming event. If the WEBHOOK_SECRET environment variable is wrong or rotated, all events are rejected with a 403 — and draft releases stop updating without any obvious error in GitHub's webhook delivery UI.

Add a webhook delivery health check. In your Release Drafter logs, count signature validation rejections:

// Instrument signature rejection in your Express middleware
app.use('/api/github/webhooks', (req, res, next) => {
  const originalSend = res.send.bind(res);
  res.send = (body) => {
    if (res.statusCode === 403) {
      console.error('Webhook signature rejected — check WEBHOOK_SECRET');
      // Optionally: emit metric or send alert via Vigilmon webhook
    }
    return originalSend(body);
  };
  next();
});

Alert on any 403 from the webhook path. Add a Vigilmon keyword monitor if your logs are accessible via a log aggregator URL, or route signature failures to a Vigilmon alert channel webhook directly.

Step 6: Monitor PR Label Compliance

Release Drafter categorizes PRs by label (e.g., bug, feature, documentation, breaking-change). PRs merged without required labels produce uncategorized entries in the draft release — or are silently omitted depending on your configuration.

Add a GitHub Actions workflow to enforce label compliance:

# .github/workflows/label-check.yml
name: Label compliance check

on:
  pull_request:
    types: [opened, labeled, unlabeled, synchronize]

jobs:
  check-labels:
    runs-on: ubuntu-latest
    steps:
      - name: Check required labels
        uses: actions/github-script@v7
        with:
          script: |
            const labels = context.payload.pull_request.labels.map(l => l.name);
            const required = ['bug', 'feature', 'documentation', 'breaking-change', 'chore'];
            const hasRequired = required.some(l => labels.includes(l));
            if (!hasRequired) {
              core.setFailed(`PR must have one of: ${required.join(', ')}`);
            }

Track label compliance rate as a Vigilmon keyword monitor by watching for Label compliance check failures in your CI dashboard, or export weekly label compliance metrics to a Vigilmon status page.

Step 7: Monitor GitHub App Installation Health

Release Drafter requires GitHub App installation on each target repository. If the installation is suspended or the app is uninstalled, webhook events for that repo stop arriving silently.

Add a periodic installation health check to your Release Drafter service:

const { Octokit } = require('@octokit/app');

async function checkInstallations(app) {
  try {
    const installations = await app.octokit.apps.listInstallations();
    console.log(`Release Drafter installed on ${installations.data.length} accounts`);

    for (const installation of installations.data) {
      if (installation.suspended_at) {
        console.error(`Installation ${installation.id} suspended for account: ${installation.account.login}`);
      }
    }
  } catch (error) {
    console.error('Failed to check GitHub App installations:', error);
  }
}

// Run installation health check every 30 minutes
setInterval(() => checkInstallations(probot.app), 30 * 60 * 1000);

Route suspended_at detections to a Vigilmon alert channel webhook to page your team immediately.

Step 8: Monitor TLS Certificate Expiry

Release Drafter's webhook endpoint must be reachable over HTTPS (GitHub rejects HTTP webhook URLs for App installations). A lapsed TLS certificate makes the endpoint unreachable and breaks all event delivery.

1. Add Monitor → SSL Certificate Check.
2. Enter your Release Drafter domain: release-drafter.yourdomain.com.
3. Set Alert when certificate expires in less than: 14 days.
4. Check interval: 1 day.

Vigilmon will alert you two weeks before the certificate expires — enough time to renew before GitHub starts rejecting webhook deliveries.

Step 9: Configure Alerting

In Vigilmon, go to Alert Channels and configure:

• Slack #releases — notify on webhook server downtime, draft update heartbeat misses, and signature rejection alerts.
• PagerDuty — page on-call if the webhook server is unreachable for more than 5 minutes (all PR merges are being missed).
• Email — TLS certificate expiry warnings and installation suspension alerts (lower urgency, but important to catch before they escalate).

Recommended thresholds:

• Webhook server HTTP monitor: alert after 2 consecutive failures (allow for brief restarts).
• Draft release heartbeat: alert after missed window + 30 minutes (buffer for rare quiet PR periods).
• GitHub API monitor: alert after 3 consecutive failures.
• TLS certificate: alert at 14 days remaining.

Conclusion

Release Drafter runs silently in the background — when it works, it's invisible; when it breaks, your release notes are stale and your next release ships with incomplete changelogs. Self-hosting adds reliability at the cost of operational responsibility: the webhook server, GitHub App installation, webhook secrets, and TLS certificate all need monitoring. With Vigilmon covering the webhook endpoint, a draft-update heartbeat, API connectivity, and certificate expiry, your Release Drafter instance becomes as observable as any other critical service in your pipeline.