Monitoring Allstar with Vigilmon

Allstar is an open source GitHub App from the OpenSSF (Open Source Security Foundation) that continuously audits your GitHub repositories for security policy compliance — enforcing branch protection rules, checking for SECURITY.md files, detecting binary artifacts, and restricting outside collaborators. When Allstar is healthy, violations get caught and remediated automatically. When Allstar is down, your repositories silently drift out of compliance. Vigilmon gives you visibility into every signal that matters: server health, API connectivity, scan cycle completion, and remediation job outcomes.

What You'll Set Up

• HTTP uptime monitor for the Allstar webhook server (port 3000)
• Cron heartbeat for scan cycle completion
• Webhook for remediation job outcomes and violation rate spikes
• SSL certificate alert for the webhook endpoint

Prerequisites

• Allstar self-hosted and running (Go binary + GitHub App installation)
• Allstar webhook server accessible at a public HTTPS URL on port 3000
• A free Vigilmon account

Why Monitoring Allstar Matters

Allstar's value is entirely dependent on it being alive and scanning. A silent Allstar is worse than no Allstar — you believe you're protected when you're not. Key failure modes:

• Webhook server down: GitHub can't deliver events, so configuration changes go undetected until the next full scan cycle.
• GitHub API rate limit exhaustion: Allstar reads branch protection rules, collaborator lists, and file trees across every repo. On large organizations, rate limit budget runs out silently.
• Stuck scan cycle: Allstar periodically rescans all repositories. A scan that never completes leaves violations undetected indefinitely.
• Failed remediations: When Allstar tries to fix a violation (e.g. enforce branch protection) and fails, the repo remains non-compliant with no visible signal.
• Missing installations: New repositories added to the org may not have Allstar installed, creating blind spots.

Step 1: Monitor the Allstar Webhook Server

Allstar runs a webhook receiver on port 3000 that accepts GitHub App event payloads. This is the entry point for all real-time policy checks triggered by repository configuration changes.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your Allstar webhook URL: https://allstar.yourdomain.com:3000/ (or your reverse-proxy URL).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200 (Allstar returns 200 on the root path when healthy).
6. Click Save.

If Allstar is behind a reverse proxy (nginx, Caddy), monitor the proxied URL instead:

https://allstar.yourdomain.com/health

You can add a lightweight health endpoint using Allstar's metrics exporter or a sidecar nginx stub endpoint if the root path returns a non-200 status.

Step 2: SSL Certificate Alert for the Webhook Endpoint

GitHub delivers webhook events to your Allstar server over HTTPS. An expired TLS certificate will cause GitHub to reject all webhook deliveries silently.

1. Open the Allstar HTTP monitor created in Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

This gives you three weeks to renew before GitHub stops sending events.

Step 3: Heartbeat Monitoring for Scan Cycles

Allstar's scheduled scan cycle periodically rescans every repository in the organization. This is the backstop for catching violations that weren't triggered by a webhook event (e.g. a repo was created before Allstar was installed). If the scan cycle hangs or fails, you won't know until a violation report is missed.

Add a heartbeat that Allstar emits at the end of each successful scan:

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set the expected interval to match your Allstar scan schedule (e.g. 360 minutes for a 6-hour cycle).
3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/abc123.

Emit the ping from a wrapper script or a post-scan hook in your Allstar deployment:

#!/bin/bash
# allstar-scan-wrapper.sh
./allstar --config /etc/allstar/config.yaml scan-all

# Only ping if exit code was 0
if [ $? -eq 0 ]; then
  curl -sf https://vigilmon.online/heartbeat/abc123
fi

If running Allstar as a systemd service, add an ExecStartPost or a timer-triggered scan script with the curl call appended.

If the scan takes longer than expected or crashes, the heartbeat stops arriving and Vigilmon alerts you.

Step 4: Monitor GitHub API Connectivity

Allstar makes extensive use of the GitHub API — reading branch protection rules, listing collaborators, fetching file trees. API connectivity failures or rate limit exhaustion cause scans to silently skip repositories.

Set up an external HTTP monitor to confirm GitHub API reachability from your Allstar host:

1. Add a new HTTP / HTTPS monitor in Vigilmon.
2. Set URL to https://api.github.com (returns 200 with a JSON body when reachable).
3. Set Check interval to 5 minutes.
4. Set Expected HTTP status to 200.

For rate limit awareness, add a script to your Allstar host that pings Vigilmon only when the rate limit is healthy:

#!/bin/bash
# check-github-rate-limit.sh
REMAINING=$(curl -sf -H "Authorization: token $GITHUB_TOKEN" \
  https://api.github.com/rate_limit | jq '.rate.remaining')

if [ "$REMAINING" -gt 500 ]; then
  curl -sf https://vigilmon.online/heartbeat/github-ratelimit-xyz
fi

Run this every 15 minutes via cron. If the heartbeat stops arriving, Allstar's API budget is dangerously low.

Step 5: Webhook for Remediation Job Health

Allstar's remediation actions — auto-enforcing branch protection, creating violation issues — can fail silently if the GitHub App permissions are misconfigured or the API call errors. Emit a Vigilmon webhook notification on remediation failure.

1. In Vigilmon, go to Alert Channels → Add Webhook.
2. Copy the inbound webhook URL.

Patch your Allstar deployment to call the webhook on remediation errors:

# In your Allstar error handling / log pipeline
# Trigger on log lines containing "remediation failed" or "error enforcing policy"
journalctl -u allstar -f | grep -i "remediation failed" | while read line; do
  curl -sf -X POST https://vigilmon.online/api/webhook/abc123 \
    -H "Content-Type: application/json" \
    -d "{\"message\": \"Allstar remediation failed\", \"detail\": \"$line\"}"
done

For a more structured approach, configure Allstar's structured log output (if using a JSON log formatter) and pipe it through a log aggregator that triggers on error events.

Step 6: Monitor Policy Violation Rate

A sudden spike in Allstar violation detections indicates policy drift — either a bulk configuration change across repos, a new policy was added that many repos violate, or someone made unauthorized changes to branch protection rules.

Add a cron heartbeat that represents normal operation — it only pings when the violation count per scan cycle is within expected bounds:

#!/bin/bash
# after each scan cycle
VIOLATION_COUNT=$(parse-allstar-output.sh)  # extract from logs or metrics

THRESHOLD=10  # tune to your baseline
if [ "$VIOLATION_COUNT" -le "$THRESHOLD" ]; then
  curl -sf https://vigilmon.online/heartbeat/violation-rate-xyz
fi

Set the heartbeat interval to the scan cycle duration. If violation rates spike above threshold, the heartbeat stops arriving and Vigilmon alerts you.

Step 7: Configure Alert Channels

1. Go to Alert Channels in Vigilmon.
2. Add Slack (for team-visible alerts) and email (for the security team DL).
3. Set Consecutive failures before alert to 2 on the webhook server monitor to absorb transient restarts.
4. Leave the heartbeat monitors at 1 missed ping — a missed scan cycle is always worth alerting immediately.

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP uptime | https://allstar.yourdomain.com:3000 | Webhook server down | | SSL certificate | Allstar webhook endpoint | Certificate expiry | | Scan cycle heartbeat | Heartbeat URL (6h interval) | Stuck or crashed scan cycle | | GitHub API heartbeat | https://api.github.com | API connectivity failure | | Remediation webhook | Inbound webhook | Failed policy remediation | | Violation rate heartbeat | Heartbeat URL | Policy drift spike |

Allstar gives you automated GitHub security policy enforcement — but only as long as it's actually running, scanning, and remediating. With Vigilmon watching every layer of Allstar's operation, you'll know the moment your security enforcement goes dark before any of your repositories drift out of compliance.