tutorial

Monitoring OpenSSF Scorecard in Your CI Pipeline with Vigilmon

OpenSSF Scorecard automates security scoring for open-source projects — but scores decay as code changes. Here's how to monitor your Scorecard pipeline health, track score regressions, and alert on failing security checks with Vigilmon.

OpenSSF Scorecard is an automated security assessment tool that scores open-source projects across dozens of checks: branch protection, CI/CD configuration, dependency update tooling, signed releases, and more. A high Scorecard score signals to users and downstream consumers that your project follows security best practices. But scores are not static — a new commit can disable branch protection, remove SAST, or introduce a new dependency without a pinned version, and your score quietly drops. Vigilmon keeps your Scorecard pipeline visible: heartbeat monitors confirm your weekly scoring runs complete, webhook integrations alert on score drops, and HTTP monitors verify the tooling infrastructure itself is reachable.

What You'll Set Up

  • Cron heartbeat monitors for your Scorecard CI pipeline runs
  • HTTP monitors for the Scorecard API and results dashboard
  • Webhook-based alerting when score thresholds are crossed
  • SSL certificate monitoring for any Scorecard-adjacent endpoints you operate

Prerequisites

  • A GitHub repository with OpenSSF Scorecard already configured (via GitHub Actions or the scorecard CLI)
  • Scorecard results either published to the OSSF API or stored as CI artifacts
  • A free Vigilmon account

Step 1: Understand What Needs Monitoring

OpenSSF Scorecard has two failure modes you need to detect:

  1. The Scorecard job itself fails — the GitHub Actions workflow crashes, times out, or is accidentally deleted. You get no score at all, and no one notices until a downstream consumer or security audit flags the stale result.
  2. The score drops — the job runs successfully but a check regresses (e.g., branch protection is relaxed, a dependency goes unpinned, or a new maintainer disables signed commits). The job "passes" from a CI perspective but your security posture degrades.

Vigilmon addresses the first failure mode with heartbeat monitoring. The second requires scripting a score comparison into your pipeline and using Vigilmon's webhook capability to fire alerts.

Step 2: Add a Heartbeat to Your Scorecard Workflow

If you run Scorecard via GitHub Actions, add a Vigilmon heartbeat ping as the final step in your workflow:

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set Expected ping interval to match your Scorecard schedule — 10080 minutes (7 days) for a weekly run, or 1440 minutes for a daily run.
  3. Copy the heartbeat URL.

Add the ping to your workflow:

# .github/workflows/scorecard.yml
name: OpenSSF Scorecard
on:
  schedule:
    - cron: '0 6 * * 1'   # Every Monday at 06:00 UTC
  push:
    branches: [main]

jobs:
  analysis:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          persist-credentials: false

      - name: Run Scorecard
        uses: ossf/scorecard-action@v2.3.1
        with:
          results_file: scorecard-results.json
          results_format: json
          publish_results: true
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -s https://vigilmon.online/heartbeat/YOUR_ID

The if: success() condition is critical — the heartbeat only fires when the Scorecard analysis completes without errors. If the job is cancelled, times out, or throws an error, the ping never fires, and Vigilmon alerts you after the expected interval passes.

Step 3: Alert on Score Regressions

A heartbeat confirms the job ran. A score comparison confirms the result is acceptable. Add a score threshold check to your workflow:

      - name: Check score threshold
        run: |
          SCORE=$(cat scorecard-results.json | jq '.score')
          THRESHOLD=7.0
          echo "Current score: $SCORE"
          if (( $(echo "$SCORE < $THRESHOLD" | bc -l) )); then
            echo "Score $SCORE is below threshold $THRESHOLD"
            # Fire Vigilmon webhook alert
            curl -s -X POST https://vigilmon.online/api/alerts \
              -H "Authorization: Bearer ${{ secrets.VIGILMON_API_KEY }}" \
              -H "Content-Type: application/json" \
              -d "{\"message\": \"Scorecard score dropped to $SCORE (threshold: $THRESHOLD)\", \"severity\": \"high\"}"
            exit 1
          fi
          echo "Score $SCORE meets threshold"

Set your VIGILMON_API_KEY as a GitHub Actions secret. When the score drops below your threshold, the step posts an alert to Vigilmon and exits with code 1, which fails the workflow and triggers normal GitHub notification mechanisms as well.

For projects with specific compliance requirements, check individual Scorecard checks rather than the aggregate score:

# Alert if the "Branch-Protection" check scores below 8
BP_SCORE=$(cat scorecard-results.json | jq '.checks[] | select(.name == "Branch-Protection") | .score')
if (( $(echo "$BP_SCORE < 8" | bc -l) )); then
  echo "Branch protection score too low: $BP_SCORE"
  exit 1
fi

Step 4: Monitor the Scorecard Results API

If you publish Scorecard results to the public OSSF API (api.securityscorecards.dev), you can monitor the availability of your project's score endpoint. This confirms the results are accessible to downstream consumers:

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter https://api.securityscorecards.dev/projects/github.com/YOUR_ORG/YOUR_REPO.
  3. Set Expected HTTP status to 200.
  4. Set Check interval to 60 minutes.
  5. Click Save.

A 60-minute check interval is appropriate here — the API is external and changes slowly. This monitor catches scenarios where the OSSF API is down or your project's results have been removed (which can happen if the repository is renamed or made private).

Step 5: SSL Certificate Monitoring for Scorecard Tooling

If your organization runs an internal Scorecard instance or self-hosts any part of the scoring infrastructure, add SSL certificate monitoring:

  1. Open the HTTP monitor for your internal Scorecard endpoint.
  2. Enable Monitor SSL certificate under the SSL section.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

For the public OSSF API endpoint, you can add SSL monitoring there too — a certificate expiry on the public API would prevent scorecard-action from uploading results and break the heartbeat chain you set up in Step 2.

Step 6: Track Score History with a Custom Metrics Endpoint

For teams that want to track score trends over time without a full Prometheus stack, a lightweight approach is to write score data to a simple endpoint and monitor that endpoint:

# score_tracker.py — runs at end of Scorecard CI job
import json, os, requests

with open('scorecard-results.json') as f:
    data = json.load(f)

score = data['score']
repo = data['repo']['name']

# Push to your metrics collector
requests.post(
    'https://metrics.yourdomain.com/scores',
    json={'repo': repo, 'score': score, 'date': os.environ['GITHUB_RUN_ID']},
    headers={'Authorization': f"Bearer {os.environ['METRICS_API_KEY']}"}
)

Then monitor your metrics collector with Vigilmon:

  1. Click Add Monitor → HTTP / HTTPS.
  2. Enter https://metrics.yourdomain.com/healthz.
  3. Set Expected HTTP status to 200.
  4. Click Save.

This gives you a running scorecard score history without depending on third-party dashboards.

Step 7: Configure Alert Routing

Scorecard alerts should reach your security team, not just your DevOps on-call rotation. Set up a dedicated alert channel:

  1. Go to Alert Channels in Vigilmon and add a Slack channel dedicated to security — e.g. #security-alerts.
  2. Create a separate channel group for Scorecard monitors and route them exclusively to #security-alerts.
  3. For score regression webhook alerts (from Step 3), also route to PagerDuty if the regression is in a security-critical check like Token-Permissions or Vulnerabilities.

Set Consecutive failures before alert to 1 for Scorecard heartbeat monitors — a single missed weekly run should alert immediately, not after two misses.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Cron heartbeat | GitHub Actions Scorecard workflow | Workflow crash, timeout, disabled workflow | | HTTP API check | api.securityscorecards.dev/projects/... | Public API down, results missing | | Score threshold | Inline workflow check + webhook | Score regression, failing security check | | SSL certificate | Internal Scorecard tooling endpoints | Certificate expiry blocking uploads | | HTTP metrics | Internal score tracker /healthz | Score history collector down |

OpenSSF Scorecard gives your project a security posture score — but a score is only useful if you know when it changes. Vigilmon keeps your Scorecard pipeline alive and your security team informed: heartbeats confirm the scoring job ran, webhook alerts surface regressions before they become security findings in an audit, and HTTP monitors verify the tooling is reachable. Together, they turn a weekly batch job into a continuous security signal.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles