How to Monitor ExCoveralls with Vigilmon

ExCoveralls is the standard code coverage reporting tool for Elixir projects. It hooks into ExUnit, measures which lines your tests exercise, and uploads the results to Coveralls.io or your CI coverage dashboard. When ExCoveralls stops reporting — because a CI job fails, a token expires, or a test suite takes too long — your coverage dashboard goes dark. Developers merge code without coverage feedback, regressions slip through, and nobody notices until the problem is already in production.

Vigilmon adds the safety net: heartbeat monitors that fire when your CI coverage pipeline runs successfully, HTTP monitors for your Coveralls dashboard, and Slack alerts when coverage reporting breaks or coverage drops below threshold.

This tutorial covers:

• Heartbeat monitoring for your CI coverage pipeline
• HTTP monitoring for Coveralls.io availability
• Alerting on coverage report failures and threshold drops
• Integrating Vigilmon into GitHub Actions and local CI workflows

What to Monitor in an ExCoveralls Pipeline

| Layer | What it is | How to monitor | |---|---|---| | CI coverage job | The mix coveralls.github or mix coveralls.post step | Heartbeat after successful upload | | Coveralls.io service | The external dashboard where reports land | HTTP uptime check | | Coverage threshold | Whether coverage stays above your minimum | Heartbeat only on passing threshold | | Nightly full-suite run | Scheduled comprehensive test run | Heartbeat with 24-hour grace period |

Step 1: Add ExCoveralls to your project

If you haven't already, add ExCoveralls:

# mix.exs
defp deps do
  [
    {:excoveralls, "~> 0.18", only: :test}
  ]
end

# mix.exs — project config
def project do
  [
    # ...
    test_coverage: [tool: ExCoveralls],
    preferred_cli_env: [
      coveralls: :test,
      "coveralls.detail": :test,
      "coveralls.post": :test,
      "coveralls.html": :test,
      "coveralls.github": :test
    ]
  ]
end

Add a coveralls.json to configure thresholds:

{
  "coverage_options": {
    "minimum_coverage": 80,
    "treat_no_relevant_lines_as_covered": true
  },
  "skip_files": [
    "lib/my_app_web/telemetry.ex",
    "test/"
  ]
}

Step 2: Add a heartbeat to your CI pipeline

The most important signal is whether your coverage report actually uploaded successfully. Add a Vigilmon heartbeat ping at the end of the coverage step in your CI workflow:

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main, develop]
  pull_request:

env:
  MIX_ENV: test

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          elixir-version: "1.16"
          otp-version: "26"

      - name: Install dependencies
        run: mix deps.get

      - name: Run tests with coverage
        env:
          COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
        run: mix coveralls.github

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS --retry 3 "${{ secrets.VIGILMON_COVERAGE_HEARTBEAT_URL }}"

The if: success() guard ensures the heartbeat only fires when mix coveralls.github exits 0 — meaning tests passed and the coverage report uploaded successfully. A failed upload, a flaky test, or a coverage threshold violation will suppress the ping and trigger your alert.

Add the secret in GitHub → Settings → Secrets → VIGILMON_COVERAGE_HEARTBEAT_URL.

In Vigilmon:

1. Click New Monitor → Heartbeat.
2. Name it ExCoveralls CI upload.
3. Set interval to 24 hours if you only run on merges to main, or 1 hour for active PRs.
4. Copy the heartbeat URL into your GitHub secret.

Step 3: Monitor Coveralls.io uptime

Your coverage reports are useless if Coveralls.io itself is unreachable. Add an HTTP monitor:

1. Sign in at vigilmon.online.
2. Click New Monitor → HTTP.
3. URL: https://coveralls.io.
4. Check interval: 5 minutes.
5. Expected status: 200.

If Coveralls.io goes down mid-deployment, you'll know before your team starts wondering why coverage stats aren't updating.

Step 4: Enforce coverage thresholds in CI with a Vigilmon signal

ExCoveralls returns a non-zero exit code when coverage drops below minimum_coverage. Wire this into a separate heartbeat so you have an independent signal for threshold violations:

#!/bin/bash
# scripts/coverage_check.sh
set -e

echo "Running test suite with coverage..."
if mix coveralls.html; then
  echo "Coverage threshold met — pinging Vigilmon"
  curl -fsS --retry 3 "${VIGILMON_THRESHOLD_HEARTBEAT_URL}"
else
  echo "Coverage threshold FAILED — suppressing heartbeat"
  exit 1
fi

Use a separate Vigilmon heartbeat monitor for threshold health vs. upload health — they fail independently:

| Monitor | Fires when | Grace period | |---|---|---| | ExCoveralls upload | Report successfully uploaded | 24 h (or per-pipeline interval) | | Coverage threshold | Coverage ≥ minimum_coverage | 24 h (or per-merge interval) |

Separating them means a flaky upload doesn't mask a real coverage regression, and vice versa.

Step 5: Local development heartbeat for nightly full-suite runs

If you run a more comprehensive test suite nightly on a dedicated machine or runner, add a heartbeat with a longer grace period:

#!/bin/bash
# cron: 0 2 * * * /path/to/scripts/nightly_coverage.sh

set -e
cd /opt/my_app

mix test --cover 2>&1 | tee /var/log/coverage.log

# Only ping on full success
curl -fsS "${VIGILMON_NIGHTLY_HEARTBEAT_URL}"

Set the Vigilmon heartbeat grace period to 25 hours so a single missed night triggers an alert, but a one-hour delay (e.g. slow test suite) does not.

Step 6: Configure alerts

In Vigilmon go to Alerts and configure notification channels:

• Slack (#engineering or #ci-alerts) — coverage pipeline failures
• Email — lead developer or QA owner
• GitHub Issues (via webhook) — auto-create an issue when coverage drops

Recommended alert policies for ExCoveralls:

| Monitor | Condition | Action | |---|---|---| | CI upload heartbeat | Missed ≥ 1× | Alert engineering — CI broken | | Coverage threshold heartbeat | Missed ≥ 1× | Alert lead dev — coverage regressed | | Coveralls.io HTTP | Non-200 or timeout | Warning — external service issue | | Nightly run heartbeat | Missed ≥ 1× | Alert QA team |

Key Metrics to Watch

| Metric | Threshold | Meaning | |---|---|---| | CI heartbeat | 0 missed per pipeline run | Coverage upload succeeded | | Coverage threshold | 0 missed per merge | Coverage ≥ minimum_coverage | | Coveralls.io HTTP | Must be 200 | Dashboard accessible | | Nightly heartbeat | < 1 missed per week | Full suite still running | | Upload latency | < 60 s | No token or API issues |

Why Monitor ExCoveralls?

Coverage reporting fails silently:

• Token expiry — Coveralls repo tokens expire and mix coveralls.github silently exits 1
• CI budget limits — test jobs get cancelled mid-run without alerting the team
• Threshold drift — new code ships without tests, coverage drops below minimum, nobody notices
• External dependency — Coveralls.io itself has incidents; your pipeline looks broken when it isn't

Vigilmon's heartbeat pattern is the right fit: every successful coverage pipeline run pings the heartbeat. When pings stop — for any reason — you get an alert before the next code review depends on stale data.

Conclusion

ExCoveralls keeps your Elixir test coverage visible. Vigilmon keeps ExCoveralls itself under watch. With heartbeat monitors for your CI coverage jobs, HTTP monitoring for Coveralls.io, and threshold-aware alerts, you always know whether your coverage pipeline is healthy — not just whether your tests pass.

Get started free at vigilmon.online.