tutorial

Monitoring Karma Test Runner Pipelines with Vigilmon: Detect Silent CI Failures

Use Vigilmon heartbeat monitors to detect when your Karma test runner stops executing — missed browser test runs, stuck CI jobs, and pipelines that silently go dark.

Karma is the backbone of browser-based JavaScript testing for Angular projects and many other frameworks. It spins up real browsers, runs your test suite inside them, and reports results back to the terminal. But Karma only reports results when it runs. If the CI step that triggers Karma gets skipped — because of a misconfigured path filter, a broken build upstream, or an accidentally disabled scheduled job — you get silence, not failure.

This is the silent suite-failure problem. HTTP monitors can't help — there is nothing to ping. The fix is heartbeat monitoring: your Karma run pings a unique URL after every successful execution, and if Vigilmon doesn't receive that ping within the expected window, you get alerted.

This tutorial shows you how to instrument Karma test runner pipelines with Vigilmon heartbeat monitors so you know when browser tests stop running, not just when they fail.

What You'll Cover

  • Heartbeat monitoring for Karma test suites
  • Detecting browser test runs that stop executing entirely
  • A custom Karma reporter plugin that pings on completion
  • Multi-browser and multi-project monitoring
  • Integrating with CI/CD pipelines (GitHub Actions, GitLab CI)

Prerequisites

  • A project with Karma configured (Angular, plain JS, or other)
  • Node.js and npm installed
  • A free account at vigilmon.online

The Problem: What Karma's Built-In Reporting Misses

Karma gives you spec counts, browser launch feedback, and coverage reports — but only when it actually runs. It won't alert you when:

  • A webpack or esbuild preprocessing step fails silently and Karma starts with no files
  • A singleRun launch times out waiting for a browser to start in headless CI
  • A files glob pattern in karma.conf.js matches nothing (zero tests loaded)
  • A scheduled nightly run is accidentally disabled in CI
  • ChromeHeadless crashes before running a single spec and Karma exits with an ambiguous code

Vigilmon's heartbeat pattern closes these gaps by requiring an explicit "I finished successfully" signal that Karma emits only on a clean, complete run.

Step 1: Create a Heartbeat Monitor in Vigilmon

  1. Log in to Vigilmon and click New Monitor → Heartbeat.
  2. Name it something specific: Karma Browser Tests — main or Karma Angular Unit — nightly.
  3. Set the Expected interval to match your test schedule. For suites on every push, use 2 hours. For nightly suites, use 25 hours.
  4. Set the Grace period to 30 minutes for push-triggered suites, or 1 hour for scheduled suites.
  5. Save and copy the Ping URL — it looks like https://vigilmon.online/api/heartbeat/<unique-id>.

Step 2: Write a Custom Karma Reporter Plugin

Karma's reporter API exposes an onRunComplete callback that fires after all browsers finish. Wire the heartbeat ping there.

Create karma-vigilmon-reporter.js in your project root:

// karma-vigilmon-reporter.js
const https = require("https");

function VigilmonReporter(config, logger) {
  const log = logger.create("reporter.vigilmon");
  const heartbeatUrl =
    process.env.VIGILMON_HEARTBEAT_URL ||
    (config.vigilmon && config.vigilmon.heartbeatUrl) ||
    "";

  this.onRunComplete = function (browsers, results) {
    if (!heartbeatUrl) {
      return; // skip if not configured
    }

    if (results.failed > 0 || results.error || results.disconnected) {
      return; // let the window expire — Vigilmon will alert
    }

    if (results.success === 0) {
      return; // zero tests passed — do not ping; treat as failure
    }

    const url = new URL(heartbeatUrl);
    const req = https.request(
      {
        hostname: url.hostname,
        path: url.pathname + url.search,
        method: "POST",
        timeout: 10000,
      },
      (res) => {
        res.resume();
        log.info("Heartbeat ping sent to Vigilmon.");
      }
    );
    req.on("error", (err) => {
      log.warn("Vigilmon heartbeat ping failed:", err.message);
    });
    req.on("timeout", () => {
      req.destroy();
      log.warn("Vigilmon heartbeat ping timed out.");
    });
    req.end();
  };
}

VigilmonReporter.$inject = ["config", "logger"];

module.exports = {
  "reporter:vigilmon": ["type", VigilmonReporter],
};

Step 3: Register the Reporter in karma.conf.js

// karma.conf.js
const path = require("path");

module.exports = function (config) {
  config.set({
    basePath: "",
    frameworks: ["jasmine"],   // or 'mocha', 'qunit', etc.
    files: ["src/**/*.spec.js"],
    preprocessors: {},
    reporters: ["progress", "vigilmon"],

    // Optional: configure heartbeat URL in the karma config
    // (env var takes priority)
    vigilmon: {
      heartbeatUrl: process.env.VIGILMON_HEARTBEAT_URL || "",
    },

    plugins: [
      "karma-jasmine",
      "karma-chrome-launcher",
      require("./karma-vigilmon-reporter"),   // load local reporter
    ],

    browsers: ["ChromeHeadless"],
    singleRun: true,
    concurrency: Infinity,
  });
};

For Angular CLI projects, extend the Karma config via karma.conf.js in the project root (Angular generates this file automatically):

// karma.conf.js (Angular CLI)
module.exports = function (config) {
  config.set({
    // ...existing Angular CLI config...
    reporters: ["progress", "kjhtml", "vigilmon"],
    plugins: [
      require("karma-jasmine"),
      require("karma-chrome-launcher"),
      require("karma-jasmine-html-reporter"),
      require("karma-coverage"),
      require("./karma-vigilmon-reporter"),
    ],
    // ...
  });
};

Step 4: Integrate with GitHub Actions

# .github/workflows/test.yml
name: Karma Tests

on:
  push:
    branches: [main, develop]
  schedule:
    - cron: "0 3 * * *"  # nightly 03:00 UTC

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Run Karma (headless)
        env:
          VIGILMON_HEARTBEAT_URL: ${{ secrets.VIGILMON_KARMA_HEARTBEAT_URL }}
          CHROMIUM_FLAGS: "--no-sandbox --disable-setuid-sandbox"
        run: npx karma start --single-run
        # VigilmonReporter.onRunComplete fires here on success and pings Vigilmon

Store the heartbeat URL in GitHub repo → Settings → Secrets and variables → Actions → New repository secret:

  • Name: VIGILMON_KARMA_HEARTBEAT_URL
  • Value: ping URL from Step 1

If you prefer to own the ping at the workflow level rather than inside Karma:

      - name: Run Karma
        run: npx karma start --single-run

      - name: Ping Vigilmon heartbeat
        if: success()
        run: |
          curl -fsS -X POST "${{ secrets.VIGILMON_KARMA_HEARTBEAT_URL }}" \
            --max-time 10 \
            --retry 3 \
            --retry-delay 2

Step 5: Integrate with GitLab CI

# .gitlab-ci.yml
test:karma:
  image: node:20
  stage: test
  variables:
    CHROMIUM_FLAGS: "--no-sandbox --disable-setuid-sandbox"
  before_script:
    - apt-get update && apt-get install -y chromium
    - export CHROME_BIN=$(which chromium)
  script:
    - npm ci
    - VIGILMON_HEARTBEAT_URL=$VIGILMON_KARMA_HEARTBEAT_URL npx karma start --single-run
  only:
    - main
    - schedules

Set VIGILMON_KARMA_HEARTBEAT_URL in GitLab → Settings → CI/CD → Variables.

Step 6: Monitor Multiple Browser Configurations

Projects often run tests in multiple browsers (Chrome, Firefox) or in separate unit vs. integration passes. Create one heartbeat monitor per configuration:

| Configuration | Monitor name | Interval | |---|---|---| | ChromeHeadless unit tests | Karma Chrome Unit — push | 2 h | | FirefoxHeadless unit tests | Karma Firefox Unit — push | 2 h | | Nightly full browser suite | Karma Full Suite — nightly | 25 h |

Create a separate karma.conf.js per configuration and point each to its own VIGILMON_HEARTBEAT_URL environment variable:

# .github/workflows/test.yml (multi-browser)
jobs:
  chrome:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "20", cache: "npm" }
      - run: npm ci
      - run: npx karma start karma.chrome.conf.js --single-run
        env:
          VIGILMON_HEARTBEAT_URL: ${{ secrets.VIGILMON_KARMA_CHROME_HEARTBEAT_URL }}

  firefox:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "20", cache: "npm" }
      - run: npm ci
      - run: npx karma start karma.firefox.conf.js --single-run
        env:
          VIGILMON_HEARTBEAT_URL: ${{ secrets.VIGILMON_KARMA_FIREFOX_HEARTBEAT_URL }}

Step 7: Alert Channels

In Vigilmon, go to Notifications → New Channel and configure:

  • Email — immediate alert when a heartbeat is missed
  • Slack webhook — ping your #dev or #ci-alerts channel

When a Karma run stops pinging:

🔴 MISSED HEARTBEAT: Karma Chrome Unit — push
Last ping: 3 hours ago
Expected interval: 2 hours

When it recovers:

✅ HEARTBEAT RECOVERED: Karma Chrome Unit — push
Gap: 3 hours

What You're Now Catching

| Silent failure mode | How Vigilmon detects it | |---|---| | files glob matches nothing — 0 specs loaded | Reporter skips ping (zero success) → alert | | ChromeHeadless crashes before tests run | Karma exits with error, reporter skips → alert | | CI step skipped by branch/path filter | No ping within interval → alert | | Preprocessing (webpack) fails, no output | Karma errors out, reporter skips → alert | | Nightly run accidentally disabled | No ping → alert after interval | | Specs fail, results.failed > 0 | Reporter skips ping → alert after grace period |

Karma tells you which specs fail inside the browser. Vigilmon tells you when Karma stops running entirely. Together, no browser test failure — silent or loud — goes undetected.

Start monitoring your Karma pipelines today — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles