tutorial

Monitoring CodeceptJS Test Suites with Vigilmon: Catch Silent E2E Failures

Use Vigilmon heartbeat monitors to detect when your CodeceptJS end-to-end tests stop running — catch scheduler failures, hung helpers, and silently skipped test suites.

Your CodeceptJS suite ran on schedule against a staging environment for weeks. Then an intern accidentally deleted the cron job that triggered it. The suite had been green, so the dashboard stayed green — but it was the green of an unmeasured system, not a tested one. Three sprints later, a critical regression in the user registration flow shipped to production.

This is why heartbeat monitoring matters for E2E tests. CodeceptJS can tell you if a test fails — but it can't tell you when it stops being invoked at all. Vigilmon fills that gap: your suite pings a unique URL after every successful run, and Vigilmon alerts you when the ping stops arriving.

What You'll Cover

  • Heartbeat monitoring for scheduled CodeceptJS runs
  • Catching CodeceptJS runners that stop triggering entirely
  • Alerting when helpers (Playwright, WebDriver, Puppeteer) fail to launch
  • Multi-environment E2E monitoring (staging and production)
  • Detecting test drift before it causes production incidents

Prerequisites

The Problem: What Standard CodeceptJS Reporting Misses

CodeceptJS has a rich plugin ecosystem and detailed reporters — but no built-in mechanism to alert you when:

  • The CI schedule or cron job that invokes CodeceptJS is paused or deleted
  • The underlying helper (Playwright, WebDriver, Puppeteer) fails to start
  • A feature flag or environment variable change causes all tests to be skipped (exit 0)
  • The test machine goes offline and jobs queue silently
  • A retried flaky test eventually passes after the CI timeout window, skipping your heartbeat window

Vigilmon's heartbeat pattern closes every one of these blind spots.

Step 1: Create a Heartbeat Monitor in Vigilmon

  1. Log in to Vigilmon and click New Monitor → Heartbeat.
  2. Name it after your suite: CodeceptJS E2E — Nightly or CodeceptJS Smoke — Staging.
  3. Set Expected interval to match your test schedule. For nightly runs, use 24 hours; for pre-deploy checks, try 4 hours.
  4. Set Grace period to 1 hour to account for helper browser startup and slow network pages.
  5. Save and copy the Ping URL — it looks like https://vigilmon.online/api/heartbeat/<unique-id>.

Step 2: Create a Custom CodeceptJS Plugin

CodeceptJS supports custom plugins via its configuration. Create a plugin that pings Vigilmon after a fully successful run:

// codecept-plugins/vigilmon.js
const https = require("https");

function pingVigilmon(url) {
  return new Promise((resolve, reject) => {
    const req = https.request(url, { method: "POST" }, (res) => {
      resolve(res.statusCode);
    });
    req.on("error", reject);
    req.setTimeout(10000, () => {
      req.destroy(new Error("Vigilmon ping timed out"));
    });
    req.end();
  });
}

module.exports = (config) => {
  const heartbeatUrl = process.env.VIGILMON_HEARTBEAT_URL;

  event.dispatcher.on(event.all.result, async (result) => {
    if (!heartbeatUrl) return;

    // Only send heartbeat when there are no failures
    const hasFailed = result.stats.failures > 0 || result.stats.pending > 0;
    if (!hasFailed) {
      try {
        await pingVigilmon(heartbeatUrl);
        output.print("Vigilmon heartbeat sent");
      } catch (e) {
        output.error(`Vigilmon heartbeat failed: ${e.message}`);
      }
    }
  });
};

The event.all.result event fires after all test suites complete. Checking stats.failures > 0 ensures the heartbeat only fires on fully clean runs.

Step 3: Register the Plugin in CodeceptJS Config

Add the plugin to your codecept.conf.js (or codecept.conf.ts):

// codecept.conf.js
const { setHeadlessWhen } = require("@codeceptjs/configure");

setHeadlessWhen(process.env.CI);

exports.config = {
  tests: "./tests/**/*.test.js",
  output: "./output",

  helpers: {
    Playwright: {
      url: process.env.APP_URL || "http://localhost:3000",
      show: false,
      browser: "chromium",
    },
  },

  plugins: {
    vigilmon: {
      require: "./codecept-plugins/vigilmon.js",
      enabled: true,
    },
    retryFailedStep: {
      enabled: true,
    },
    screenshotOnFail: {
      enabled: true,
    },
  },

  name: "my-app-e2e",
};

Local development

VIGILMON_HEARTBEAT_URL="https://vigilmon.online/api/heartbeat/<id>" \
  npx codeceptjs run --steps

Step 4: Add to CI/CD Pipelines

GitHub Actions

# .github/workflows/e2e.yml
name: CodeceptJS E2E

on:
  schedule:
    - cron: "0 2 * * *"   # 02:00 UTC nightly
  push:
    branches: [main]

jobs:
  codeceptjs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install chromium --with-deps

      - name: Start application
        run: npm run start:ci &

      - name: Wait for server
        run: npx wait-on http://localhost:3000 --timeout 30000

      - name: Run CodeceptJS
        env:
          CI: true
          VIGILMON_HEARTBEAT_URL: ${{ secrets.VIGILMON_CODECEPT_HEARTBEAT_URL }}
        run: npx codeceptjs run --steps

      # The Vigilmon plugin automatically sends the heartbeat when all tests pass.
      # Failures suppress the ping; Vigilmon alerts after the grace period.

GitLab CI

# .gitlab-ci.yml
codeceptjs:nightly:
  stage: test
  image: mcr.microsoft.com/playwright:v1.44.0-jammy
  variables:
    CI: "true"
    VIGILMON_HEARTBEAT_URL: $VIGILMON_CODECEPT_HEARTBEAT_URL
  only:
    - schedules
  script:
    - npm ci
    - npm run start:ci &
    - npx wait-on http://localhost:3000
    - npx codeceptjs run --steps

Step 5: Run Tests in Parallel with Workers

CodeceptJS 3.x supports parallel test execution via --workers. To send a heartbeat only after all workers succeed, use a post-run shell step instead of the plugin:

# .github/workflows/e2e-parallel.yml
jobs:
  codeceptjs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright install chromium --with-deps
      - run: npm run start:ci &
      - run: npx wait-on http://localhost:3000

      - name: Run CodeceptJS (parallel)
        run: npx codeceptjs run-workers 4

      - name: Ping Vigilmon heartbeat
        if: success()
        run: |
          curl -fsS -X POST "${{ secrets.VIGILMON_CODECEPT_HEARTBEAT_URL }}" \
            --max-time 10 --retry 3 --retry-delay 2

The if: success() condition ensures the heartbeat fires only when run-workers 4 exits 0.

Step 6: Multi-Environment Monitoring

For teams running CodeceptJS against staging and production:

| Suite | Environment | Monitor name | Interval | |---|---|---|---| | Nightly full regression | Staging | CodeceptJS E2E — Staging — Nightly | 25 h | | Pre-deploy smoke | Staging | CodeceptJS Smoke — Staging | 4 h | | Post-deploy verify | Production | CodeceptJS Smoke — Production | 2 h | | Weekly API + UI | Staging | CodeceptJS Full — Weekly | 8 days |

Parameterize the plugin:

// codecept-plugins/vigilmon.js (multi-env version)
event.dispatcher.on(event.all.result, async (result) => {
  const env = process.env.CODECEPT_ENV || "staging";
  const heartbeatVar = `VIGILMON_${env.toUpperCase()}_HEARTBEAT_URL`;
  const heartbeatUrl = process.env[heartbeatVar];

  if (heartbeatUrl && result.stats.failures === 0) {
    await pingVigilmon(heartbeatUrl).catch(output.error);
  }
});

CODECEPT_ENV=production \
  VIGILMON_PRODUCTION_HEARTBEAT_URL="..." \
  npx codeceptjs run --steps

Step 7: Configure Alert Channels

In Vigilmon, go to Notifications → New Channel:

  • Email — immediate alert when a heartbeat is missed
  • Slack webhook — notify #qa-alerts or #e2e-tests channel

When a CodeceptJS suite stops pinging:

🔴 MISSED HEARTBEAT: CodeceptJS E2E — Nightly
Last ping: 26 hours ago
Expected interval: 24 hours

When it recovers:

✅ HEARTBEAT RECOVERED: CodeceptJS E2E — Nightly
Gap: 26 hours

What You're Now Catching

| Silent failure mode | How Vigilmon detects it | |---|---| | CI schedule job deleted or paused | No ping → alert after grace period | | Playwright/WebDriver helper fails to start | Process exits → no ping → alert | | All tests skipped by tag/feature filter (exit 0) | No ping on empty run → alert | | Test machine is offline | No ping → alert after grace period | | CodeceptJS hangs waiting for a page element | Timeout fires → no ping → alert | | Environment variable change skips test init | No ping → alert |

CodeceptJS abstracts browser complexity so your team can focus on behavior — until the runner quietly disappears. Vigilmon's heartbeat monitors give you the external signal CodeceptJS can't: confirmation that your end-to-end tests ran and passed within the expected window.

Add heartbeat monitoring to your CodeceptJS suite today — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles