tutorial

Monitoring Artillery Load Testing Pipelines with Vigilmon: Heartbeat, HTTP, and Alerting

How to monitor Artillery load testing pipelines with Vigilmon — heartbeat monitoring for scheduled test runs, HTTP monitors for Artillery Cloud dashboards, and alerting when load tests fail or go silent.

Artillery is a modern, cloud-scale load testing platform built on Node.js. It can run HTTP, WebSocket, and gRPC scenarios against any endpoint, integrating with CI/CD pipelines, Docker, and Artillery Cloud. Because Artillery is a test runner rather than a long-running server, standard uptime monitors can't point at it directly — Vigilmon fills this gap with heartbeat monitors for scheduled test jobs, HTTP monitors for Artillery Cloud dashboards, and alerting when performance baselines regress.

What You'll Build

  • A Vigilmon heartbeat monitor that fires when a scheduled Artillery run completes successfully
  • An HTTP monitor for your Artillery Cloud project dashboard
  • A lightweight results webhook that reports test outcome to Vigilmon
  • Alert thresholds for failed runs and long silences

Prerequisites

  • Artillery installed (npm install -g artillery) or running in CI
  • A free account at vigilmon.online
  • (Optional) An Artillery Cloud account for remote dashboards

Step 1: Verify Your Artillery Installation

Confirm the CLI is reachable and your test scenario is valid before wiring up monitoring:

# Check the installed version
artillery version

# Validate a scenario file without running it
artillery run --dry-run my-load-test.yml

A simple scenario that targets an HTTP API looks like:

# my-load-test.yml
config:
  target: "https://api.example.com"
  phases:
    - duration: 60
      arrivalRate: 10
      name: "Warm up"
    - duration: 120
      arrivalRate: 50
      name: "Ramp up"

scenarios:
  - name: "Homepage"
    flow:
      - get:
          url: "/health"
      - get:
          url: "/api/users"

Run it once to confirm it executes end-to-end:

artillery run my-load-test.yml

Step 2: Set Up a Heartbeat Monitor for Scheduled Runs

Artillery load tests are typically triggered on a schedule (nightly, pre-release, or at a fixed cadence). A Vigilmon heartbeat monitor alerts you when the expected ping stops arriving — meaning your load test job silently failed to start or crashed before completion.

Create the Heartbeat Monitor

  1. Log in to Vigilmon → Add Monitor → Heartbeat.
  2. Name: artillery-nightly-load-test.
  3. Expected interval: Match your test cadence (e.g., 86400 seconds for daily).
  4. Grace period: 30 minutes (tests can run long — allow buffer before alerting).
  5. Copy the unique heartbeat URL shown: https://vigilmon.online/heartbeat/your-unique-id.

Wire the Heartbeat Into Artillery

After a successful run, hit the heartbeat URL. The cleanest integration is a wrapper script:

Bash (for cron or CI):

#!/bin/bash
# run-load-test.sh

set -euo pipefail

VIGILMON_HEARTBEAT="${VIGILMON_HEARTBEAT_URL}"

echo "Starting Artillery load test..."
artillery run my-load-test.yml --output results.json

# Only ping Vigilmon if the test completed without non-zero exit
echo "Test complete. Pinging Vigilmon heartbeat..."
curl -fsS "${VIGILMON_HEARTBEAT}" > /dev/null

echo "Done."

Add to crontab for nightly runs:

0 2 * * * /opt/load-tests/run-load-test.sh >> /var/log/artillery-load-test.log 2>&1

Node.js wrapper:

// run-with-heartbeat.js
const { execSync } = require('child_process');
const https = require('https');

const HEARTBEAT_URL = process.env.VIGILMON_HEARTBEAT_URL;

function pingHeartbeat() {
  return new Promise((resolve, reject) => {
    https.get(HEARTBEAT_URL, (res) => {
      res.resume();
      resolve(res.statusCode);
    }).on('error', reject);
  });
}

async function main() {
  console.log('Running Artillery load test...');
  try {
    execSync('artillery run my-load-test.yml --output results.json', {
      stdio: 'inherit',
    });
    console.log('Test passed. Signaling heartbeat...');
    await pingHeartbeat();
  } catch (err) {
    console.error('Artillery run failed:', err.message);
    process.exit(1);
  }
}

main();

Step 3: Report Test Results to a Webhook Endpoint

For richer signal — not just "did it finish" but "did it pass performance thresholds" — build a results endpoint and check that from Vigilmon.

Simple Results Server

// artillery-results-server.js
const express = require('express');
const fs = require('fs');

const app = express();
app.use(express.json());

let latestResult = null;

// Artillery can POST JSON reports via a custom plugin or wrapper
app.post('/artillery/results', (req, res) => {
  latestResult = {
    timestamp: new Date().toISOString(),
    summary: req.body,
  };
  res.json({ received: true });
});

app.get('/health/artillery', (req, res) => {
  if (!latestResult) {
    return res.status(503).json({ status: 'no_results', message: 'No test results received yet' });
  }

  const ageMs = Date.now() - new Date(latestResult.timestamp).getTime();
  const maxAgeMs = 25 * 60 * 60 * 1000; // 25 hours

  if (ageMs > maxAgeMs) {
    return res.status(503).json({
      status: 'stale',
      message: 'Last result is older than 25 hours',
      lastRun: latestResult.timestamp,
    });
  }

  const summary = latestResult.summary;
  const p99 = summary?.aggregate?.latency?.p99 || 0;
  const errorRate = summary?.aggregate?.counters?.['http.response_code']?.['5xx'] || 0;

  if (p99 > 2000 || errorRate > 0.01) {
    return res.status(503).json({
      status: 'degraded',
      p99LatencyMs: p99,
      errorRate,
      lastRun: latestResult.timestamp,
    });
  }

  res.json({
    status: 'ok',
    p99LatencyMs: p99,
    errorRate,
    lastRun: latestResult.timestamp,
  });
});

app.listen(3010, () => console.log('Artillery results server on :3010'));

Post Results From Your Test Script

#!/bin/bash
# After running the test:
artillery run my-load-test.yml --output results.json

# Parse and post summary to the results server
node -e "
const r = require('./results.json');
const https = require('http');
const data = JSON.stringify(r);
const opts = { hostname: 'localhost', port: 3010, path: '/artillery/results', method: 'POST',
  headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data) } };
const req = https.request(opts, res => console.log('Posted:', res.statusCode));
req.write(data); req.end();
"

# Ping heartbeat only on test success
curl -fsS "\${VIGILMON_HEARTBEAT_URL}"

Step 4: Add a Vigilmon HTTP Monitor for the Results Endpoint

Once the results server is running:

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: http://your-server.example.com:3010/health/artillery.
  3. Check interval: 60 minutes (aligns with expected test cadence).
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: "status":"ok".
  7. Click Save.

Step 5: Monitor Artillery Cloud Dashboards

If you use Artillery Cloud (artillery cloud:run), the dashboard is reachable over HTTPS:

  1. Add Monitor → HTTP.
  2. URL: https://app.artillery.io (or your self-hosted Artillery Cloud URL).
  3. Check interval: 5 minutes.
  4. Expected status: 200.
  5. Click Save.

This catches outages in Artillery Cloud itself — if the dashboard is unreachable, your engineers can't review test reports.

Step 6: Integrate With CI/CD

GitHub Actions

# .github/workflows/load-test.yml
name: Nightly Load Test

on:
  schedule:
    - cron: '0 2 * * *'
  workflow_dispatch:

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Artillery
        run: npm install -g artillery

      - name: Run load test
        run: artillery run my-load-test.yml --output results.json

      - name: Upload results
        run: |
          node -e "
            const r = require('./results.json');
            const { execSync } = require('child_process');
            process.stdout.write(JSON.stringify(r));
          " | curl -s -X POST \
            -H 'Content-Type: application/json' \
            -d @- \
            http://your-server.example.com:3010/artillery/results

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS "${{ secrets.VIGILMON_HEARTBEAT_URL }}"

GitLab CI

# .gitlab-ci.yml
load-test:
  image: node:20
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
  script:
    - npm install -g artillery
    - artillery run my-load-test.yml --output results.json
    - curl -fsS "$VIGILMON_HEARTBEAT_URL"

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, configure alert routing:

| Monitor | Trigger | Recommended Action | |---|---|---| | Heartbeat (nightly run) | Ping not received within grace period | Check CI logs; confirm job scheduled; inspect machine uptime | | HTTP (results endpoint) | 503 or keyword absent | Review last Artillery run output; check for timeout or connection errors | | HTTP (Artillery Cloud) | Dashboard unreachable | Check Artillery Cloud status page; retry or run locally |

Recommended thresholds:

  • Heartbeat grace period: 30–60 minutes beyond your expected test duration
  • Confirmation period: 1 consecutive failure (Artillery jobs are not chatty; a single miss is signal)
  • Recovery notification: Enable so your team knows when tests are green again

Common Artillery Monitoring Failure Modes

| Scenario | What Vigilmon Catches | |---|---| | Cron job misconfigured or server rebooted | Heartbeat goes silent | | Artillery scenario file has syntax errors | Job fails before heartbeat ping; heartbeat fires | | Target service is down (test can't reach it) | Test exits non-zero; heartbeat not sent | | p99 latency above threshold | Results endpoint returns 503 | | Artillery Cloud outage | HTTP monitor on dashboard fires | | CI quota exhausted or runner offline | Heartbeat goes silent |

Artillery gives you powerful load scenarios but no built-in alerting for when tests silently stop running. Vigilmon's heartbeat and HTTP monitors close this gap — you'll know within minutes whether your nightly load tests are running, passing, and staying within your performance baselines.

Get started free at vigilmon.online — your Artillery pipeline monitor is running in under 5 minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles