How to Monitor Mox with Vigilmon

Mox is the de-facto mock library for Elixir. Unlike ad-hoc mocking solutions, Mox enforces that every mock is backed by a behaviour — a real interface defined in your production code. When a mock call doesn't match a declared callback, Mox raises at test time, not at runtime. That contract-first discipline catches integration drift before it reaches production.

But Mox only helps if your test suite runs and passes. Flaky CI pipelines, Mox verification timeouts, and silent regressions in behaviour contracts can quietly erode the safety net Mox is supposed to provide. This guide shows how to monitor the health of your Mox-backed test pipeline with Vigilmon.

Why Monitor a Mox-Based Test Suite?

Mox tests catch contract drift between your modules — but only when they actually run and pass. Common failure modes worth monitoring:

• Stale expectations: Mox.expect/4 calls that no longer match updated behaviours cause test suite failures that block deploys
• Async race conditions: Mox.set_mox_from_context/1 in concurrent tests can produce non-deterministic failures that appear and disappear between CI runs
• Behaviour drift: a collaborating service changes its API; the behaviour definition lags; Mox tests keep passing against the old contract while production breaks
• Verification skipped: if a test crashes before Mox.verify_on_exit!/1 runs, expectations are never verified and contract violations slip through

Vigilmon lets you monitor CI pipeline outcomes via heartbeats and HTTP checks, alerting your team the moment any of these failure modes surfaces in a broken pipeline.

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | CI pipeline pass rate | Whether Mox-backed tests are passing consistently | | Test suite duration | Trend toward slower verification (async leakage, expectation accumulation) | | Verification failure count | Expectations set but never called — indicates unused mock paths | | Async test flakiness rate | Race conditions in Mox.set_mox_from_context/1 usage | | Behaviour coverage | How many defined behaviours have at least one Mox-backed test |

Step 1: Expose Test Results via a Health Endpoint

The most direct way to give Vigilmon visibility into your test pipeline is to emit a heartbeat after a successful CI run. First, add a script that runs your tests and conditionally pings Vigilmon:

#!/bin/bash
# scripts/ci_test.sh
set -e

mix test --warnings-as-errors

# All tests passed — ping Vigilmon heartbeat
if [ -n "$VIGILMON_TEST_HEARTBEAT_URL" ]; then
  curl -s "$VIGILMON_TEST_HEARTBEAT_URL" > /dev/null
fi

In your CI configuration (GitHub Actions example):

# .github/workflows/test.yml
name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      VIGILMON_TEST_HEARTBEAT_URL: ${{ secrets.VIGILMON_TEST_HEARTBEAT_URL }}
    steps:
      - uses: actions/checkout@v4
      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.16'
          otp-version: '26'
      - run: mix deps.get
      - run: mix test --warnings-as-errors
      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -s "$VIGILMON_TEST_HEARTBEAT_URL"

The heartbeat only fires when tests pass. If tests fail, or if CI doesn't run at all (e.g., a broken pipeline configuration), no heartbeat is sent — and Vigilmon alerts you.

Step 2: Create a Heartbeat Monitor in Vigilmon

1. Sign in at vigilmon.online
2. Click New Monitor → Heartbeat
3. Name it Mox Test Suite — main branch
4. Set the expected interval to match your CI cadence, e.g. 6 hours for a team that merges several times a day, or 24 hours for slower release cycles
5. Copy the heartbeat URL and add it as a CI secret (VIGILMON_TEST_HEARTBEAT_URL)

If no heartbeat arrives within the interval, Vigilmon triggers your alert channels.

Step 3: Track Mox Verification Failures in Application Logs

For runtime-level Mox usage (rare, but possible in integration test harnesses or development scaffolding), you can surface verification errors in your application health check:

# lib/my_app_web/plugs/health_check.ex
defmodule MyAppWeb.Plugs.HealthCheck do
  import Plug.Conn

  def init(opts), do: opts

  def call(%Plug.Conn{request_path: "/health"} = conn, _opts) do
    checks = %{
      database: check_db(),
      app: :ok
    }

    status = if Enum.all?(checks, fn {_, v} -> v == :ok end), do: 200, else: 503

    conn
    |> put_resp_content_type("application/json")
    |> send_resp(status, Jason.encode!(%{status: if(status == 200, do: "ok", else: "degraded"), checks: checks}))
    |> halt()
  end

  def call(conn, _opts), do: conn

  defp check_db do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", []) do
      {:ok, _} -> :ok
      _ -> :error
    end
  end
end

Add HTTP monitoring in Vigilmon pointing to /health for production uptime coverage alongside your CI heartbeat.

Step 4: Alerting

In Vigilmon, go to Notifications → New Channel and configure your preferred channel:

Slack:

🔴 MISSED: Mox Test Suite — main branch
Last heartbeat: 8 hours ago (expected every 6 hours)

Email or PagerDuty for on-call escalation when the main branch pipeline goes silent.

Set up two monitors:

• Heartbeat monitor — catches pipeline failures or CI outages
• HTTP monitor on /health — catches production regressions introduced after tests passed

Step 5: Enforce Behaviour Coverage in CI

Add a custom Mix task to verify all declared behaviours have at least one Mox-backed test:

# lib/mix/tasks/check_behaviour_coverage.ex
defmodule Mix.Tasks.CheckBehaviourCoverage do
  use Mix.Task

  @shortdoc "Verify all behaviours are covered by at least one Mox test"

  def run(_args) do
    behaviours = find_behaviours()
    mocks = find_mocks()

    missing = behaviours -- mocks

    if missing == [] do
      Mix.shell().info("All behaviours have Mox coverage.")
    else
      Mix.shell().error("Missing Mox coverage for behaviours: #{inspect(missing)}")
      exit({:shutdown, 1})
    end
  end

  defp find_behaviours do
    Path.wildcard("lib/**/*.ex")
    |> Enum.flat_map(&extract_behaviours/1)
  end

  defp find_mocks do
    Path.wildcard("test/**/*.ex")
    |> Enum.flat_map(&extract_mocks/1)
  end

  defp extract_behaviours(path) do
    path
    |> File.read!()
    |> then(&Regex.scan(~r/@callback\s+\w+/, &1))
    |> List.flatten()
    |> Enum.map(fn _ -> path end)
    |> Enum.uniq()
  end

  defp extract_mocks(path) do
    path
    |> File.read!()
    |> then(&Regex.scan(~r/Mox\.defmock\((\w+)/, &1))
    |> Enum.map(fn [_, mod] -> mod end)
  end
end

Run it in CI before the heartbeat ping to catch coverage gaps before they silently accumulate.

Step 6: Status Page

Create a Vigilmon status page that shows:

• Mox CI pipeline — heartbeat monitor
• Production API — HTTP monitor on /health

Share the URL with your team so anyone can check test pipeline health without logging into CI.

What You've Built

| What | How | |------|-----| | CI heartbeat monitoring | Vigilmon heartbeat monitor + CI secret | | Pipeline failure alerts | Slack/email/PagerDuty notification channel | | Production health check | Custom plug + Vigilmon HTTP monitor | | Behaviour coverage gate | Custom Mix task in CI pipeline | | Team status page | Vigilmon public status page |

Mox keeps your contracts honest. Vigilmon keeps your pipeline honest.

Next Steps

• Add separate heartbeat monitors per branch if you maintain long-lived feature branches
• Use Vigilmon's response time history on /health to detect performance regressions introduced alongside Mox contract changes
• Alert on missed heartbeats with a grace period slightly longer than your slowest CI run to avoid false positives

Get started free at vigilmon.online.