tutorial

How to Monitor Patch (Elixir) with Vigilmon

Monitor your Elixir Patch mocking setup — detect stub registration failures, spy misconfiguration, and mock isolation regressions before they let test pollution corrupt CI or mask real bugs.

How to Monitor Patch (Elixir) with Vigilmon

Patch is a powerful mocking and patching library for Elixir that replaces module functions at runtime without requiring behaviours or dependency injection. Where Mox and Hammox require behaviour contracts and test-friendly injection points, Patch works directly on any module: you call patch/3 to replace a function with a stub or spy, and Patch restores the original after the test finishes.

Teams use Patch to test code that calls third-party modules directly, to inject values without refactoring callers, and to verify call counts and arguments through spies. When Patch works correctly, tests are isolated and deterministic. When it's misconfigured — patches not restored after tests, spy assertions not verified, or a Patch version update changes cleanup semantics — test isolation silently breaks and tests influence each other. Vigilmon heartbeats paired with Patch health checks catch these regressions automatically.

Why Monitor Patch?

Patch failures tend to pollute test suites rather than fail loudly:

  • Patches not restored — if a test crashes mid-run before Patch can restore the original function, subsequent tests run against the patched version and fail for unrelated reasons
  • Async patch conflicts — async: true tests patching the same module interfere with each other because Patch modifies process-local or global module state depending on configuration
  • Spy assertions skipped — assert_called/2 is only meaningful if called; a spy is set up but the assertion is never written, so call-count bugs go undetected
  • Patch scoping drift — a patch/3 call intended for one test leaks into a setup block and applies to all tests in the module, masking real function calls
  • Value injection stale — patch/3 with a fixed return value becomes stale when the real function's return shape changes; tests continue passing but assert against the wrong shape

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | CI test suite exit code | Whether ExUnit passes with Patch mocks in place | | Test isolation health | Whether tests run in a consistent order and produce consistent results | | Async test conflict rate | Whether async: true tests patching the same module produce flakes | | Spy assertion coverage | Whether every spy has a corresponding assert_called | | Daily CI heartbeat | Whether Patch-guarded tests ran and passed | | Patch version pinning | Whether a Patch version bump changed cleanup or async semantics |

Step 1: Add Patch to Your Project

# mix.exs
defp deps do
  [
    {:patch, "~> 0.13", only: :test},
    # rest of your deps
  ]
end

mix deps.get

Step 2: Write Basic Patch Tests

# test/my_app/notifications_test.exs
defmodule MyApp.NotificationsTest do
  use ExUnit.Case
  use Patch

  test "sends email notification when order is confirmed" do
    patch(MyApp.Mailer, :send_email, fn _to, _subject, _body -> :ok end)

    MyApp.Orders.confirm(%{id: "ord_123", user_email: "alice@example.com"})

    assert_called(MyApp.Mailer.send_email("alice@example.com", :_, :_))
  end

  test "logs warning when email fails" do
    patch(MyApp.Mailer, :send_email, fn _to, _subject, _body ->
      {:error, :smtp_connection_failed}
    end)

    patch(Logger, :warning, fn _msg -> :ok end)

    MyApp.Orders.confirm(%{id: "ord_124", user_email: "bob@example.com"})

    assert_called(Logger.warning(:_))
  end

  test "does not send email when order is already confirmed" do
    patch(MyApp.Mailer, :send_email, fn _to, _subject, _body -> :ok end)

    MyApp.Orders.confirm(%{id: "ord_125", user_email: "carol@example.com", status: :confirmed})

    refute_called(MyApp.Mailer.send_email(:_, :_, :_))
  end
end

Step 3: Use Spies for Call Verification

# test/my_app/analytics_test.exs
defmodule MyApp.AnalyticsTest do
  use ExUnit.Case
  use Patch

  test "tracks purchase event with correct properties" do
    spy(MyApp.Analytics)

    MyApp.Orders.purchase(%{
      user_id: "usr_456",
      product_id: "prod_789",
      amount_cents: 2999,
      currency: "USD"
    })

    assert_called(MyApp.Analytics.track("purchase", %{
      user_id: "usr_456",
      product_id: "prod_789",
      amount_cents: 2999
    }))
  end

  test "tracks exactly one purchase event per order" do
    spy(MyApp.Analytics)

    MyApp.Orders.purchase(%{user_id: "usr_1", product_id: "prod_1", amount_cents: 100, currency: "USD"})

    assert_called_once(MyApp.Analytics.track("purchase", :_))
  end

  test "does not track event when purchase fails validation" do
    spy(MyApp.Analytics)

    # amount_cents: 0 fails validation
    {:error, :invalid_amount} =
      MyApp.Orders.purchase(%{user_id: "usr_2", product_id: "prod_2", amount_cents: 0, currency: "USD"})

    refute_called(MyApp.Analytics.track(:_, :_))
  end
end

Step 4: Value Injection Without Behaviours

# test/my_app/feature_flags_test.exs
defmodule MyApp.FeatureFlagsTest do
  use ExUnit.Case
  use Patch

  test "shows new checkout UI when feature flag is enabled" do
    patch(MyApp.FeatureFlags, :enabled?, fn :new_checkout, _user -> true end)

    conn = build_conn(:get, "/checkout")
    response = MyApp.CheckoutController.index(conn, %{})

    assert response.assigns.use_new_checkout == true
  end

  test "shows legacy checkout UI when feature flag is disabled" do
    patch(MyApp.FeatureFlags, :enabled?, fn :new_checkout, _user -> false end)

    conn = build_conn(:get, "/checkout")
    response = MyApp.CheckoutController.index(conn, %{})

    assert response.assigns.use_new_checkout == false
  end

  test "injects fixed timestamp for deterministic date formatting" do
    fixed_now = ~U[2026-01-15 12:00:00Z]
    patch(DateTime, :utc_now, fn -> fixed_now end)

    assert MyApp.Formatter.today_label() == "January 15, 2026"
  end
end

Step 5: Add a Patch Isolation Smoke-Test Mix Task

# lib/mix/tasks/check_patch.ex
defmodule Mix.Tasks.CheckPatch do
  use Mix.Task

  @shortdoc "Verify Patch can replace and restore module functions correctly"

  defmodule SampleModule do
    def answer, do: 42
  end

  def run(_args) do
    Code.ensure_loaded!(SampleModule)

    # Verify original value
    unless SampleModule.answer() == 42 do
      Mix.shell().error("Baseline check failed: SampleModule.answer() != 42")
      exit({:shutdown, 1})
    end

    # Apply patch (Patch requires a test process; simulate manually here)
    # Instead, verify library loads and exports are correct
    exports = Patch.module_info(:exports)
    required = [:patch, :spy, :assert_called, :refute_called]

    missing = Enum.filter(required, fn fn_name ->
      not Enum.any?(exports, fn {name, _arity} -> name == fn_name end)
    end)

    if missing == [] do
      Mix.shell().info("Patch OK — library loaded with all required functions: #{inspect(required)}")
    else
      Mix.shell().error("Patch missing exports: #{inspect(missing)}")
      exit({:shutdown, 1})
    end
  end
end

Run it:

MIX_ENV=test mix check_patch
# Patch OK — library loaded with all required functions: [:patch, :spy, :assert_called, :refute_called]

Step 6: Run Patch Tests in CI with a Heartbeat

# .github/workflows/patch-health.yml
name: Patch Mock Isolation Health

on:
  push:
    branches: [main]
  schedule:
    - cron: '0 2 * * *'   # daily at 02:00 UTC

jobs:
  patch-health:
    name: Patch Isolation and Test Suite
    runs-on: ubuntu-latest
    env:
      VIGILMON_PATCH_HEARTBEAT_URL: ${{ secrets.VIGILMON_PATCH_HEARTBEAT_URL }}
    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.17'
          otp-version: '27'

      - name: Cache deps
        uses: actions/cache@v3
        with:
          path: deps
          key: ${{ runner.os }}-deps-${{ hashFiles('mix.lock') }}

      - run: mix deps.get

      - name: Patch library smoke test
        run: MIX_ENV=test mix check_patch

      - name: Run full test suite with Patch
        run: mix test --color

      - name: Run tests in random order to detect isolation issues
        run: mix test --seed $RANDOM --color

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS "$VIGILMON_PATCH_HEARTBEAT_URL"

Running tests twice — once in default order and once in random order — catches test pollution caused by patches not being properly restored between tests.

Step 7: Create a Heartbeat Monitor in Vigilmon

  1. Sign in at vigilmon.online
  2. Click New Monitor → Heartbeat
  3. Name it Patch Mock Isolation Health — main branch
  4. Set the expected interval to 25 hours (matches the daily CI schedule)
  5. Copy the URL and store it as VIGILMON_PATCH_HEARTBEAT_URL in CI secrets

If a patch leaks between tests, the random-order run fails while the default-order run passes, CI exits non-zero, and no heartbeat fires. Vigilmon alerts you within the missed-interval window.

Step 8: Alerting

In Vigilmon, go to Notifications → New Channel → Slack:

Heartbeat missed:

🔴 MISSED: Patch Mock Isolation Health — main branch
Last healthy run: 30 hours ago
Action: Check CI — patch leak, spy not restored, or async test conflict detected

Configure a 1-hour grace period so transient CI delays don't generate false alerts. A 25-hour heartbeat with a 1-hour grace catches any full day where Patch-guarded tests didn't complete cleanly.

What You Built

| What | How | |------|-----| | Basic patch tests | patch/3 stubs for mailer and logger functions | | Spy verification | spy/1 with assert_called/2 and assert_called_once/2 | | Value injection | Fixed return values patched onto DateTime and feature flags | | Library smoke test | Mix task: verifies Patch exports all required functions | | Isolation verification | CI runs tests in random seed order to catch patch leaks | | CI heartbeat | Vigilmon heartbeat — alerts when Patch isolation tests miss | | Daily schedule | GitHub Actions cron at 02:00 UTC | | Slack alerting | Vigilmon Slack notification channel |

Patch is powerful precisely because it requires no behaviour contracts — but that flexibility means isolation regressions are harder to catch. Running tests in randomised order and monitoring the result with Vigilmon ensures Patch-based mocks stay clean across every push and every day.

Next Steps

  • Use Patch.History.calls/1 to assert the full sequence of calls made to a spy, not just whether a specific call occurred
  • Separate Patch tests from behaviour-based Mox/Hammox tests by directory so CI can run them independently and attribute failures accurately
  • Add a Vigilmon uptime monitor for your staging environment alongside the Patch heartbeat — if both fire together, the issue is infrastructure, not mock isolation
  • Pin the Patch version explicitly in mix.exs and review the changelog before upgrading — cleanup semantics can change between minor versions

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles