How to Monitor Hammox with Vigilmon

Hammox is a strict mock library for Elixir built on top of Mox. Where Mox verifies that mocked functions are called as expected, Hammox goes further: it validates that every call and return value matches the typespec declared in the behaviour. If a mocked function is called with a wrong argument type, or returns a value that doesn't match the spec, Hammox raises immediately — before the assertion. This surfaces type contract bugs that Mox would let pass silently.

Teams use Hammox to enforce that implementations respect their behaviour contracts throughout the test suite. When Hammox works correctly, type violations are caught at the mock boundary rather than in production. When it's misconfigured — behaviours drift out of sync with typespecs, Hammox is bypassed for a module, or a spec is removed — tests run but no longer enforce the contract. Vigilmon heartbeats paired with Hammox health checks catch these regressions automatically.

Why Monitor Hammox?

Hammox failures are subtle and often deferred:

• Typespec removal — a developer removes or loosens a @spec to silence a Dialyzer warning; Hammox silently stops enforcing that parameter on all mocks
• Behaviour/implementation drift — an interface callback is added to the behaviour but the mock is not updated; tests continue using the old callback signature and Hammox never fires
• Hammox bypassed for a module — a developer switches from use Hammox to use Mox on a module under deadline pressure; type enforcement is silently removed for that mock
• Spec ambiguity — a union type (integer() | String.t()) in the spec allows both; a caller accidentally passes the wrong type and Hammox can't distinguish the intended branch
• Missing Hammox.verify_on_exit!/0 — without this hook, Hammox does not assert unused expectations at the end of each test; mock call-count drift goes undetected

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | CI test suite exit code | Whether ExUnit passes with Hammox enforcement active | | Typespec coverage | Whether all behaviour callbacks have @spec annotations | | Dialyzer clean run | Whether typespecs are internally consistent and Hammox-enforceable | | Hammox verify hook count | Whether verify_on_exit!/0 is set up in every test case | | Daily CI heartbeat | Whether Hammox-guarded tests ran and passed | | Mock drift rate | Whether behaviours and mock modules stay in sync over time |

Step 1: Add Hammox to Your Project

# mix.exs
defp deps do
  [
    {:hammox, "~> 0.7", only: :test},
    {:dialyxir, "~> 1.4", only: [:dev], runtime: false},
    # rest of your deps
  ]
end

mix deps.get

Step 2: Define a Behaviour with Typespecs

# lib/my_app/payment_gateway.ex
defmodule MyApp.PaymentGateway do
  @callback charge(amount :: pos_integer(), currency :: String.t()) ::
              {:ok, transaction_id :: String.t()} | {:error, reason :: atom()}

  @callback refund(transaction_id :: String.t()) ::
              :ok | {:error, reason :: atom()}

  @callback status(transaction_id :: String.t()) ::
              {:ok, status :: :pending | :completed | :failed} | {:error, reason :: atom()}
end

Step 3: Create the Hammox Mock

# test/support/mocks.ex
Hammox.defmock(MyApp.MockPaymentGateway, for: MyApp.PaymentGateway)

Set up Hammox verification in your test case:

# test/test_helper.exs
Hammox.protect(MyApp.MockPaymentGateway, MyApp.PaymentGateway)
ExUnit.start()

Step 4: Write Hammox-Guarded Tests

# test/my_app/checkout_test.exs
defmodule MyApp.CheckoutTest do
  use ExUnit.Case, async: true

  import Hammox

  setup :verify_on_exit!

  test "charges correct amount and currency" do
    MyApp.MockPaymentGateway
    |> expect(:charge, fn amount, currency ->
      assert amount == 1999
      assert currency == "USD"
      {:ok, "txn_abc123"}
    end)

    assert {:ok, "txn_abc123"} =
             MyApp.Checkout.process(%{amount: 1999, currency: "USD"}, MyApp.MockPaymentGateway)
  end

  test "handles charge failure" do
    MyApp.MockPaymentGateway
    |> expect(:charge, fn _amount, _currency ->
      {:error, :insufficient_funds}
    end)

    assert {:error, :payment_failed} =
             MyApp.Checkout.process(%{amount: 9999, currency: "USD"}, MyApp.MockPaymentGateway)
  end

  test "refunds successful transaction" do
    MyApp.MockPaymentGateway
    |> expect(:charge, fn _amount, _currency -> {:ok, "txn_xyz789"} end)
    |> expect(:refund, fn transaction_id ->
      assert transaction_id == "txn_xyz789"
      :ok
    end)

    assert :ok = MyApp.Checkout.charge_and_refund(1000, "EUR", MyApp.MockPaymentGateway)
  end
end

Step 5: Add a Hammox Typespec Coverage Check

# lib/mix/tasks/check_hammox.ex
defmodule Mix.Tasks.CheckHammox do
  use Mix.Task

  @shortdoc "Verify all behaviour callbacks have @spec annotations for Hammox enforcement"

  @behaviours_to_check [
    MyApp.PaymentGateway,
    MyApp.NotificationService,
    MyApp.StorageProvider,
  ]

  def run(_args) do
    Mix.Task.run("compile")

    results =
      Enum.flat_map(@behaviours_to_check, fn behaviour ->
        callbacks = behaviour.behaviour_info(:callbacks)

        Enum.map(callbacks, fn {name, arity} ->
          specs = Code.Typespec.fetch_specs(behaviour)

          has_spec =
            case specs do
              {:ok, spec_list} -> Enum.any?(spec_list, fn {{fn_name, fn_arity}, _} ->
                fn_name == name and fn_arity == arity
              end)
              _ -> false
            end

          {behaviour, name, arity, has_spec}
        end)
      end)

    missing = Enum.filter(results, fn {_mod, _name, _arity, has_spec} -> not has_spec end)

    if missing == [] do
      Mix.shell().info("Hammox typespec coverage OK — all #{length(results)} callbacks have @spec")
    else
      Enum.each(missing, fn {mod, name, arity, _} ->
        Mix.shell().error("Missing @spec: #{inspect(mod)}.#{name}/#{arity}")
      end)
      Mix.shell().error("#{length(missing)} callback(s) missing @spec — Hammox cannot enforce type contracts")
      exit({:shutdown, 1})
    end
  end
end

Run it:

mix check_hammox
# Hammox typespec coverage OK — all 9 callbacks have @spec

Step 6: Run Hammox Checks in CI with a Heartbeat

# .github/workflows/hammox-health.yml
name: Hammox Mock Type Health

on:
  push:
    branches: [main]
  schedule:
    - cron: '0 4 * * *'   # daily at 04:00 UTC

jobs:
  hammox-check:
    name: Hammox Typespec Coverage and Tests
    runs-on: ubuntu-latest
    env:
      VIGILMON_HAMMOX_HEARTBEAT_URL: ${{ secrets.VIGILMON_HAMMOX_HEARTBEAT_URL }}
    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.17'
          otp-version: '27'

      - name: Cache deps
        uses: actions/cache@v3
        with:
          path: deps
          key: ${{ runner.os }}-deps-${{ hashFiles('mix.lock') }}

      - run: mix deps.get

      - name: Check Hammox typespec coverage
        run: mix check_hammox

      - name: Run Dialyzer
        run: mix dialyzer --halt-exit-status

      - name: Run Hammox-guarded tests
        run: mix test --color

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS "$VIGILMON_HAMMOX_HEARTBEAT_URL"

Step 7: Create a Heartbeat Monitor in Vigilmon

1. Sign in at vigilmon.online
2. Click New Monitor → Heartbeat
3. Name it Hammox Mock Type Health — main branch
4. Set the expected interval to 25 hours (matches the daily CI schedule)
5. Copy the URL and store it as VIGILMON_HAMMOX_HEARTBEAT_URL in CI secrets

If a typespec is removed, a behaviour drifts, or Hammox is bypassed for a module, the typespec coverage check or Dialyzer run fails, CI exits non-zero, and no heartbeat fires. Vigilmon alerts you within the missed-interval window.

Step 8: Alerting

In Vigilmon, go to Notifications → New Channel → Slack:

Heartbeat missed:

🔴 MISSED: Hammox Mock Type Health — main branch
Last healthy run: 30 hours ago
Action: Check CI — typespec removed, behaviour drift, or Hammox bypass detected

Configure a 1-hour grace period so transient CI delays don't produce false alerts. A 25-hour heartbeat with a 1-hour grace catches any full day where Hammox type enforcement didn't pass.

What You Built

| What | How | |------|-----| | Typed behaviour | @callback definitions with full @spec annotations | | Hammox mock | Hammox.defmock/2 with verify_on_exit!/0 in every test | | Typespec coverage check | Mix task: validates all callbacks have @spec | | Dialyzer integration | Catches inconsistent specs before Hammox tries to enforce them | | CI heartbeat | Vigilmon heartbeat — alerts when Hammox type-check CI job misses | | Daily schedule | GitHub Actions cron at 04:00 UTC | | Slack alerting | Vigilmon Slack notification channel |

Hammox is most valuable when behaviours stay fully typed and every mock uses verify_on_exit!/0. Without monitoring, teams gradually erode these invariants under deadline pressure. Vigilmon ensures Hammox type enforcement stays active across every push and every day.

Next Steps

• Add Hammox type checks to your CI matrix for all OTP versions you support
• Use Hammox.protect/2 in test setup to apply Hammox enforcement globally without modifying individual test modules
• Write a nightly Dialyzer run separate from your test heartbeat — Dialyzer catches spec drift that Hammox cannot see until tests actually run
• Set up a Vigilmon status page showing Hammox health over time for your engineering leads

Get started free at vigilmon.online.