tutorial

How to Monitor Bypass with Vigilmon

Monitor your Elixir Bypass HTTP stub server setup — detect stub registration failures, unexpected request leaks, and test isolation regressions before they corrupt CI or hit real external APIs.

How to Monitor Bypass with Vigilmon

Bypass is the standard HTTP stubbing library for Elixir tests. It spins up a real HTTP server on a local port for each test, letting your application code make genuine HTTP requests that Bypass intercepts and answers with canned responses — no network required. Teams use it to test HTTP client integrations (Finch, HTTPoison, Tesla) without hitting real third-party APIs.

When Bypass works correctly, tests are fast, isolated, and deterministic. When it breaks — stubs are registered incorrectly, an unexpected request leaks through to a real endpoint, or the stub server fails to start — test suites either fail with cryptic connection refused errors or silently make real API calls that burn rate limits or alter production data. Vigilmon heartbeats paired with Bypass health checks catch these regressions automatically.

Why Monitor Bypass?

Bypass failures tend to be silent or misleading:

  • Stub server port conflicts — two test workers attempt to bind the same port; one fails with :eaddrinuse and all tests in that worker produce connection refused
  • Unexpected passthrough requests — a misconfigured Bypass instance allows a request through to the real endpoint; you burn a live API quota or mutate production data
  • Stub registration race conditions — Bypass.expect/3 called after the HTTP client fires; the request arrives before the stub is registered and the test fails non-deterministically
  • Missing Bypass.down/1 in teardown — the stub server keeps running after the test; subsequent tests bind a new port but old workers leave dangling sockets
  • Bypass not called — Bypass.expect_once/3 asserts the stub was hit exactly once; forgetting to call it causes silent test pollution where a real HTTP call succeeded

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | CI test suite exit code | Whether ExUnit passes with Bypass stubs in place | | Bypass server start success rate | Whether Bypass.open/0 succeeds across all test workers | | Unexpected real HTTP calls | Whether any HTTP requests escaped the stub boundary | | Test suite duration | Unexpected slowdown may indicate real network calls | | Flaky test rate | Port conflicts or race conditions produce intermittent failures | | Daily CI heartbeat | Whether Bypass-protected integration tests ran at all |

Step 1: Add Bypass to Your Project

# mix.exs
defp deps do
  [
    {:bypass, "~> 2.1", only: :test},
    {:finch, "~> 0.18"},
    # rest of your deps
  ]
end

mix deps.get

Step 2: Write a Basic Bypass Integration Test

# test/my_app/http_client_test.exs
defmodule MyApp.HttpClientTest do
  use ExUnit.Case, async: true

  setup do
    bypass = Bypass.open()
    {:ok, bypass: bypass}
  end

  test "fetches user data from external API", %{bypass: bypass} do
    Bypass.expect_once(bypass, "GET", "/users/42", fn conn ->
      Plug.Conn.resp(conn, 200, ~s({"id": 42, "name": "Alice"}))
    end)

    base_url = "http://localhost:#{bypass.port}"
    assert {:ok, %{"id" => 42, "name" => "Alice"}} = MyApp.HttpClient.get_user(base_url, 42)
  end

  test "handles 404 from external API", %{bypass: bypass} do
    Bypass.expect_once(bypass, "GET", "/users/999", fn conn ->
      Plug.Conn.resp(conn, 404, ~s({"error": "not found"}))
    end)

    base_url = "http://localhost:#{bypass.port}"
    assert {:error, :not_found} = MyApp.HttpClient.get_user(base_url, 999)
  end

  test "handles server error from external API", %{bypass: bypass} do
    Bypass.expect_once(bypass, "GET", "/users/1", fn conn ->
      Plug.Conn.resp(conn, 500, ~s({"error": "internal server error"}))
    end)

    base_url = "http://localhost:#{bypass.port}"
    assert {:error, :server_error} = MyApp.HttpClient.get_user(base_url, 1)
  end
end

Step 3: Test Stub Server Down Behaviour

# test/my_app/http_client_down_test.exs
defmodule MyApp.HttpClientDownTest do
  use ExUnit.Case, async: true

  setup do
    bypass = Bypass.open()
    {:ok, bypass: bypass}
  end

  test "returns error when external API is unreachable", %{bypass: bypass} do
    Bypass.down(bypass)

    base_url = "http://localhost:#{bypass.port}"
    assert {:error, :connection_failed} = MyApp.HttpClient.get_user(base_url, 1)
  end

  test "retries and eventually fails when API is intermittently down", %{bypass: bypass} do
    call_count = :counters.new(1, [])

    Bypass.expect(bypass, "GET", "/users/1", fn conn ->
      count = :counters.get(call_count, 1) + 1
      :counters.put(call_count, 1, count)

      if count < 3 do
        # Simulate transient failure by closing connection
        Bypass.pass(bypass)
        Plug.Conn.resp(conn, 503, "Service Unavailable")
      else
        Plug.Conn.resp(conn, 200, ~s({"id": 1, "name": "Alice"}))
      end
    end)

    base_url = "http://localhost:#{bypass.port}"
    assert {:ok, _} = MyApp.HttpClient.get_user(base_url, 1)
    assert :counters.get(call_count, 1) == 3
  end
end

Step 4: Add a Bypass Smoke-Test Mix Task

# lib/mix/tasks/check_bypass.ex
defmodule Mix.Tasks.CheckBypass do
  use Mix.Task

  @shortdoc "Verify Bypass can open and respond to HTTP requests"

  def run(_args) do
    Application.ensure_all_started(:bypass)
    Application.ensure_all_started(:finch)

    Finch.start_link(name: BypassCheck.Finch)

    bypass = Bypass.open()

    Bypass.expect_once(bypass, "GET", "/health", fn conn ->
      Plug.Conn.resp(conn, 200, "ok")
    end)

    url = "http://localhost:#{bypass.port}/health"
    request = Finch.build(:get, url)

    case Finch.request(request, BypassCheck.Finch) do
      {:ok, %Finch.Response{status: 200, body: "ok"}} ->
        Mix.shell().info("Bypass OK — stub server started and responded on port #{bypass.port}")
        Bypass.down(bypass)

      {:ok, resp} ->
        Mix.shell().error("Bypass stub returned unexpected response: #{inspect(resp)}")
        exit({:shutdown, 1})

      {:error, reason} ->
        Mix.shell().error("Bypass request failed: #{inspect(reason)}")
        exit({:shutdown, 1})
    end
  end
end

Run it:

MIX_ENV=test mix check_bypass
# Bypass OK — stub server started and responded on port 51234

Step 5: Run Bypass Checks in CI with a Heartbeat

# .github/workflows/bypass-health.yml
name: Bypass HTTP Stub Health

on:
  push:
    branches: [main]
  schedule:
    - cron: '0 5 * * *'   # daily at 05:00 UTC

jobs:
  bypass-smoke:
    name: Bypass Stub Smoke Test
    runs-on: ubuntu-latest
    env:
      VIGILMON_BYPASS_HEARTBEAT_URL: ${{ secrets.VIGILMON_BYPASS_HEARTBEAT_URL }}
    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.17'
          otp-version: '27'

      - name: Cache deps
        uses: actions/cache@v3
        with:
          path: deps
          key: ${{ runner.os }}-deps-${{ hashFiles('mix.lock') }}

      - run: mix deps.get

      - name: Bypass smoke test
        run: MIX_ENV=test mix check_bypass

      - name: Full Bypass integration tests
        run: mix test test/my_app/http_client_test.exs test/my_app/http_client_down_test.exs --color

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS "$VIGILMON_BYPASS_HEARTBEAT_URL"

Step 6: Create a Heartbeat Monitor in Vigilmon

  1. Sign in at vigilmon.online
  2. Click New Monitor → Heartbeat
  3. Name it Bypass HTTP Stub Health — main branch
  4. Set the expected interval to 25 hours (matches the daily CI schedule)
  5. Copy the URL and store it as VIGILMON_BYPASS_HEARTBEAT_URL in CI secrets

If Bypass fails to start, a port conflict occurs, or an HTTP client library upgrade breaks the integration, CI exits non-zero and no heartbeat fires. Vigilmon alerts you within the missed-interval window.

Step 7: Alerting

In Vigilmon, go to Notifications → New Channel → Slack:

Heartbeat missed:

🔴 MISSED: Bypass HTTP Stub Health — main branch
Last healthy run: 30 hours ago
Action: Check CI — Bypass stub server failure or HTTP client misconfiguration

Configure a 1-hour grace period so transient CI queue delays don't produce false pages. A 25-hour heartbeat with a 1-hour grace catches any full day where Bypass-protected integration tests didn't pass.

What You Built

| What | How | |------|-----| | Basic stub tests | ExUnit tests with Bypass.expect_once/3 for happy and error paths | | Server-down tests | Bypass.down/1 to validate client error handling | | Mix task smoke test | mix check_bypass verifies Bypass opens and responds | | CI heartbeat | Vigilmon heartbeat — alerts when Bypass integration tests miss | | Daily schedule | GitHub Actions cron at 05:00 UTC | | Slack alerting | Vigilmon Slack notification channel |

Bypass keeps your tests fast and isolated, but a broken stub configuration can silently leak requests to real APIs or cause test suites to fail in misleading ways. Vigilmon ensures your Bypass-based integration tests stay healthy across every push and every day.

Next Steps

  • Add request body and header assertions inside Bypass expect blocks to catch serialization regressions
  • Use Bypass.stub/3 for repeated-call endpoints and Bypass.expect_once/3 for single-call endpoints to catch call-count bugs early
  • Instrument test duration with Telemetry to alert on slowdowns that indicate real network calls sneaking through
  • Set up a Vigilmon status page showing Bypass test health over time for your QA team

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles