tutorial

How to Monitor ExVCR (Elixir) with Vigilmon

Learn how to keep ExVCR cassettes fresh, monitor CI pipeline health, and detect HTTP dependency regressions in Elixir test suites with Vigilmon.

How to Monitor ExVCR (Elixir) with Vigilmon

ExVCR is the Elixir port of Ruby's VCR library. It intercepts outbound HTTP calls during tests, records the real responses as JSON or YAML "cassettes," and replays them on subsequent runs — making your test suite fast, deterministic, and safe to run without live external dependencies. ExVCR integrates with HTTPoison, hackney, ibrowse, and httpc.

But cassette-based testing has a fundamental tension with production reliability. Cassettes record a point-in-time snapshot of an external API. When that API changes its response schema, adds new required fields, or rotates credentials, your tests continue passing against the stale cassette while production calls fail against the live API. Without a monitoring strategy, ExVCR cassettes become a silent liability: the more they diverge from reality, the less your test suite tells you about production.

This tutorial adds a monitoring strategy to an Elixir project using ExVCR:

  • HTTP uptime monitoring for the external APIs your cassettes cover with Vigilmon
  • A CI heartbeat monitor confirming your test suite runs on schedule
  • A live API smoke test job that bypasses cassettes to detect API drift
  • Cassette freshness tracking to enforce maximum cassette age
  • Alerts on API downtime and cassette staleness

Step 1: Add ExVCR to your project

# mix.exs
defp deps do
  [
    {:exvcr, "~> 0.15", only: :test},
    {:httpoison, "~> 2.2"},
    {:jason, "~> 1.4"},
    {:req, "~> 0.4"}
  ]
end

Configure ExVCR in your test helper:

# test/test_helper.exs
ExUnit.start()

ExVCR.Config.cassette_library_dir("fixture/vcr_cassettes")
ExVCR.Config.filter_sensitive_data("Bearer [a-zA-Z0-9_.-]+", "Bearer <TOKEN>")
ExVCR.Config.filter_sensitive_data("api_key=[^&]+", "api_key=<FILTERED>")

Step 2: Use ExVCR in tests

# test/my_app/weather_client_test.exs
defmodule MyApp.WeatherClientTest do
  use ExUnit.Case
  use ExVCR.Mock, adapter: ExVCR.Adapter.Hackney

  setup_all do
    HTTPoison.start()
    :ok
  end

  test "fetches current weather for a city" do
    use_cassette "weather_london" do
      assert {:ok, %{"temperature" => temp}} = MyApp.WeatherClient.get_current("London")
      assert is_number(temp)
    end
  end

  test "returns error for unknown city" do
    use_cassette "weather_unknown_city" do
      assert {:error, :not_found} = MyApp.WeatherClient.get_current("NotARealCity12345")
    end
  end
end

The first time this test runs without a cassette file, ExVCR calls the real API and saves the response to fixture/vcr_cassettes/weather_london.json. Subsequent runs replay the cassette without network access.

Step 3: Track cassette age and fail on stale cassettes

Add a Mix task that checks cassette file ages and fails when any cassette exceeds your freshness threshold:

# lib/mix/tasks/vcr.check_freshness.ex
defmodule Mix.Tasks.Vcr.CheckFreshness do
  use Mix.Task

  @shortdoc "Fail if any ExVCR cassette is older than MAX_CASSETTE_AGE_DAYS"
  @max_age_days String.to_integer(System.get_env("MAX_CASSETTE_AGE_DAYS", "30"))

  def run(_args) do
    cassette_dir = "fixture/vcr_cassettes"
    now = System.os_time(:second)
    max_age_seconds = @max_age_days * 86_400

    stale =
      cassette_dir
      |> File.ls!()
      |> Enum.filter(&String.ends_with?(&1, ".json"))
      |> Enum.map(fn file ->
        path = Path.join(cassette_dir, file)
        %File.Stat{mtime: mtime} = File.stat!(path, time: :posix)
        age_days = div(now - mtime, 86_400)
        {file, age_days}
      end)
      |> Enum.filter(fn {_, age} -> age > @max_age_days end)

    if stale == [] do
      Mix.shell().info("All cassettes are fresh (max age: #{@max_age_days} days)")
    else
      Mix.shell().error("Stale cassettes found (older than #{@max_age_days} days):")
      Enum.each(stale, fn {file, age} ->
        Mix.shell().error("  #{file}: #{age} days old")
      end)

      Mix.shell().error("""

      Refresh stale cassettes by running tests with cassette recording enabled:
        MIX_ENV=test MIX_VCR_RECORD=true mix test

      Or delete individual cassettes and re-run the test to re-record them.
      """)

      System.halt(1)
    end
  end
end

Add to your CI pipeline:

# .github/workflows/ci.yml
- name: Check cassette freshness
  run: mix vcr.check_freshness
  env:
    MAX_CASSETTE_AGE_DAYS: 30

Step 4: Set up HTTP monitoring for external APIs your cassettes cover

For each external API your cassettes record, create a Vigilmon monitor. This gives you early warning when the real API changes or goes down — before your cassettes become dangerously stale:

  1. Sign up at vigilmon.online
  2. New Monitor → HTTP for each external API:
    • Weather API: https://api.openweathermap.org/data/2.5/weather?q=London&appid=YOURKEY
    • Payment gateway: https://api.stripe.com/v1/charges (expect 401 — that confirms reachability)
    • Any other APIs your cassettes cover
  3. Set check interval: 5 minutes (paid) or match your SLA requirement
  4. Under Advanced, set expected status code (200 for public endpoints, 401 for auth-gated ones)
  5. Save

Why external monitoring matters alongside ExVCR:

ExVCR protects your tests from flaky network conditions. But it cannot tell you when the API changes. A Vigilmon monitor on the live API tells you when:

  • The API goes down (cassette tests still pass, but production calls fail)
  • The API returns a different status code (schema change, deprecation, auth rotation)
  • The API response time degrades (future cassette refreshes may behave differently)

The combination of ExVCR (test stability) + Vigilmon (API health) gives you both fast tests and early production-drift detection.

Step 5: CI heartbeat monitoring

Create a heartbeat monitor that confirms your test suite runs on schedule. This catches CI configuration drift (cron job disabled, runner offline, pipeline removed) before it leaves you flying blind:

# .github/workflows/ci.yml
name: CI

on:
  schedule:
    - cron: '0 6 * * *'   # 06:00 UTC daily
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up Elixir
        uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.17'
          otp-version: '27'

      - name: Install dependencies
        run: mix deps.get

      - name: Check cassette freshness
        run: mix vcr.check_freshness

      - name: Run tests
        run: mix test

      - name: Ping Vigilmon CI heartbeat
        if: success()
        run: curl -fsS "${{ secrets.VIGILMON_CI_HEARTBEAT_URL }}"

In Vigilmon, create a Heartbeat monitor:

  1. New Monitor → Heartbeat
  2. Set interval to 25 hours (daily CI run with 1-hour grace)
  3. Copy the ping URL → add as VIGILMON_CI_HEARTBEAT_URL to your GitHub repository secrets

Step 6: Live API smoke tests that bypass cassettes

Run a separate smoke test job weekly that disables cassette playback and hits the live API. This validates that your cassettes still match reality and catches API drift before it reaches production:

# test/smoke/weather_api_smoke_test.exs
defmodule MyApp.Smoke.WeatherApiTest do
  use ExUnit.Case

  @moduletag :smoke

  # ExVCR is NOT loaded in this test — runs against the live API
  test "live weather API returns expected schema" do
    assert {:ok, result} = MyApp.WeatherClient.get_current("London")
    assert Map.has_key?(result, "temperature"), "API response missing 'temperature' field"
    assert Map.has_key?(result, "conditions"), "API response missing 'conditions' field"
  end
end

Run only smoke tests in a separate CI job:

# .github/workflows/smoke.yml
name: API Smoke Tests

on:
  schedule:
    - cron: '0 8 * * 1'   # Weekly, Monday 08:00 UTC

jobs:
  smoke:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.17'
          otp-version: '27'
      - run: mix deps.get
      - run: mix test --only smoke
        env:
          WEATHER_API_KEY: ${{ secrets.WEATHER_API_KEY }}
      - name: Ping smoke test heartbeat
        if: success()
        run: curl -fsS "${{ secrets.VIGILMON_SMOKE_HEARTBEAT_URL }}"

Create a second Vigilmon Heartbeat monitor for the smoke test job with an 8-day interval.

Step 7: Alerts via Slack

In Vigilmon, go to Notifications → New Channel → Slack and paste your incoming webhook URL.

Enable the channel on all monitors. ExVCR-related incidents appear as:

  • External API HTTP monitor down (upstream outage — cassettes are masking a production failure)
  • API response time spike (external API degraded — cassettes are shielding tests from the slowness)
  • CI heartbeat missed (test pipeline broken — cassette drift may be accumulating undetected)
  • Smoke test heartbeat missed (live API diverged from cassettes — schema drift detected)

Alert your engineering team on CI and smoke heartbeat misses. Alert your on-call rotation on external API downtime.

Step 8: Status page

  1. Status Pages → New Status Page in Vigilmon
  2. Add external API HTTP monitors and both heartbeat monitors
  3. Share with your team so engineers can check upstream API health before investigating test failures

README badge:

![CI Status](https://vigilmon.online/badge/your-monitor-slug)

What you've built

| What | How | |------|-----| | Cassette freshness enforcement | mix vcr.check_freshness task failing CI on stale cassettes | | External API uptime monitoring | Vigilmon HTTP monitors per upstream API | | CI pipeline heartbeat | Heartbeat ping at end of successful CI run | | Weekly live API smoke tests | Separate CI job running tests with cassettes disabled | | Smoke test heartbeat | Heartbeat monitor with 8-day interval | | API drift early warning | Vigilmon alerts when upstream API changes or goes down | | Slack alerts | Vigilmon Slack notification channel | | Status page | Vigilmon public status page |

ExVCR makes your Elixir test suite fast and deterministic. Vigilmon ensures the real APIs behind your cassettes stay healthy and that your CI pipeline keeps running — so you catch production regressions before your users do.

Next steps

  • Add cassette age to your test output using a custom ExUnit formatter so developers see cassette staleness warnings inline during mix test
  • Create one Vigilmon monitor per external API domain so API-specific outages are attributed correctly in alerts
  • Use ExVCR.Config.response_headers_blacklist to strip volatile headers from cassettes and reduce re-recording noise
  • Add Vigilmon response time monitoring to detect when upstream APIs are degrading so you can proactively re-record cassettes before they diverge

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles