How to Monitor Wallaby with Vigilmon

Wallaby is Elixir's concurrent browser testing framework. It drives real browsers — Chrome via ChromeDriver, or Firefox via GeckoDriver — to run end-to-end integration tests against your Phoenix app. Because Wallaby spins up browser sessions, connects to WebDriver, and often talks to a running application server, there are more moving parts than a typical unit test suite.

When something goes wrong in a Wallaby test run — ChromeDriver crashes, the test node can't reach the app, or a CI runner runs out of memory — you want to know fast. This tutorial shows you how to add observability to your Wallaby test infrastructure with Vigilmon.

Why Monitor Wallaby?

Browser tests are expensive and brittle by nature. Problems that silent CI logs won't surface:

• ChromeDriver crashes mid-suite — tests hang until timeout, CI burns 30 minutes before failing
• Scheduled browser test runs stop firing — a cron misconfiguration or runner exhaustion means regressions go undetected
• ChromeDriver version mismatch after a Chrome update — tests fail with cryptic WebDriver protocol errors
• Test environment drift — the test app server stops responding between runs

Vigilmon adds a heartbeat layer: if your scheduled test run doesn't fire and report success, you get an alert before the next deploy ships an untested build.

Key Metrics to Track

| Metric | Why It Matters | |--------|---------------| | Test run heartbeat | Confirms the scheduled run fired and completed | | ChromeDriver process health | ChromeDriver crashing silently blocks all browser sessions | | Test suite pass/fail rate | Trend data shows flakiness before it becomes blocking | | Run duration | Sudden increases signal resource contention or new slow tests | | Application server health | Wallaby needs the app running; a dead server = 100% test failure |

Step 1: Add a Health Endpoint to Your Test Application

Wallaby tests run against a Phoenix server started in test mode. Add a minimal health check so you can verify the test app is actually serving:

# lib/my_app_web/plugs/health_check.ex
defmodule MyAppWeb.Plugs.HealthCheck do
  import Plug.Conn

  def init(opts), do: opts

  def call(%Plug.Conn{request_path: "/health"} = conn, _opts) do
    conn
    |> put_resp_content_type("application/json")
    |> send_resp(200, ~s({"status":"ok"}))
    |> halt()
  end

  def call(conn, _opts), do: conn
end

# lib/my_app_web/endpoint.ex
plug MyAppWeb.Plugs.HealthCheck
plug MyAppWeb.Router

Configure Wallaby to start the server on a known port in test mode:

# config/test.exs
config :my_app, MyAppWeb.Endpoint,
  http: [port: 4002],
  server: true

config :wallaby,
  driver: Wallaby.Chrome,
  chrome: [headless: true],
  base_url: "http://localhost:4002"

Step 2: Add a Heartbeat Ping After Test Suite Completion

Wallaby integrates with ExUnit. Use ExUnit.after_suite/1 to ping Vigilmon after every successful run:

# test/test_helper.exs
ExUnit.start()

Application.ensure_all_started(:wallaby)

ExUnit.after_suite(fn %{failures: failures, excluded: _excluded} ->
  if failures == 0 do
    ping_vigilmon_heartbeat()
  end
end)

defp ping_vigilmon_heartbeat do
  url = System.get_env("VIGILMON_WALLABY_HEARTBEAT_URL")

  if url do
    case :httpc.request(:get, {String.to_charlist(url), []}, [timeout: 5000], []) do
      {:ok, _} ->
        IO.puts("Vigilmon heartbeat sent.")
      {:error, reason} ->
        IO.warn("Vigilmon heartbeat failed: #{inspect(reason)}")
    end
  end
end

Or, if you prefer using Req (already in most Phoenix projects):

# test/test_helper.exs
ExUnit.after_suite(fn %{failures: 0} ->
  url = System.get_env("VIGILMON_WALLABY_HEARTBEAT_URL")
  if url, do: Req.get!(url, receive_timeout: 5_000)
end)

ExUnit.after_suite(fn _ -> :ok end)

Set the environment variable in your CI environment:

# .env.ci or CI settings
VIGILMON_WALLABY_HEARTBEAT_URL=https://vigilmon.online/heartbeat/your-unique-token

Step 3: Create a Heartbeat Monitor in Vigilmon

1. Sign in at vigilmon.online
2. Click New Monitor → Heartbeat
3. Name it Wallaby Browser Tests
4. Set the expected interval to match your test schedule (e.g. 1 hour for every-hour CI, 24 hours for nightly runs)
5. Set a grace period of 10 minutes to allow for slow runners
6. Copy the heartbeat URL and add it to your CI environment variables

Vigilmon will alert you if no ping arrives within the window. A silent miss means either:

• The CI run was never triggered
• The run failed before the heartbeat fired (test failures or setup crashes)
• ChromeDriver died and ExUnit never reached after_suite

Step 4: Monitor ChromeDriver with an HTTP Check

ChromeDriver exposes a status endpoint you can poll:

curl http://localhost:9515/status
# {"value":{"build":{"version":"114.0.5735.90"},"message":"ChromeDriver ready for new sessions","ready":true}}

If you run a persistent ChromeDriver process (not spawned per test run), add an HTTP monitor in Vigilmon:

1. Click New Monitor → HTTP
2. URL: http://your-ci-host:9515/status
3. Check for keyword: "ready":true
4. Interval: 1 minute

This catches ChromeDriver version mismatches (the endpoint returns "ready":false) and process crashes (connection refused).

Step 5: Set Up Alerts

In Vigilmon, go to Notifications → New Channel and configure Slack, email, or PagerDuty:

Recommended alert rules for Wallaby:

• Heartbeat missed → page on-call immediately (this means untested code could ship)
• ChromeDriver down → alert the dev team (next test run will fail)
• App server HTTP check fails → alert immediately (test environment is broken)

Example Slack message when the heartbeat is missed:

🔴 MISSED: Wallaby Browser Tests heartbeat
Expected every 1 hour — last ping was 73 minutes ago
This may indicate CI failure or test environment outage

Step 6: Track Test Duration Trends

Wallaby test suites slow down as the app grows. Use Vigilmon's response time history on the heartbeat monitor to spot duration regressions:

• A sudden 2x increase often means a new test has a missing async: false annotation and is serializing the suite
• A gradual creep suggests database fixture growth or heavier page renders
• Spikes correlate with CI runner resource contention

Combine this with Wallaby's own timing output:

# Enable verbose timing in CI
MIX_ENV=test mix test --trace 2>&1 | tee test_output.log

What You've Built

| What | How | |------|-----| | Test run liveness | Heartbeat ping in ExUnit.after_suite/1 | | Missed run alerts | Vigilmon Heartbeat monitor with interval matching CI schedule | | ChromeDriver health | Vigilmon HTTP monitor on /status endpoint | | App server health | Vigilmon HTTP monitor on /health plug | | Slack alerts | Vigilmon notification channel | | Duration trends | Vigilmon response time history |

Your Wallaby suite runs fast and is invisible when healthy. Now you'll know the moment it stops.

Next Steps

• Add separate heartbeat monitors for different test tags (e.g. @tag :smoke vs full regression)
• Use Vigilmon's status page to show test environment health to the broader team
• Set up a monitor for your staging environment's health endpoint so Wallaby tests always have a live target
• Consider adding a Wallaby helper that pings Vigilmon mid-suite for long-running test jobs

Get started free at vigilmon.online.