tutorial

How to Monitor NimbleOptions with Vigilmon

Validate Elixir function options and configuration with NimbleOptions schemas and use Vigilmon to alert your team when option validation drifts or schema changes break callers.

How to Monitor NimbleOptions with Vigilmon

NimbleOptions is a schema-based validation library for Elixir keyword lists and option maps. It's the de facto standard across the Elixir ecosystem for validating configuration and function arguments — used inside Broadway, Finch, Req, Ecto, and dozens of other production libraries. You define a schema once, call NimbleOptions.validate!/2, and get validated options with correct types, defaults applied, and a clear error message when callers pass invalid arguments.

The validation happens at runtime, not compile time. When a NimbleOptions schema changes — adding a required option, narrowing a type, removing an option that callers still pass — those callers break at runtime with a NimbleOptions.ValidationError. In a library with many callers, a schema change without a test coverage sweep causes production failures in modules the change author never considered.

Vigilmon heartbeat monitors paired with schema validation CI checks catch these regressions before they reach production.

Why Monitor NimbleOptions?

NimbleOptions failures surface at the call site, not at the schema definition:

  • Breaking schema changes — removing an option or making an optional key required breaks every caller that relied on the old schema
  • Type coercion silently failing — a :pos_integer type check that was :non_neg_integer now rejects zero where it was previously allowed
  • Default value drift — changing a default changes behavior for every caller that relies on that default, without a compile-time warning
  • Missing required options in tests — test fixtures that pass [] to a function with required options produce confusing validation errors rather than clear test failures
  • Schema deprecations not enforced — a deprecated: "use X instead" option that still accepts values means callers never migrate off the deprecated key

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Validation test pass rate | Whether all callers pass schemas with current option sets | | ValidationError rate in production | How often callers pass invalid options at runtime | | Schema coverage | Percentage of public options-accepting functions with schemas | | Deprecated option usage rate | Whether callers are migrating off deprecated keys | | CI pass rate for option contract tests | Whether schema/caller contract tests run regularly |

Step 1: Add NimbleOptions to Your Project

# mix.exs
defp deps do
  [
    {:nimble_options, "~> 1.1"},
    # rest of your deps
  ]
end

mix deps.get

Define and validate a schema:

# lib/my_app/http_client.ex
defmodule MyApp.HttpClient do
  @options_schema NimbleOptions.new!([
    base_url: [
      type: :string,
      required: true,
      doc: "Base URL for all requests"
    ],
    timeout: [
      type: :pos_integer,
      default: 5_000,
      doc: "Request timeout in milliseconds"
    ],
    retries: [
      type: :non_neg_integer,
      default: 3,
      doc: "Number of retry attempts on failure"
    ],
    pool_size: [
      type: :pos_integer,
      default: 10,
      doc: "Connection pool size"
    ],
    headers: [
      type: {:list, :any},
      default: [],
      doc: "Default headers to include with every request"
    ],
    telemetry_prefix: [
      type: {:list, :atom},
      default: [:my_app, :http],
      doc: "Telemetry event prefix"
    ]
  ])

  @moduledoc """
  HTTP client with connection pooling and retry logic.

  ## Options

  #{NimbleOptions.docs(@options_schema)}
  """

  def start_link(opts) do
    validated = NimbleOptions.validate!(opts, @options_schema)
    # Start the client with validated options
    GenServer.start_link(__MODULE__, validated, name: __MODULE__)
  end

  def init(opts) do
    {:ok, %{
      base_url: opts[:base_url],
      timeout: opts[:timeout],
      retries: opts[:retries],
      pool_size: opts[:pool_size],
      headers: opts[:headers],
      telemetry_prefix: opts[:telemetry_prefix]
    }}
  end
end

Step 2: Write Schema Contract Tests

For each NimbleOptions schema, write tests that cover the contract explicitly:

# test/my_app/http_client_test.exs
defmodule MyApp.HttpClientTest do
  use ExUnit.Case, async: true

  @valid_opts [base_url: "https://api.example.com"]

  describe "option schema validation" do
    test "accepts valid minimal options" do
      assert {:ok, validated} = NimbleOptions.validate(@valid_opts, MyApp.HttpClient.__schema__())
      assert validated[:base_url] == "https://api.example.com"
      assert validated[:timeout] == 5_000  # default
      assert validated[:retries] == 3      # default
    end

    test "rejects missing required base_url" do
      assert {:error, %NimbleOptions.ValidationError{} = err} =
               NimbleOptions.validate([], MyApp.HttpClient.__schema__())
      assert Exception.message(err) =~ "base_url"
      assert Exception.message(err) =~ "required"
    end

    test "rejects non-positive timeout" do
      assert {:error, err} =
               NimbleOptions.validate([base_url: "https://example.com", timeout: 0], MyApp.HttpClient.__schema__())
      assert Exception.message(err) =~ "timeout"
    end

    test "rejects negative retries" do
      assert {:error, err} =
               NimbleOptions.validate([base_url: "https://example.com", retries: -1], MyApp.HttpClient.__schema__())
      assert Exception.message(err) =~ "retries"
    end

    test "applies defaults" do
      assert {:ok, validated} = NimbleOptions.validate(@valid_opts, MyApp.HttpClient.__schema__())
      assert validated[:pool_size] == 10
      assert validated[:headers] == []
      assert validated[:telemetry_prefix] == [:my_app, :http]
    end

    test "rejects unknown options" do
      assert {:error, err} =
               NimbleOptions.validate([base_url: "https://example.com", unknown_key: true], MyApp.HttpClient.__schema__())
      assert Exception.message(err) =~ "unknown_key"
    end
  end
end

Expose the schema for testing:

# In MyApp.HttpClient
def __schema__, do: @options_schema

Step 3: Add Schema Coverage Check

Track which option-accepting functions have NimbleOptions schemas:

# lib/mix/tasks/check_option_schemas.ex
defmodule Mix.Tasks.CheckOptionSchemas do
  use Mix.Task

  @shortdoc "Verify key public functions have NimbleOptions schemas"

  # Functions that accept options keyword lists and should have schemas
  @option_functions [
    {MyApp.HttpClient, :start_link, 1},
    {MyApp.Worker, :start_link, 1},
    {MyApp.Mailer, :deliver, 2}
  ]

  def run(_args) do
    missing =
      @option_functions
      |> Enum.reject(fn {mod, fun, _arity} ->
        # Check if the module exports __schema__/0
        Code.ensure_loaded!(mod)
        function_exported?(mod, :__schema__, 0)
      end)
      |> Enum.map(fn {mod, fun, arity} -> "#{inspect(mod)}.#{fun}/#{arity}" end)

    if missing == [] do
      Mix.shell().info("All option-accepting functions have NimbleOptions schemas.")
    else
      Mix.shell().error("Missing NimbleOptions schema for:\n" <> Enum.join(missing, "\n"))
      exit({:shutdown, 1})
    end
  end
end

Step 4: Run Schema Tests in CI

# .github/workflows/options.yml
name: Options Contract Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  option-contracts:
    name: NimbleOptions Schema Tests
    runs-on: ubuntu-latest
    env:
      VIGILMON_OPTIONS_HEARTBEAT_URL: ${{ secrets.VIGILMON_OPTIONS_HEARTBEAT_URL }}
    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.16'
          otp-version: '26'

      - name: Cache deps
        uses: actions/cache@v3
        with:
          path: deps
          key: ${{ runner.os }}-deps-${{ hashFiles('mix.lock') }}

      - run: mix deps.get

      - name: Check option schema coverage
        run: mix check_option_schemas

      - name: Run schema contract tests
        run: mix test --color --seed 0

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS "$VIGILMON_OPTIONS_HEARTBEAT_URL"

Step 5: Instrument Validation Errors in Production

Track ValidationError occurrences so you know when callers are passing invalid options:

# lib/my_app/validated_options.ex
defmodule MyApp.ValidatedOptions do
  require Logger

  @doc """
  Validate options against a schema, logging and emitting telemetry on failure.
  Raises on invalid options — same as NimbleOptions.validate!/2 but with observability.
  """
  def validate!(opts, schema, context \\ []) do
    case NimbleOptions.validate(opts, schema) do
      {:ok, validated} ->
        validated

      {:error, %NimbleOptions.ValidationError{} = err} ->
        message = Exception.message(err)
        caller = Keyword.get(context, :caller, "unknown")

        Logger.error("NimbleOptions validation failed",
          caller: caller,
          error: message,
          opts_keys: Keyword.keys(opts)
        )

        :telemetry.execute(
          [:my_app, :options, :validation_error],
          %{count: 1},
          %{caller: caller}
        )

        raise err
    end
  end
end

Expose via health check:

defp check_options_validation do
  # Validate a known-good fixture against the schema
  fixture = [base_url: "https://health.internal"]
  case NimbleOptions.validate(fixture, MyApp.HttpClient.__schema__()) do
    {:ok, _} -> :ok
    {:error, err} ->
      require Logger
      Logger.error("Schema health check failed: #{Exception.message(err)}")
      :error
  end
end

Step 6: Create a Heartbeat Monitor in Vigilmon

  1. Sign in at vigilmon.online
  2. Click New Monitor → Heartbeat
  3. Name it NimbleOptions Contract Tests — main branch
  4. Set the expected interval to 25 hours — tests run on every push to main
  5. Copy the URL and store it as VIGILMON_OPTIONS_HEARTBEAT_URL in your CI secrets

If schema contract tests fail — due to a breaking schema change or missing test coverage — no heartbeat is sent and Vigilmon alerts you.

Step 7: Alerting

In Vigilmon, configure Notifications → New Channel:

Slack:

🔴 MISSED: NimbleOptions Contract Tests — main branch
Last passing options tests: 30 hours ago (expected every 25 hours)
Action: Check CI — possible schema change broke caller contract tests

Email for production ValidationError spikes — configure your observability stack to alert when [:my_app, :options, :validation_error] telemetry exceeds a threshold, and route those alerts to the team email via Vigilmon or your alerting provider.

Common NimbleOptions Pitfalls

Schema changes that break callers:

# Before: timeout was optional with default
timeout: [type: :pos_integer, default: 5_000]

# After: required — breaks any caller that relied on the default
timeout: [type: :pos_integer, required: true]

Prefer deprecating old behavior before removing it:

timeout: [
  type: :pos_integer,
  default: 5_000,
  deprecated: "Set timeout explicitly; the default will be removed in v2.0"
]

Type mismatch between schema and struct:

# Schema says :string but struct field is :atom
@options_schema NimbleOptions.new!([
  level: [type: :atom, default: :info]
])

# Don't forget to convert in the consumer
def init(opts) do
  %{level: opts[:level]}  # already an atom — no conversion needed
end

Not validating in the right place:

Validate at the boundary — in start_link/1, new/1, or the first public function that accepts options — not deep in a private function. Early validation produces clear error messages at the call site.

# Good: validate at the public boundary
def start_link(opts) do
  validated = NimbleOptions.validate!(opts, @schema)
  GenServer.start_link(__MODULE__, validated)
end

# Bad: validate deep in a private function — error message is confusing
defp setup(opts) do
  NimbleOptions.validate!(opts, @schema)  # ← hard to trace back to the caller
end

What You've Built

| What | How | |------|-----| | NimbleOptions CI integration | Schema contract tests on every push | | Schema coverage gate | Custom Mix task checking exported schemas | | Validation error observability | Telemetry + Logger on production failures | | Schema health check | Known-good fixture validation in /health endpoint | | CI monitoring | Vigilmon heartbeat — alerts when tests miss | | Failure alerting | Slack notification channel |

NimbleOptions makes function argument validation declarative and self-documenting. Vigilmon ensures the tests that protect those schemas never silently stop running.

Next Steps

  • Generate documentation directly from schemas using NimbleOptions.docs/1 in your @moduledoc
  • Add property-based tests with StreamData to generate valid and invalid option combinations automatically
  • Track deprecated option usage in telemetry to measure migration progress
  • Set up a separate heartbeat for integration tests that exercise the full options validation stack end-to-end

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles