tutorial

How to Monitor ExDoc with Vigilmon

Track ExDoc documentation builds in your Elixir CI pipeline and use Vigilmon heartbeat monitors to alert your team when doc generation stops running or breaks.

How to Monitor ExDoc with Vigilmon

ExDoc is the documentation generation tool for Elixir projects. It reads your module docstrings, @doc annotations, and @moduledoc blocks, then produces beautiful HTML and EPUB documentation with full-text search, cross-references, and syntax-highlighted code examples. The output is what you see on hexdocs.pm for every published Hex package.

Documentation quality degrades silently. A broken @doc annotation compiles without error. A missing @spec omits type information from the generated docs. A doc build that never runs means outdated docs ship alongside the updated code. Unlike type errors or failing tests, broken documentation rarely blocks a deploy — which is exactly why you need monitoring to catch it.

Vigilmon heartbeat monitors ensure your documentation pipeline stays active and healthy.

Why Monitor ExDoc?

ExDoc failures are quiet and costly:

  • Broken doc generation — a misconfigured mix.exs or incompatible ExDoc version causes builds to fail silently in CI when doc generation is optional
  • Missing modules — modules excluded from docs via @moduledoc false unintentionally, leaving gaps in published documentation
  • Dead cross-references — t:MyApp.User.t/0 references that point to removed types, breaking the generated HTML
  • Outdated published docs — Hex packages published without a matching docs push mean users read stale documentation
  • Broken doc tests — doctest MyModule failures that pass locally but break in CI due to environment differences

Catching these issues requires treating documentation as a first-class CI artifact, not an afterthought.

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Doc build CI pass rate | Whether ExDoc runs and generates output successfully | | Broken cross-reference count | Number of invalid See also or type reference links | | Doc coverage by module | Percentage of public functions with @doc | | Doc build duration | Regression indicator when new modules inflate build time | | Time since last doc publish | Whether published docs are current with the latest release |

Step 1: Add ExDoc to Your Project

# mix.exs
defp deps do
  [
    {:ex_doc, "~> 0.34", only: :dev, runtime: false},
    # rest of your deps
  ]
end

mix deps.get

Configure documentation metadata in mix.exs:

# mix.exs
def project do
  [
    app: :my_app,
    version: "0.1.0",
    elixir: "~> 1.16",
    name: "MyApp",
    source_url: "https://github.com/myorg/my_app",
    homepage_url: "https://myapp.example.com",
    docs: [
      main: "readme",
      extras: ["README.md", "CHANGELOG.md"],
      groups_for_modules: [
        "Core": [MyApp.Core, MyApp.Core.Schema],
        "HTTP": [MyApp.Web, MyApp.Web.Router]
      ],
      before_closing_head_tag: &before_closing_head_tag/1
    ],
    # rest of project config
  ]
end

defp before_closing_head_tag(:html) do
  """
  <script defer src="https://cdn.example.com/analytics.js"></script>
  """
end
defp before_closing_head_tag(_), do: ""

Generate docs locally to verify output:

mix docs
open doc/index.html

Step 2: Add a Doc Coverage Check

ExDoc doesn't enforce doc coverage by default. Add a Mix task to catch missing @doc annotations on public functions:

# lib/mix/tasks/check_docs.ex
defmodule Mix.Tasks.CheckDocs do
  use Mix.Task

  @shortdoc "Verify all public functions in core modules have @doc"

  @required_modules [
    MyApp.Accounts,
    MyApp.Orders,
    MyApp.Payments
  ]

  def run(_args) do
    missing =
      @required_modules
      |> Enum.flat_map(fn mod ->
        Code.ensure_loaded!(mod)
        fns = mod.__info__(:functions) |> Enum.reject(fn {name, _} -> String.starts_with?(to_string(name), "__") end)

        fns
        |> Enum.reject(fn {name, arity} ->
          case Code.fetch_docs(mod) do
            {:docs_v1, _, _, _, _, _, docs} ->
              Enum.any?(docs, fn
                {{:function, ^name, ^arity}, _, _, %{"en" => _}, _} -> true
                _ -> false
              end)
            _ -> false
          end
        end)
        |> Enum.map(fn {name, arity} -> "#{inspect(mod)}.#{name}/#{arity}" end)
      end)

    if missing == [] do
      Mix.shell().info("All required public functions are documented.")
    else
      Mix.shell().error("Missing @doc on:\n" <> Enum.join(missing, "\n"))
      exit({:shutdown, 1})
    end
  end
end

Step 3: Run ExDoc in CI

# .github/workflows/docs.yml
name: Documentation

on:
  push:
    branches: [main]
  pull_request:
  release:
    types: [published]

jobs:
  docs:
    name: Generate and Verify Docs
    runs-on: ubuntu-latest
    env:
      VIGILMON_EXDOC_HEARTBEAT_URL: ${{ secrets.VIGILMON_EXDOC_HEARTBEAT_URL }}
    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          elixir-version: '1.16'
          otp-version: '26'

      - name: Cache deps
        uses: actions/cache@v3
        with:
          path: deps
          key: ${{ runner.os }}-deps-${{ hashFiles('mix.lock') }}

      - run: mix deps.get

      - name: Check doc coverage
        run: mix check_docs

      - name: Generate docs
        run: mix docs

      - name: Verify doc output exists
        run: |
          test -f doc/index.html || (echo "Doc generation produced no output" && exit 1)
          test -d doc/api-reference.html || echo "Warning: no API reference generated"

      - name: Upload doc artifact
        uses: actions/upload-artifact@v4
        with:
          name: docs-${{ github.sha }}
          path: doc/

      - name: Ping Vigilmon heartbeat
        if: success()
        run: curl -fsS "$VIGILMON_EXDOC_HEARTBEAT_URL"

The if: success() guard ensures Vigilmon only receives a heartbeat when doc generation actually succeeds — a failed build produces no ping and Vigilmon alerts you.

Step 4: Create a Heartbeat Monitor in Vigilmon

  1. Sign in at vigilmon.online
  2. Click New Monitor → Heartbeat
  3. Name it ExDoc Build — main branch
  4. Set the expected interval to 25 hours — doc builds on every push to main; 25 hours catches a missed day
  5. Copy the URL and store it as VIGILMON_EXDOC_HEARTBEAT_URL in your CI secrets

If ExDoc fails or CI is skipped, no heartbeat is sent and Vigilmon alerts you within the grace period.

Step 5: Track Doc Build Health in Your Health Endpoint

# lib/my_app_web/plugs/health_check.ex
defmodule MyAppWeb.Plugs.HealthCheck do
  import Plug.Conn

  def init(opts), do: opts

  def call(%Plug.Conn{request_path: "/health"} = conn, _opts) do
    checks = %{
      database: check_db(),
      docs_freshness: check_docs_freshness()
    }

    status = if Enum.all?(checks, fn {_, v} -> v == :ok end), do: 200, else: 503

    conn
    |> put_resp_content_type("application/json")
    |> send_resp(status, Jason.encode!(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      checks: checks
    }))
    |> halt()
  end

  def call(conn, _opts), do: conn

  defp check_db do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", []) do
      {:ok, _} -> :ok
      _ -> :error
    end
  end

  defp check_docs_freshness do
    last_built_at = Application.get_env(:my_app, :exdoc_last_built_at)
    if last_built_at && DateTime.diff(DateTime.utc_now(), last_built_at, :hour) < 26 do
      :ok
    else
      :stale
    end
  end
end

Set :exdoc_last_built_at in runtime config stamped by CI:

# config/runtime.exs
if built_at = System.get_env("EXDOC_LAST_BUILT_AT") do
  config :my_app, exdoc_last_built_at: DateTime.from_unix!(String.to_integer(built_at))
end

In CI, add before the heartbeat ping:

- name: Stamp doc build time
  run: echo "EXDOC_LAST_BUILT_AT=$(date +%s)" >> $GITHUB_ENV

Step 6: Alerting

In Vigilmon, configure Notifications → New Channel:

Slack:

🔴 MISSED: ExDoc Build — main branch
Last successful doc build: 30 hours ago (expected every 25 hours)
Action: Check CI — possible broken @doc reference or ExDoc version incompatibility

Email for longer gaps:

Configure a second alert that fires when the heartbeat misses by more than 48 hours — indicating doc generation was explicitly disabled or removed from CI.

Common ExDoc Issues and Fixes

Broken cross-references:

# Warning: See `t:MyApp.OldType.t/0` — type was removed
@doc """
Returns a user record.

See `t:MyApp.User.t/0` for the struct layout.
"""

Fix by updating references when you rename or remove types. Run mix docs locally and check for warnings in the output.

Modules excluded unintentionally:

# This excludes the entire module from docs
@moduledoc false

# If you only want to hide internal docs from the index, use :nodoc instead
# (ExDoc 0.29+)
@moduledoc tags: [:internal]

Doc test failures:

# This doctest will fail if the output format changes
@doc """
    iex> MyApp.format_date(~D[2024-01-15])
    "January 15, 2024"
"""

Run mix test --only doctest in CI separately to distinguish doc test failures from unit test failures.

What You've Built

| What | How | |------|-----| | ExDoc CI integration | mix docs with artifact upload | | Doc coverage gate | Custom Mix task checking @doc on critical modules | | Doc build monitoring | Vigilmon heartbeat — alerts when build misses | | Build freshness signal | HTTP health endpoint with doc timestamp | | Deploy-time doc stamp | Runtime config from CI env var | | Failure alerting | Slack notification channel |

ExDoc makes your Elixir documentation beautiful and searchable. Vigilmon makes sure that documentation stays current.

Next Steps

  • Publish docs to GitHub Pages from the uploaded CI artifact on every release
  • Track doc coverage percentage over time to measure documentation quality improvements
  • Add a separate heartbeat for EPUB generation if your users consume offline docs
  • Use Vigilmon's response time history on /health to correlate deploys with service changes

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles