How to Monitor Ecto.Repo with Vigilmon

Ecto.Repo is the central module in Ecto that wraps a database connection pool. Every call to Repo.get/2, Repo.insert/2, Repo.update/2, Repo.delete/2, Repo.all/2, and Repo.transaction/2 flows through the Repo. Under the hood it owns a pool of database connections (via DBConnection), manages checkout and checkin, handles timeouts, and maps query results to Elixir structs through your schemas.

Because everything talks to the database through the Repo, it is the single highest-leverage monitoring target in most Elixir applications. A pool exhaustion event silently queues or rejects all database operations across every feature of your app. A slow query that holds a connection degrades unrelated requests. A Postgres failover that the Repo hasn't recovered from yet causes all queries to error with connection exceptions that propagate up as 500s.

Vigilmon lets you verify that your Repo can execute queries and alert when connection pool health degrades.

Why Monitor Ecto.Repo?

Repo failures have wide blast radius — they affect every feature simultaneously:

• Connection pool exhaustion — all connections checked out and waiting; new callers queue until the pool timeout fires, returning {:error, :timeout} to the application layer
• Slow query starvation — a single slow query holds a connection; under concurrent load, one slow query causes pool exhaustion for everyone else
• Postgres failover recovery lag — after a primary failover, the Repo's connection pool must drain stale connections and reconnect; during this window all queries fail
• Migration-triggered lock contention — a running migration holds locks on tables; concurrent application queries that touch those tables queue until the lock is released
• Idle connection termination — the database or load balancer closes idle connections without informing the pool; the Repo returns closed-connection errors until DBConnection reconnects
• Max connections exceeded — deploying additional nodes without adjusting pool_size can push total connections beyond Postgres's max_connections limit, causing connection refusal

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Pool checkout wait time | How long callers wait before getting a connection | | Pool queue depth | Number of callers waiting for a free connection | | Query duration (p50/p95/p99) | Latency distribution and slow query detection | | Pool timeout rate | Frequency of DBConnection.ConnectionError from timeouts | | Active connections | Connections currently checked out vs pool size | | Idle connections | Connections in the pool not being used | | Heartbeat query freshness | Whether the Repo can execute any query at all |

Step 1: Configure the Repo with Observability in Mind

Set pool_size explicitly and enable telemetry logging:

# config/runtime.exs
config :my_app, MyApp.Repo,
  url: System.fetch_env!("DATABASE_URL"),
  pool_size: String.to_integer(System.get_env("POOL_SIZE", "10")),
  queue_target: 50,      # warn if checkout takes > 50ms (milliseconds)
  queue_interval: 1_000, # measured over rolling 1-second windows
  migration_lock: :pg_advisory_lock,
  log: :debug

queue_target and queue_interval are DBConnection parameters. When checkout time exceeds queue_target for more than half a queue_interval window, DBConnection logs a warning automatically. Setting them explicitly surfaces pool pressure before it becomes exhaustion.

Step 2: Attach Ecto Telemetry Handlers

Ecto emits [:my_app, :repo, :query] events (with otp_app prefix) for every query. Attach handlers to track duration and failure:

# lib/my_app/application.ex
def start(_type, _args) do
  :telemetry.attach_many(
    "my-app-ecto-handler",
    [
      [:my_app, :repo, :query],
      [:db_connection, :checkout]
    ],
    &MyApp.RepoTelemetry.handle_event/4,
    nil
  )

  # ... rest of start/2
end

# lib/my_app/repo_telemetry.ex
defmodule MyApp.RepoTelemetry do
  require Logger

  @slow_query_threshold_ms 500

  def handle_event([:my_app, :repo, :query], measurements, meta, _config) do
    query_time_ms = div(measurements.query_time, 1_000_000)
    total_time_ms = div(measurements.total_time, 1_000_000)

    :telemetry.execute(
      [:my_app, :repo, :query_duration],
      %{duration_ms: total_time_ms},
      %{source: meta.source, result: if(match?({:ok, _}, meta.result), do: "ok", else: "error")}
    )

    if query_time_ms > @slow_query_threshold_ms do
      Logger.warning("[Repo] slow query #{query_time_ms}ms on #{meta.source}: #{meta.query}")
    end

    if match?({:error, _}, meta.result) do
      Logger.error("[Repo] query error on #{meta.source}: #{inspect(meta.result)}")
    end
  end

  def handle_event([:db_connection, :checkout], measurements, meta, _config) do
    wait_ms = div(measurements.idle_time, 1_000_000)

    :telemetry.execute(
      [:my_app, :repo, :checkout_wait],
      %{wait_ms: wait_ms},
      %{pool: meta[:pool]}
    )
  end
end

Step 3: Track Pool State with a Poller

DBConnection exposes pool state through DBConnection.get_connection_metrics/1. Poll it periodically:

# lib/my_app/repo_poller.ex
defmodule MyApp.RepoPoller do
  use GenServer
  require Logger

  @poll_interval :timer.seconds(30)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:poll, state) do
    report_pool_metrics()
    schedule()
    {:noreply, state}
  end

  defp report_pool_metrics do
    # Ecto exposes pool status via the repo's DBConnection pool
    pool_pid = Process.whereis(MyApp.Repo)

    if pool_pid do
      # Query process info for the pool supervisor
      {:links, linked} = Process.info(pool_pid, :links)
      connection_count = Enum.count(linked)

      :telemetry.execute(
        [:my_app, :repo, :pool],
        %{connection_count: connection_count},
        %{repo: MyApp.Repo}
      )
    end
  rescue
    e ->
      Logger.warning("[RepoPoller] error collecting pool metrics: #{inspect(e)}")
  end

  defp schedule, do: Process.send_after(self(), :poll, @poll_interval)
end

For a more complete pool snapshot, use Ecto.Adapters.SQL.query!/3 to query Postgres's pg_stat_activity directly:

defp report_postgres_pool_metrics do
  result =
    Ecto.Adapters.SQL.query!(
      MyApp.Repo,
      """
      SELECT
        count(*) FILTER (WHERE state = 'active') AS active,
        count(*) FILTER (WHERE state = 'idle') AS idle,
        count(*) FILTER (WHERE wait_event_type = 'Lock') AS waiting_on_lock
      FROM pg_stat_activity
      WHERE datname = current_database()
        AND pid <> pg_backend_pid()
      """,
      []
    )

  [[active, idle, waiting]] = result.rows

  :telemetry.execute(
    [:my_app, :repo, :pg_pool],
    %{active: active, idle: idle, waiting_on_lock: waiting},
    %{}
  )

  Logger.debug("[Repo] pg_stat_activity: active=#{active} idle=#{idle} waiting=#{waiting}")
rescue
  e -> Logger.warning("[RepoPoller] pg_stat_activity error: #{inspect(e)}")
end

Step 4: Build a Repo Health Endpoint

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    repo_check = check_repo()
    pool_check = check_pool_pressure()

    checks = %{
      repo: format_check(repo_check),
      pool_pressure: pool_check
    }

    status = if match?({:ok, _}, repo_check), do: 200, else: 503

    conn
    |> put_status(status)
    |> json(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      checks: checks
    })
  end

  defp check_repo do
    start = System.monotonic_time(:millisecond)

    result =
      case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", [], timeout: 3_000) do
        {:ok, _} -> {:ok, System.monotonic_time(:millisecond) - start}
        {:error, reason} -> {:error, inspect(reason)}
      end

    result
  rescue
    e -> {:error, inspect(e)}
  end

  defp check_pool_pressure do
    # Return pool size vs process estimate as a simple signal
    pool_size = Application.get_env(:my_app, MyApp.Repo)[:pool_size] || 10
    %{configured_pool_size: pool_size}
  end

  defp format_check({:ok, latency_ms}), do: %{status: "ok", latency_ms: latency_ms}
  defp format_check({:error, reason}), do: %{status: "error", reason: reason}
end

Register the route:

# lib/my_app_web/router.ex
scope "/health", MyAppWeb do
  get "/", HealthController, :index
end

Step 5: Add a Repo Liveness Heartbeat

# lib/my_app/repo_heartbeat.ex
defmodule MyApp.RepoHeartbeat do
  use GenServer
  require Logger

  @heartbeat_url System.get_env("VIGILMON_REPO_HEARTBEAT_URL")
  @interval :timer.minutes(1)
  @query_timeout 5_000

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:check, state) do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", [], timeout: @query_timeout) do
      {:ok, _} ->
        ping_vigilmon()

      {:error, reason} ->
        Logger.error("[RepoHeartbeat] DB unreachable: #{inspect(reason)}")
    end

    schedule()
    {:noreply, state}
  end

  defp ping_vigilmon do
    if @heartbeat_url do
      :httpc.request(
        :get,
        {String.to_charlist(@heartbeat_url), []},
        [{:timeout, 5_000}],
        []
      )
    end
  end

  defp schedule, do: Process.send_after(self(), :check, @interval)
end

Add to your supervision tree:

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  MyAppWeb.Endpoint,
  MyApp.RepoPoller,
  MyApp.RepoHeartbeat
]

Step 6: Set Up Monitoring in Vigilmon

HTTP Monitor (Repo health endpoint)

1. Sign in at vigilmon.online
2. Click New Monitor → HTTP
3. Set URL to https://your-app.example.com/health
4. Expected status: 200
5. Body must contain: "repo":{"status":"ok"}
6. Check interval: 60 seconds

Heartbeat Monitor (Repo liveness)

1. Click New Monitor → Heartbeat
2. Name: Ecto.Repo DB Liveness
3. Expected interval: 2 minutes
4. Copy the URL → set as VIGILMON_REPO_HEARTBEAT_URL in your environment

A missed heartbeat means the Repo cannot execute any query — every database-backed feature in your application is failing.

Step 7: Alerting

In Vigilmon, configure Notifications → New Channel:

Slack for Repo health failure:

🔴 DOWN: Ecto.Repo — MyApp
Monitor: /health returned repo.status = "error"
Action: Check DATABASE_URL, Postgres connectivity, and connection pool exhaustion

PagerDuty for heartbeat miss:

A missed heartbeat on Repo liveness means total database unavailability. Configure a high-priority PagerDuty integration in Vigilmon's notification settings.

Prometheus alert for slow queries (if using PromEx or Telemetry.Metrics exporter):

- alert: EctoRepoSlowQueryHigh
  expr: histogram_quantile(0.95, rate(my_app_repo_query_duration_ms_bucket[5m])) > 500
  for: 5m
  annotations:
    summary: "p95 Repo query latency above 500ms — check for slow queries or lock contention"

- alert: EctoRepoConnectionPoolPressure
  expr: my_app_repo_pg_pool_waiting_on_lock > 3
  for: 2m
  annotations:
    summary: "More than 3 connections waiting on database locks — possible migration or long transaction"

Common Ecto.Repo Issues and Fixes

Pool exhaustion under load:

# Increase pool_size in runtime config, but also audit checkout_timeout
# Default checkout_timeout is 5_000ms — if callers are queuing longer than this, they error
config :my_app, MyApp.Repo,
  pool_size: 20,
  ownership_timeout: 60_000,  # for Sandbox in tests
  timeout: 15_000             # query timeout

Idle connection termination by load balancer:

# Keep connections alive with TCP keepalive options via the socket options
# For Postgrex (the underlying adapter):
config :my_app, MyApp.Repo,
  socket_options: [:inet6]
  # Or configure PgBouncer keepalive on the load balancer side

Slow queries holding connections:

# Set explicit timeouts on long-running queries
# This ensures one slow operation doesn't hold a connection indefinitely
MyApp.Repo.all(query, timeout: 10_000)
MyApp.Repo.transaction(fn -> ... end, timeout: 30_000)

Migration lock contention:

# Run migrations with a lock timeout to prevent blocking production traffic
# In migration files or via Ecto.Adapters.SQL.query:
Ecto.Adapters.SQL.query!(
  MyApp.Repo,
  "SET lock_timeout = '5s'",
  []
)

What You've Built

| What | How | |------|-----| | Query telemetry | Ecto query events with duration and result tags | | Pool pressure monitoring | pg_stat_activity queried by poller GenServer | | Slow query detection | Warning log when query time exceeds threshold | | Repo health endpoint | /health runs SELECT 1 with a 3-second timeout | | Liveness heartbeat | GenServer pinging Vigilmon only on successful query | | HTTP and heartbeat monitors | Vigilmon checks endpoint + heartbeat liveness | | Real-time alerting | Slack for endpoint failure, PagerDuty for heartbeat miss |

Ecto.Repo is the backbone of your application's data layer. Vigilmon makes sure that backbone stays healthy under production load.

Next Steps

• Export Ecto telemetry to Grafana via PromEx for per-table query breakdowns
• Add Vigilmon response-time checks on /health to catch slow connection checkout before it cascades
• Create separate heartbeat monitors for read-replica Repos if your application uses them
• Use Vigilmon incident history to correlate pool exhaustion events with traffic spikes or deployments

Get started free at vigilmon.online.