tutorial

How to Monitor Phoenix.Presence with Vigilmon

Track Phoenix.Presence connection counts, join/leave rates, and cluster-wide user tracking in your Elixir application, and use Vigilmon to alert when presence data diverges or nodes lose sync.

How to Monitor Phoenix.Presence with Vigilmon

Phoenix.Presence is the distributed presence tracking system built on top of Phoenix.PubSub. It lets your application know which users are currently online — across every node in an Erlang cluster — using a conflict-free replicated data type (CRDT) under the hood. Every time a user connects, disconnects, or updates their status, Presence handles the bookkeeping and broadcasts diffs to all subscribers.

When Presence fails, user lists go stale. A chat room shows users who disconnected hours ago. A collaborative editor lets two people edit the same section because their cursors aren't visible to each other. In a cluster, a network partition can cause presence data to diverge between nodes with no visible error.

Vigilmon lets you verify that your Presence system is tracking, syncing, and broadcasting correctly — not just that the processes are alive.

Why Monitor Phoenix.Presence?

Presence failures are subtle and insidious:

  • Stale presence list — a Channel process crash without untrack/1 leaves ghost entries in the presence map; users appear online indefinitely
  • Node sync drift — after a network partition heals, the CRDT merge may produce duplicate or missing entries depending on timing
  • Join flood — a reconnect storm (all clients reconnecting after a server restart) causes thousands of track/3 calls simultaneously, potentially overwhelming the Presence supervisor
  • Metadata staleness — metadata (status, cursor position) associated with a presence entry may become stale if the update path fails silently
  • Memory growth — in applications with high churn (users joining/leaving frequently), the Presence CRDT state can grow if garbage collection of old entries lags

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Total tracked users (cluster-wide) | Overall presence load and trend | | Join rate | Rate of new presence entries per topic | | Leave rate | Rate of removals; gap between join and leave indicates leaks | | Presence list size per topic | Topic-level load and potential staleness | | CRDT diff broadcast latency | How quickly presence changes propagate to subscribers | | Node presence divergence | Whether node-local counts match cluster-wide count |

Step 1: Set Up Phoenix.Presence

# mix.exs — Phoenix.Presence is included with Phoenix; no extra dep needed

# lib/my_app_web/presence.ex
defmodule MyAppWeb.Presence do
  use Phoenix.Presence,
    otp_app: :my_app,
    pubsub_server: MyApp.PubSub
end

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  {Phoenix.PubSub, name: MyApp.PubSub},
  MyAppWeb.Presence,
  MyAppWeb.Endpoint
]

Track users from your Channel:

# lib/my_app_web/channels/room_channel.ex
defmodule MyAppWeb.RoomChannel do
  use Phoenix.Channel
  alias MyAppWeb.Presence

  @impl true
  def join("room:" <> room_id, _payload, socket) do
    send(self(), :after_join)
    {:ok, assign(socket, :room_id, room_id)}
  end

  @impl true
  def handle_info(:after_join, socket) do
    push(socket, "presence_state", Presence.list(socket))

    {:ok, _ref} =
      Presence.track(socket, socket.assigns.user_id, %{
        online_at: System.system_time(:second),
        status: "online"
      })

    :telemetry.execute([:my_app, :presence, :join], %{count: 1}, %{topic: "room"})
    {:noreply, socket}
  end

  @impl true
  def terminate(_reason, socket) do
    :telemetry.execute([:my_app, :presence, :leave], %{count: 1}, %{topic: "room"})
    :ok
  end
end

Step 2: Instrument Presence Telemetry

# lib/my_app/telemetry.ex
def metrics do
  [
    counter("my_app.presence.join.count", tags: [:topic]),
    counter("my_app.presence.leave.count", tags: [:topic]),
    last_value("my_app.presence.count", tags: [:topic]),
    distribution("my_app.presence.sync.duration",
      unit: {:native, :millisecond},
      reporter_options: [buckets: [1, 5, 10, 50, 100]]
    )
  ]
end

Emit a presence count gauge on a schedule:

# lib/my_app/presence_poller.ex
defmodule MyApp.PresencePoller do
  use GenServer
  alias MyAppWeb.Presence

  @poll_interval :timer.seconds(30)
  @topics ["room:lobby", "room:general"]

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:poll, state) do
    Enum.each(@topics, fn topic ->
      count = topic |> Presence.list() |> map_size()

      :telemetry.execute(
        [:my_app, :presence, :count],
        %{value: count},
        %{topic: topic}
      )
    end)

    schedule()
    {:noreply, state}
  end

  defp schedule, do: Process.send_after(self(), :poll, @poll_interval)
end

Step 3: Build a Presence Health Check

Verify end-to-end presence tracking: track an entry, confirm it appears in the list, then untrack:

# lib/my_app/presence_health.ex
defmodule MyApp.PresenceHealth do
  @moduledoc """
  End-to-end Presence health check. Tracks a probe entry, verifies it appears
  in the list, then untracks. Catches Presence process failures, PubSub
  connectivity issues, and CRDT sync problems.
  """

  alias MyAppWeb.Presence

  @health_topic "__health__:presence"
  @probe_user_id "vigilmon_health_probe"
  @timeout 2_000

  def check do
    start = System.monotonic_time(:millisecond)

    # Subscribe to presence updates on the health topic
    MyApp.PubSub
    |> Phoenix.PubSub.subscribe(@health_topic)

    with {:ok, _ref} <- Presence.track(self(), @health_topic, @probe_user_id, %{probe: true}),
         :ok <- await_join(),
         list when is_map(list) <- Presence.list(@health_topic),
         true <- Map.has_key?(list, @probe_user_id),
         :ok <- Presence.untrack(self(), @health_topic, @probe_user_id) do
      latency = System.monotonic_time(:millisecond) - start
      Phoenix.PubSub.unsubscribe(MyApp.PubSub, @health_topic)
      {:ok, latency}
    else
      {:error, reason} ->
        cleanup()
        {:error, reason}

      false ->
        cleanup()
        {:error, :not_in_list}

      err ->
        cleanup()
        {:error, err}
    end
  rescue
    e ->
      cleanup()
      {:error, inspect(e)}
  end

  defp await_join do
    receive do
      %Phoenix.Socket.Broadcast{event: "presence_diff"} -> :ok
    after
      @timeout -> {:error, :join_timeout}
    end
  end

  defp cleanup do
    Phoenix.PubSub.unsubscribe(MyApp.PubSub, @health_topic)
    Presence.untrack(self(), @health_topic, @probe_user_id)
  end
end

Step 4: Add a Health Endpoint

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    presence_check = MyApp.PresenceHealth.check()

    total_users =
      ["room:lobby", "room:general"]
      |> Enum.map(&(MyAppWeb.Presence.list(&1) |> map_size()))
      |> Enum.sum()

    checks = %{
      presence: format_check(presence_check),
      total_tracked_users: total_users,
      node: Node.self() |> to_string()
    }

    status = if match?({:ok, _}, presence_check), do: 200, else: 503

    conn
    |> put_status(status)
    |> json(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      checks: checks
    })
  end

  defp format_check({:ok, latency_ms}), do: %{status: "ok", latency_ms: latency_ms}
  defp format_check({:error, reason}), do: %{status: "error", reason: inspect(reason)}
end

Step 5: Detect Presence Leaks

A presence leak — entries that never leave — inflates user counts and wastes memory. Detect them by comparing join and leave rates over time:

# lib/my_app/presence_leak_detector.ex
defmodule MyApp.PresenceLeakDetector do
  use GenServer
  require Logger

  @check_interval :timer.minutes(5)
  @max_entry_age_seconds 3600

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state), do: schedule() |> then(fn _ -> {:ok, state} end)

  def handle_info(:check, state) do
    now = System.system_time(:second)

    stale =
      MyAppWeb.Presence.list("room:lobby")
      |> Enum.filter(fn {_user_id, %{metas: metas}} ->
        metas
        |> List.first()
        |> Map.get(:online_at, now)
        |> then(fn joined_at -> now - joined_at > @max_entry_age_seconds end)
      end)

    if length(stale) > 0 do
      Logger.warning("Presence leak: #{length(stale)} entries older than #{@max_entry_age_seconds}s")

      :telemetry.execute(
        [:my_app, :presence, :leak],
        %{count: length(stale)},
        %{topic: "room:lobby"}
      )
    end

    schedule()
    {:noreply, state}
  end

  defp schedule, do: Process.send_after(self(), :check, @check_interval)
end

Step 6: Create Monitors in Vigilmon

HTTP monitor for the health endpoint:

  1. Sign in at vigilmon.online
  2. Click New Monitor → HTTP
  3. Set URL to https://your-app.example.com/health
  4. Set interval to 60 seconds
  5. Alert condition: status 200 and body contains "presence":{"status":"ok"}

Heartbeat monitor for Presence liveness:

# lib/my_app/presence_heartbeat.ex
defmodule MyApp.PresenceHeartbeat do
  use GenServer

  @heartbeat_url System.get_env("VIGILMON_PRESENCE_HEARTBEAT_URL")
  @interval :timer.minutes(1)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:check, state) do
    case MyApp.PresenceHealth.check() do
      {:ok, _latency} ->
        ping_vigilmon()

      {:error, reason} ->
        require Logger
        Logger.error("Presence health failed: #{inspect(reason)}")
    end

    schedule()
    {:noreply, state}
  end

  defp ping_vigilmon do
    if @heartbeat_url do
      :httpc.request(:get, {String.to_charlist(@heartbeat_url), []}, [{:timeout, 3000}], [])
    end
  end

  defp schedule, do: Process.send_after(self(), :check, @interval)
end

Create a Heartbeat monitor in Vigilmon with a 3-minute expected interval. A missed heartbeat means Presence tracking is broken.

Step 7: Monitor Cross-Node Presence Sync

For clustered deployments, verify that presence entries are consistent across nodes:

# lib/my_app/cluster_presence_check.ex
defmodule MyApp.ClusterPresenceCheck do
  @topic "__health__:cluster_presence"
  @probe_user "cluster_probe"
  @timeout 3_000

  def check do
    nodes = Node.list()

    if nodes == [] do
      {:ok, :single_node}
    else
      # Track a probe entry locally
      MyAppWeb.Presence.track(self(), @topic, @probe_user, %{origin: Node.self()})

      # Ask a remote node to check if the entry is visible there
      remote_node = hd(nodes)

      result =
        :rpc.call(remote_node, __MODULE__, :check_entry_visible, [@topic, @probe_user], @timeout)

      MyAppWeb.Presence.untrack(self(), @topic, @probe_user)

      case result do
        true -> {:ok, :synced}
        false -> {:error, :not_synced}
        {:badrpc, reason} -> {:error, {:rpc_failed, reason}}
      end
    end
  end

  def check_entry_visible(topic, user_id) do
    # Wait briefly for CRDT sync to propagate
    Process.sleep(500)
    topic |> MyAppWeb.Presence.list() |> Map.has_key?(user_id)
  end
end

Include this check in your health endpoint response when running in a cluster.

Step 8: Alerting

In Vigilmon, configure Notifications → New Channel:

Slack for Presence tracking failure:

🔴 DOWN: Phoenix.Presence — MyApp
Monitor: /health returning presence.status = "error"
Action: Check Presence supervisor, PubSub connectivity, and Erlang cluster state
Runbook: https://internal.example.com/runbooks/presence-failure

PagerDuty for heartbeat miss:

A missed Presence heartbeat means user tracking is broken across the application. Configure a high-priority PagerDuty integration for the heartbeat monitor.

Common Phoenix.Presence Issues and Fixes

Ghost entries — users appear online after disconnect:

# Ensure untrack is called on termination; Phoenix does this automatically
# when the Channel process that called track/3 exits, but explicit cleanup
# is safer when using Presence outside of a Channel:
@impl true
def terminate(_reason, socket) do
  MyAppWeb.Presence.untrack(socket, socket.assigns.user_id)
  :ok
end

Stale metadata — user status not updating:

# Use update/3 to modify existing presence metadata
def handle_in("update_status", %{"status" => status}, socket) do
  MyAppWeb.Presence.update(socket, socket.assigns.user_id, fn meta ->
    Map.put(meta, :status, status)
  end)

  {:noreply, socket}
end

Presence CRDT memory growth in high-churn topics:

# Presence accumulates tombstones during high join/leave churn.
# Limit topic lifetimes: use short-lived topics for ephemeral sessions
# and garbage-collect topics when subscriber count reaches zero.
def handle_info(:sweep_empty_topics, state) do
  active_topics = Registry.select(MyApp.TopicRegistry, [{{:"$1", :_, :_}, [], [:"$1"]}])
  Enum.each(active_topics, fn topic ->
    if MyAppWeb.Presence.list(topic) == %{} do
      Registry.unregister(MyApp.TopicRegistry, topic)
    end
  end)
  {:noreply, state}
end

Cross-node divergence after partition heal:

This is handled automatically by the CRDT merge, but you can speed up convergence by restarting the Presence process after a partition heal is detected — the CRDT will re-sync from PubSub broadcasts.

What You've Built

| What | How | |------|-----| | End-to-end Presence test | Track probe → verify in list → untrack, with latency | | Structured health endpoint | JSON health response with Presence check and cluster info | | Telemetry instrumentation | Join/leave counters and presence count gauge per topic | | Presence process pool monitor | Periodic count and memory reporting via GenServer | | Leak detector | Age-based scan for entries older than 1 hour | | Liveness heartbeat | GenServer pinging Vigilmon only on successful track/list/untrack cycle | | Cross-node sync check | RPC probe verifying Presence entries replicate to remote nodes | | Real-time alerting | Vigilmon HTTP monitor + PagerDuty on heartbeat miss |

Phoenix.Presence makes building "who's online" features trivial. Vigilmon makes sure that data stays accurate when your application's reliability matters most.

Next Steps

  • Use Vigilmon's downtime history to report Presence availability in SLA dashboards
  • Track Presence join rate over time to forecast capacity for peak user load
  • Set up per-feature Presence monitors (e.g., separate heartbeats for chat vs. collaborative documents)
  • Configure a Grafana alert when user count drops to zero during expected peak hours

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles