How to Monitor Phoenix.Channel with Vigilmon

Phoenix.Channel is the real-time bidirectional communication layer in the Phoenix framework. It powers WebSocket-based pub/sub messaging between your server and connected clients — driving chat applications, live dashboards, collaborative editing, multiplayer games, and push notifications. Every push/3, broadcast/3, and reply/2 call flows through the Channel pipeline.

When Channels fail, real-time features degrade silently. Clients see stale data, missed notifications, and dropped messages — but your HTTP uptime monitor still shows green. A Channel crash loop can exhaust process memory on the BEAM VM. A backpressure spike can cause message queues to grow without bound. None of these appear in standard HTTP health checks.

Vigilmon lets you verify that your Channel pipeline is accepting connections and delivering messages end-to-end, not just that the Phoenix endpoint is listening.

Why Monitor Phoenix.Channel?

Channel failures are invisible to traditional uptime monitoring:

• Silent crash loop — a Channel process that raises in handle_in/3 gets restarted by its supervisor, but the client may not re-join; messages intended for that client are lost
• Join rate spike — a surge in WebSocket connections triggers rapid join/leave cycles that saturate the Channel supervisor
• Message queue backlog — a slow handle_in/3 callback builds up a mailbox that grows until the BEAM kills the process
• Authentication drift — Channel token expiry causes joins to fail silently if the client doesn't retry
• Topic leak — dynamic topics keyed by entity ID that are never cleaned up accumulate dead Channel processes
• Heartbeat miss — the Phoenix heartbeat (ping/pong) keeps WebSocket connections alive; a missed heartbeat causes client reconnect storms

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Active Channel processes | Number of connected subscribers per topic | | Join success/failure rate | Authentication failures and capacity issues | | Message throughput (in/out) | Broadcast volume and handler load | | Channel process memory | Mailbox backlog and state accumulation | | WebSocket connection count | Client load on the Endpoint | | Crash/restart rate | Handler errors causing supervision restarts |

Step 1: Verify Channel Setup

A Phoenix Channel requires a Socket, a Channel module, and a router entry:

# lib/my_app_web/channels/user_socket.ex
defmodule MyAppWeb.UserSocket do
  use Phoenix.Socket

  channel "room:*", MyAppWeb.RoomChannel
  channel "notifications:*", MyAppWeb.NotificationChannel

  @impl true
  def connect(%{"token" => token}, socket, _connect_info) do
    case MyApp.Auth.verify_token(token) do
      {:ok, user_id} -> {:ok, assign(socket, :user_id, user_id)}
      {:error, _} -> :error
    end
  end

  @impl true
  def id(socket), do: "user_socket:#{socket.assigns.user_id}"
end

# lib/my_app_web/channels/room_channel.ex
defmodule MyAppWeb.RoomChannel do
  use Phoenix.Channel
  require Logger

  @impl true
  def join("room:" <> room_id, _payload, socket) do
    :telemetry.execute([:my_app, :channel, :join], %{count: 1}, %{topic: "room"})
    {:ok, assign(socket, :room_id, room_id)}
  end

  @impl true
  def handle_in("new_message", payload, socket) do
    start = System.monotonic_time()
    broadcast!(socket, "new_message", payload)
    duration = System.monotonic_time() - start

    :telemetry.execute(
      [:my_app, :channel, :message],
      %{duration: duration},
      %{topic: "room", event: "new_message", direction: :in}
    )

    {:noreply, socket}
  end

  @impl true
  def terminate(reason, socket) do
    :telemetry.execute([:my_app, :channel, :leave], %{count: 1}, %{
      topic: "room",
      reason: inspect(reason)
    })

    :ok
  end
end

Step 2: Instrument Channel Telemetry

Phoenix ships built-in telemetry events for Channels. Attach handlers to capture them:

# lib/my_app/telemetry.ex
defmodule MyApp.Telemetry do
  import Telemetry.Metrics

  def metrics do
    [
      # Phoenix built-in Channel events
      counter("phoenix.channel_joined.count",
        tags: [:endpoint, :transport]
      ),
      counter("phoenix.channel_handled_in.count",
        tags: [:endpoint, :channel, :event]
      ),
      distribution("phoenix.channel_handled_in.duration",
        unit: {:native, :millisecond},
        reporter_options: [buckets: [1, 5, 10, 50, 100, 500]]
      ),

      # Custom application events
      counter("my_app.channel.join.count", tags: [:topic]),
      counter("my_app.channel.leave.count", tags: [:topic, :reason]),
      counter("my_app.channel.message.count", tags: [:topic, :event, :direction]),
      distribution("my_app.channel.message.duration",
        unit: {:native, :millisecond},
        reporter_options: [buckets: [1, 5, 10, 25, 50]]
      )
    ]
  end
end

Step 3: Build a Channel Health Check

Create a self-contained Channel that verifies end-to-end message delivery:

# lib/my_app_web/channels/health_channel.ex
defmodule MyAppWeb.HealthChannel do
  use Phoenix.Channel

  @impl true
  def join("health:probe", _payload, socket) do
    {:ok, socket}
  end

  @impl true
  def handle_in("ping", %{"probe_id" => probe_id}, socket) do
    push(socket, "pong", %{"probe_id" => probe_id})
    {:noreply, socket}
  end
end

# lib/my_app/channel_health.ex
defmodule MyApp.ChannelHealth do
  @moduledoc """
  End-to-end Channel delivery check using an in-process WebSocket client.
  Joins the health channel, sends a ping, and asserts it receives a pong.
  """

  @timeout 3_000
  @endpoint MyAppWeb.Endpoint

  def check do
    {:ok, socket} = Phoenix.ChannelTest.connect(MyAppWeb.UserSocket, %{}, %{})

    {:ok, _, channel} =
      Phoenix.ChannelTest.subscribe_and_join(socket, "health:probe", %{})

    probe_id = :crypto.strong_rand_bytes(4) |> Base.encode16()
    start = System.monotonic_time(:millisecond)

    Phoenix.ChannelTest.push(channel, "ping", %{"probe_id" => probe_id})

    result =
      receive do
        %Phoenix.Socket.Message{event: "pong", payload: %{"probe_id" => ^probe_id}} ->
          latency = System.monotonic_time(:millisecond) - start
          {:ok, latency}
      after
        @timeout -> {:error, :timeout}
      end

    Phoenix.ChannelTest.close(channel)
    result
  rescue
    e -> {:error, inspect(e)}
  end
end

Step 4: Expose a Health Endpoint

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    channel_check = MyApp.ChannelHealth.check()

    checks = %{
      channel: format_check(channel_check),
      database: check_db()
    }

    status = if match?({:ok, _}, channel_check), do: 200, else: 503

    conn
    |> put_status(status)
    |> json(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      checks: checks
    })
  end

  defp format_check({:ok, latency_ms}), do: %{status: "ok", latency_ms: latency_ms}
  defp format_check({:error, reason}), do: %{status: "error", reason: reason}

  defp check_db do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", []) do
      {:ok, _} -> "ok"
      _ -> "error"
    end
  rescue
    _ -> "error"
  end
end

Step 5: Track Channel Process Health

Monitor active Channel processes and their memory consumption:

# lib/my_app/channel_poller.ex
defmodule MyApp.ChannelPoller do
  use GenServer
  require Logger

  @poll_interval :timer.seconds(30)
  @max_channel_memory_mb 50

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:poll, state) do
    report_channel_metrics()
    schedule()
    {:noreply, state}
  end

  defp report_channel_metrics do
    # Count processes registered under the Channel supervisor
    channel_processes =
      Process.list()
      |> Enum.filter(&channel_process?/1)

    count = length(channel_processes)

    total_memory =
      channel_processes
      |> Enum.map(&(Process.info(&1, :memory) |> elem(1)))
      |> Enum.sum()

    :telemetry.execute(
      [:my_app, :channel, :pool],
      %{count: count, total_memory_bytes: total_memory},
      %{}
    )

    memory_mb = div(total_memory, 1024 * 1024)

    if memory_mb > @max_channel_memory_mb do
      Logger.warning("Channel pool memory high: #{memory_mb}MB across #{count} processes")
    end
  rescue
    e -> Logger.warning("ChannelPoller error: #{inspect(e)}")
  end

  defp channel_process?(pid) do
    case Process.info(pid, :dictionary) do
      {:dictionary, dict} -> Keyword.get(dict, :"$initial_call") == {Phoenix.Channel.Server, :init, 1}
      _ -> false
    end
  rescue
    _ -> false
  end

  defp schedule, do: Process.send_after(self(), :poll, @poll_interval)
end

Step 6: Create Monitors in Vigilmon

HTTP monitor for the health endpoint:

1. Sign in at vigilmon.online
2. Click New Monitor → HTTP
3. Set URL to https://your-app.example.com/health
4. Set interval to 60 seconds
5. Alert condition: status 200 and body contains "channel":{"status":"ok"}

Heartbeat monitor for Channel liveness:

# lib/my_app/channel_heartbeat.ex
defmodule MyApp.ChannelHeartbeat do
  use GenServer

  @heartbeat_url System.get_env("VIGILMON_CHANNEL_HEARTBEAT_URL")
  @interval :timer.minutes(1)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:check, state) do
    case MyApp.ChannelHealth.check() do
      {:ok, latency_ms} ->
        ping_vigilmon()
        schedule()
        {:noreply, Map.put(state, :last_latency_ms, latency_ms)}

      {:error, reason} ->
        require Logger
        Logger.error("Channel health check failed: #{inspect(reason)}")
        schedule()
        {:noreply, state}
    end
  end

  defp ping_vigilmon do
    if @heartbeat_url do
      :httpc.request(:get, {String.to_charlist(@heartbeat_url), []}, [{:timeout, 3000}], [])
    end
  end

  defp schedule, do: Process.send_after(self(), :check, @interval)
end

Add to your supervision tree:

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  MyAppWeb.Endpoint,
  MyApp.ChannelPoller,
  MyApp.ChannelHeartbeat
]

Configure the heartbeat monitor in Vigilmon with a 3-minute expected interval. A missed heartbeat means Channel delivery is broken.

Step 7: Alerting

In Vigilmon, configure Notifications → New Channel:

Slack for Channel delivery failure:

🔴 DOWN: Phoenix.Channel — MyApp
Monitor: /health returning channel.status = "error"
Action: Check Channel supervisor, Socket authentication, and WebSocket port (4000)

PagerDuty for heartbeat miss:

A missed heartbeat means WebSocket message delivery is broken for all connected clients — treat this as an incident. Configure a high-priority PagerDuty integration in Vigilmon's notification settings.

Alert on Channel process memory:

If you export telemetry to Prometheus via PromEx or a compatible exporter, add an alert:

- alert: PhoenixChannelMemoryHigh
  expr: my_app_channel_pool_total_memory_bytes > 52428800  # 50 MB
  for: 5m
  annotations:
    summary: "Channel pool memory exceeds 50MB — possible process leak"

Common Phoenix.Channel Issues and Fixes

Message queue buildup — slow handle_in callbacks:

# Move slow work to a Task so the Channel stays responsive
@impl true
def handle_in("heavy_compute", payload, socket) do
  Task.start(fn ->
    result = MyApp.HeavyWork.run(payload)
    Phoenix.Channel.push(socket, "result", result)
  end)

  {:noreply, socket}
end

Topic leak — Channel processes not terminating:

# Always implement terminate/2 to clean up external state
@impl true
def terminate(_reason, socket) do
  MyApp.Presence.untrack(socket)
  :ok
end

Authentication token expiry causing join failures:

# Return an error tuple from join/3 so the client knows to refresh
@impl true
def join("room:" <> room_id, %{"token" => token}, socket) do
  case MyApp.Auth.verify_token(token) do
    {:ok, user_id} -> {:ok, assign(socket, :user_id, user_id)}
    {:error, :expired} -> {:error, %{reason: "token_expired"}}
    {:error, _} -> {:error, %{reason: "unauthorized"}}
  end
end

Broadcast to a topic with no subscribers — silent no-op:

Use Phoenix.Channel.broadcast/3 for verified delivery to connected sockets; log when subscriber count is unexpectedly zero on high-traffic topics.

What You've Built

| What | How | |------|-----| | End-to-end Channel test | In-process WebSocket client: join → push ping → receive pong | | Structured health endpoint | JSON health response with Channel and db checks | | Telemetry instrumentation | Phoenix built-in events + custom join/leave/message counters | | Channel process pool monitor | Process count and memory tracked via GenServer poller | | Liveness heartbeat | GenServer pinging Vigilmon only on successful delivery | | Real-time alerting | Vigilmon HTTP monitor + PagerDuty on heartbeat miss |

Phoenix.Channel makes real-time communication possible. Vigilmon makes sure it keeps working when your users depend on it most.

Next Steps

• Add Phoenix.Presence monitoring to track connected user counts as a business metric
• Set up per-topic heartbeats for your highest-traffic channels
• Use Vigilmon's status history to report WebSocket availability in SLA dashboards
• Configure Grafana panels for Channel join rate and message throughput per topic

Get started free at vigilmon.online.