How to Monitor EventStore (Elixir) with Vigilmon

The Elixir EventStore library (often called eventstore or commanded/eventstore) persists event streams to PostgreSQL, provides catch-up subscriptions for projections, and supports snapshotting for aggregate reconstruction. In a CQRS/ES system, the event store is the source of truth — which means any failure here is a total failure: projections stop updating, aggregates can't be reconstructed, and commands start bouncing.

This tutorial adds monitoring for all the key failure modes:

• HTTP health endpoint that confirms EventStore is accepting appends
• Heartbeat tracking that subscriptions are processing events
• Snapshot staleness check for long-running aggregates
• Vigilmon uptime and heartbeat monitors

Why Monitor EventStore?

| Signal | What it catches | |---|---| | Append availability | Postgres connection failures, schema issues | | Subscription lag | Projections falling behind the event stream | | Snapshot staleness | Aggregates requiring full stream replay on every command | | Stream growth | Unbounded streams degrading append performance | | Process health | EventStore supervisor or subscription GenServer crashes |

EventStore failures are often silent at first — commands succeed but projections never update. Monitoring surfaces the lag before users notice stale read models.

Step 1: Configure EventStore

Ensure EventStore is initialized in your application:

# config/config.exs
config :my_app, MyApp.EventStore,
  serializer: EventStore.JsonSerializer,
  username: "postgres",
  password: "postgres",
  database: "my_app_eventstore",
  hostname: "localhost"

# lib/my_app/event_store.ex
defmodule MyApp.EventStore do
  use EventStore, otp_app: :my_app
end

Step 2: Add a Health Check

Create a module that appends a test event to a health stream and reads it back:

# lib/my_app/event_store_health.ex
defmodule MyApp.EventStoreHealth do
  alias MyApp.EventStore

  @health_stream "health-check-stream"

  def check do
    event = %EventStore.EventData{
      event_type: "HealthCheck",
      data: %{timestamp: DateTime.to_iso8601(DateTime.utc_now())},
      metadata: %{}
    }

    with :ok <- EventStore.append_to_stream(@health_stream, :any_version, [event]),
         {:ok, events} <- EventStore.read_stream_forward(@health_stream, 0, 1) do
      {:ok, %{stream: @health_stream, events_readable: length(events) > 0}}
    else
      {:error, reason} -> {:error, inspect(reason)}
    end
  rescue
    e -> {:error, Exception.message(e)}
  end
end

Note: For high-traffic systems, use a dedicated low-volume health stream rather than writing to your domain streams.

Step 3: Expose a Health Endpoint

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def event_store(conn, _params) do
    case MyApp.EventStoreHealth.check() do
      {:ok, info} ->
        json(conn, Map.put(info, :status, "ok"))

      {:error, reason} ->
        conn
        |> put_status(503)
        |> json(%{status: "error", reason: reason})
    end
  end
end

Add the route:

scope "/health" do
  get "/event-store", HealthController, :event_store
end

Test it:

curl http://localhost:4000/health/event-store
# => {"status":"ok","stream":"health-check-stream","events_readable":true}

Step 4: Add Vigilmon Uptime Monitoring

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your URL: https://yourapp.com/health/event-store.
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Under Advanced, add a Response body contains check: "status":"ok".
7. Click Save.

Vigilmon will alert you immediately if EventStore becomes unable to append or read events.

Step 5: Monitor Subscription Lag with a Heartbeat

Catch-up subscriptions drive your projections. If a subscription falls behind, read models go stale. Monitor the lag with a heartbeat:

# lib/my_app/event_store_subscription_monitor.ex
defmodule MyApp.EventStoreSubscriptionMonitor do
  use GenServer
  alias MyApp.EventStore

  @interval :timer.seconds(30)
  @heartbeat_url "https://vigilmon.online/api/heartbeat/YOUR_HEARTBEAT_ID"
  # Alert if subscription is more than 500 events behind
  @max_lag 500

  def start_link(_), do: GenServer.start_link(__MODULE__, nil, name: __MODULE__)

  def init(_) do
    schedule()
    {:ok, nil}
  end

  def handle_info(:check, state) do
    if subscription_healthy?() do
      HTTPoison.get!(@heartbeat_url)
    end

    schedule()
    {:noreply, state}
  end

  defp subscription_healthy? do
    case EventStore.stream_info("$all") do
      {:ok, %{stream_version: latest}} ->
        # Compare to your projection's last processed event number
        last_processed = MyApp.ProjectionTracker.last_event_number()
        lag = latest - last_processed
        lag <= @max_lag

      _ ->
        false
    end
  end

  defp schedule, do: Process.send_after(self(), :check, @interval)
end

Implement MyApp.ProjectionTracker.last_event_number/0 to read the watermark from your projection state store (ETS, database, or a GenServer). Add the monitor to your supervision tree:

children = [
  MyApp.EventStore,
  MyApp.EventStoreSubscriptionMonitor
]

Create the Vigilmon heartbeat:

1. Click Add Monitor → Cron / Heartbeat.
2. Set expected interval to 1 minute.
3. Paste the URL into @heartbeat_url.

Step 6: Check Snapshot Staleness

Without snapshots, aggregates with long event histories require a full stream replay on every command. Add a staleness check:

# lib/my_app/snapshot_check.ex
defmodule MyApp.SnapshotCheck do
  alias MyApp.EventStore

  # Alert if an aggregate hasn't been snapshotted in 10 000 events
  @snapshot_threshold 10_000

  def stale_aggregates do
    # List aggregate IDs you track with snapshots
    aggregate_ids = MyApp.AggregateRegistry.all_ids()

    Enum.filter(aggregate_ids, fn id ->
      stream_length = get_stream_length(id)
      snapshot_version = get_snapshot_version(id)
      events_since_snapshot = stream_length - (snapshot_version || 0)
      events_since_snapshot > @snapshot_threshold
    end)
  end

  defp get_stream_length(aggregate_id) do
    case MyApp.EventStore.stream_info("#{aggregate_id}") do
      {:ok, %{stream_version: v}} -> v
      _ -> 0
    end
  end

  defp get_snapshot_version(aggregate_id) do
    case MyApp.EventStore.read_snapshot(aggregate_id) do
      {:ok, %{source_version: v}} -> v
      _ -> nil
    end
  end
end

Run this periodically via an Oban job and wire a heartbeat to go silent when stale aggregates exist.

Step 7: Set Up Alerts

In Vigilmon:

1. Go to Alert Channels → Add Channel.
2. Configure Slack, email, or PagerDuty.
3. Set Notify after to 2 consecutive failures for the HTTP monitor; 1 failure for the subscription lag heartbeat (lag is serious).
4. Assign to:
   • The HTTP monitor on /health/event-store
   • The subscription lag heartbeat
   • The snapshot staleness heartbeat (if created)

For production CQRS systems, create a Status Page in Vigilmon listing all EventStore monitors so your on-call team has a single pane of glass.

Key Metrics to Watch

| Metric | Vigilmon feature | Alert threshold | |---|---|---| | EventStore availability | HTTP monitor | Any 5xx or timeout | | Subscription lag | Heartbeat | Missing for > 90 seconds | | Snapshot staleness | Heartbeat | Missing for > 1 heartbeat | | SSL certificate | Cert expiry monitor | Expires in < 14 days | | Append response time | Response time chart | P95 > 200ms |

Conclusion

EventStore is the backbone of a CQRS/ES system — when it fails, every downstream projection and aggregate goes wrong simultaneously. Vigilmon's HTTP monitor catches database-level failures instantly, the subscription lag heartbeat detects stalled projections before users see stale data, and the snapshot check prevents accidental full-stream replays from degrading performance. Together they give your event-sourced system the operational visibility it deserves.

Get started free at vigilmon.online.