How to Monitor Carbonite with Vigilmon

Carbonite writes every database change to append-only audit tables with timestamps and actor metadata — giving you a full compliance trail and a debugging timeline. But that write path is also a hidden dependency: if Carbonite's triggers fail, audit records silently disappear. If the audit table grows unbounded, Postgres performance degrades. If the actor context isn't being set, every row shows a blank author.

This tutorial adds monitoring for all three failure modes:

• HTTP health endpoint that validates Carbonite is capturing changes
• Heartbeat that confirms audit table growth is normal
• Alerting when the actor context is missing or the audit stream falls behind
• Vigilmon uptime and heartbeat monitors

Why Monitor Carbonite?

| Signal | What it catches | |---|---| | Trigger health | Dropped triggers after schema migrations | | Audit table growth | Runaway inserts causing Postgres bloat | | Actor context | Missing current_user metadata in change records | | Write latency | Trigger overhead slowing down OLTP operations | | Replication lag | Audit changes not propagating to read replicas |

Carbonite is invisible when working correctly. Monitoring surfaces the moment it stops being invisible.

Step 1: Add a Health Check Module

Create a module that writes a test record and confirms Carbonite captured it:

# lib/my_app/carbonite_health.ex
defmodule MyApp.CarboniteHealth do
  alias MyApp.Repo
  import Ecto.Query

  @test_table "carbonite_default.transactions"

  def check do
    # Count recent Carbonite transactions (last 5 minutes)
    since = DateTime.utc_now() |> DateTime.add(-300, :second)

    count =
      from(t in @test_table,
        where: t.inserted_at >= ^since,
        select: count(t.id)
      )
      |> Repo.one()

    case count do
      n when is_integer(n) -> {:ok, %{recent_transactions: n}}
      _ -> {:error, :query_failed}
    end
  rescue
    e -> {:error, Exception.message(e)}
  end
end

Step 2: Expose a Health Endpoint

Add a controller action or plug that Vigilmon can poll:

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def carbonite(conn, _params) do
    case MyApp.CarboniteHealth.check() do
      {:ok, info} ->
        json(conn, Map.put(info, :status, "ok"))

      {:error, reason} ->
        conn
        |> put_status(503)
        |> json(%{status: "error", reason: reason})
    end
  end
end

Register the route:

# lib/my_app_web/router.ex
scope "/health" do
  get "/carbonite", HealthController, :carbonite
end

Test it:

curl http://localhost:4000/health/carbonite
# => {"status":"ok","recent_transactions":42}

Step 3: Add Vigilmon Uptime Monitoring

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your health URL: https://yourapp.com/health/carbonite.
4. Set Check interval to 2 minutes.
5. Set Expected HTTP status to 200.
6. Under Advanced, add a Response body contains check: "status":"ok".
7. Click Save.

Vigilmon will alert you if the Carbonite health endpoint returns 5xx or becomes unreachable.

Step 4: Monitor Audit Table Growth with a Heartbeat

Audit tables should grow — but steadily. Use a GenServer heartbeat that pings Vigilmon only while growth is within normal bounds:

# lib/my_app/carbonite_heartbeat.ex
defmodule MyApp.CarboniteHeartbeat do
  use GenServer
  alias MyApp.Repo
  import Ecto.Query

  @interval :timer.minutes(5)
  @heartbeat_url "https://vigilmon.online/api/heartbeat/YOUR_HEARTBEAT_ID"
  # Alert if more than 10 000 rows written in 5 minutes (tune to your traffic)
  @max_rows_per_interval 10_000

  def start_link(_), do: GenServer.start_link(__MODULE__, nil, name: __MODULE__)

  def init(_) do
    schedule()
    {:ok, nil}
  end

  def handle_info(:check, state) do
    if audit_growth_normal?() do
      HTTPoison.get!(@heartbeat_url)
    end

    schedule()
    {:noreply, state}
  end

  defp audit_growth_normal? do
    since = DateTime.utc_now() |> DateTime.add(-300, :second)

    count =
      from(t in "carbonite_default.transactions",
        where: t.inserted_at >= ^since,
        select: count(t.id)
      )
      |> Repo.one()

    is_integer(count) and count < @max_rows_per_interval
  end

  defp schedule, do: Process.send_after(self(), :check, @interval)
end

Add it to your supervision tree in application.ex:

children = [
  MyApp.Repo,
  MyAppWeb.Endpoint,
  MyApp.CarboniteHeartbeat
]

To create the Vigilmon heartbeat monitor:

1. Click Add Monitor → Cron / Heartbeat.
2. Set the expected interval to 5 minutes.
3. Copy the URL into @heartbeat_url above.

Step 5: Detect Missing Actor Context

Carbonite records meta alongside each transaction. If your application isn't setting actor context, those fields are blank — compliance reports become useless. Add a periodic check:

# lib/my_app/carbonite_actor_check.ex
defmodule MyApp.CarboniteActorCheck do
  alias MyApp.Repo
  import Ecto.Query

  def run do
    since = DateTime.utc_now() |> DateTime.add(-3600, :second)

    blank_actor_count =
      from(t in "carbonite_default.transactions",
        where: t.inserted_at >= ^since,
        where: is_nil(fragment("meta->>'actor_id'")),
        select: count(t.id)
      )
      |> Repo.one()

    if blank_actor_count > 0 do
      :logger.warning("Carbonite: #{blank_actor_count} transactions missing actor_id in last hour")
    end

    blank_actor_count
  end
end

Call this from a scheduled task or Oban job and tie the result to a Vigilmon heartbeat that stays alive only when blank_actor_count == 0.

Step 6: Set Up Alerts

In Vigilmon, configure alert channels for all three monitors:

1. Go to Alert Channels → Add Channel.
2. Add Slack and/or Email.
3. Set Notify after to 2 consecutive failures to avoid flapping.
4. Assign the channel to:
   • The HTTP monitor on /health/carbonite
   • The audit-growth heartbeat
   • Any actor-context heartbeat

For compliance-critical deployments, also enable the Status Page feature so your internal audit team can self-serve on Carbonite availability.

Key Metrics to Watch

| Metric | Vigilmon feature | Alert threshold | |---|---|---| | Audit endpoint availability | HTTP monitor | Any 5xx or timeout | | Audit table growth | Heartbeat | Missing for > 6 minutes | | Actor context coverage | Heartbeat | Missing for > 1 heartbeat | | SSL certificate | Cert expiry monitor | Expires in < 14 days | | Response time | Response time chart | P95 > 1s |

Conclusion

Carbonite's append-only model is its strength for compliance — and its risk if monitoring is absent. With Vigilmon watching the audit health endpoint, tracking growth via heartbeat, and alerting on missing actor context, you'll catch trigger failures and runaway writes long before they become audit gaps or Postgres performance issues.

Get started free at vigilmon.online.