How to Monitor ConCache with Vigilmon

ConCache is an ETS-backed key/value store for Elixir with TTL management, row-level locking, and concurrent access patterns. It lives in-process, which makes it fast — but also means it lives and dies with your application node. Cache misses caused by a crashed ConCache process, misconfigured TTL, or eviction pressure can silently degrade your app performance long before users start complaining.

This tutorial covers:

• A health endpoint that validates ConCache is running and responsive
• HTTP uptime monitoring with Vigilmon
• Heartbeat checks that prove cache operations are succeeding
• Slack alerts when cache health degrades

Step 1: Add ConCache to your project

# mix.exs
defp deps do
  [
    {:con_cache, "~> 1.0"},
    {:req, "~> 0.4"}
  ]
end

Start it in your supervision tree:

# lib/my_app/application.ex
def start(_type, _args) do
  children = [
    # Start ConCache before any processes that depend on it
    {ConCache, [
      name: :my_cache,
      ttl_check_interval: :timer.seconds(30),
      global_ttl: :timer.minutes(5),
      touch_on_read: false
    ]},
    MyAppWeb.Endpoint
  ]

  Supervisor.start_link(children, strategy: :one_for_one)
end

Basic usage:

# Write with TTL
ConCache.put(:my_cache, "user:123", %{name: "Alice", role: :admin})

# Write with custom TTL
ConCache.put(:my_cache, "session:abc", token, ttl: :timer.minutes(30))

# Read
case ConCache.get(:my_cache, "user:123") do
  nil -> {:miss, load_from_db("user:123")}
  value -> {:hit, value}
end

# Atomic update
ConCache.update(:my_cache, "counter", fn
  nil -> {:ok, 1}
  n -> {:ok, n + 1}
end)

Step 2: Build a cache health endpoint

Add a plug that probes ConCache directly:

# lib/my_app_web/plugs/health_check.ex
defmodule MyAppWeb.Plugs.HealthCheck do
  import Plug.Conn

  @probe_key "__health_probe__"
  @probe_value "vigilmon_ok"

  def init(opts), do: opts

  def call(%Plug.Conn{request_path: "/health"} = conn, _opts) do
    checks = %{
      status: :pending,
      cache: cache_check(),
      database: db_check()
    }

    all_ok = checks.cache.status == "ok" and checks.database.status == "ok"
    checks = Map.put(checks, :status, if(all_ok, do: "ok", else: "degraded"))
    http_status = if all_ok, do: 200, else: 503

    conn
    |> put_resp_content_type("application/json")
    |> send_resp(http_status, Jason.encode!(checks))
    |> halt()
  end

  def call(conn, _opts), do: conn

  defp cache_check do
    try do
      # Write a probe value
      ConCache.put(:my_cache, @probe_key, @probe_value)

      # Read it back immediately
      case ConCache.get(:my_cache, @probe_key) do
        @probe_value ->
          info = cache_info()
          Map.put(info, :status, "ok")
        other ->
          %{status: "error", reason: "probe mismatch: got #{inspect(other)}"}
      end
    catch
      kind, reason ->
        %{status: "error", reason: "#{kind}: #{inspect(reason)}"}
    end
  end

  defp cache_info do
    # ETS table info for the ConCache backing store
    table = ConCache.ets(:my_cache)
    info = :ets.info(table)
    %{
      size: Keyword.get(info, :size, 0),
      memory_words: Keyword.get(info, :memory, 0)
    }
  rescue
    _ -> %{}
  end

  defp db_check do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", []) do
      {:ok, _} -> %{status: "ok"}
      {:error, r} -> %{status: "error", reason: inspect(r)}
    end
  end
end

Register it in your endpoint:

# lib/my_app_web/endpoint.ex
plug MyAppWeb.Plugs.HealthCheck
plug MyAppWeb.Router

Test it:

curl http://localhost:4000/health | jq .
# {
#   "status": "ok",
#   "cache": {
#     "status": "ok",
#     "size": 142,
#     "memory_words": 8320
#   },
#   "database": {"status": "ok"}
# }

The probe write-and-read-back catches the most important ConCache failure mode: the ETS table exists but writes aren't persisting, which happens when the ConCache process crashes and ETS inheritance doesn't restore state.

Step 3: Monitor with Vigilmon

1. Sign up at vigilmon.online
2. Click New Monitor → HTTP
3. Enter https://yourdomain.com/health
4. Set check interval: 1 minute
5. Add a keyword check: body must contain "status":"ok"
6. Save

The keyword check is critical here because a crashed ConCache process can leave your Phoenix endpoint still responding with 200 to other requests. Without the keyword match, Vigilmon would see a healthy app when the cache is down.

Step 4: Track cache effectiveness with a GenServer probe

Write-and-read health checks validate the cache is alive, but they don't tell you if the cache is actually doing its job — serving frequently accessed data and reducing database load.

Add a GenServer that periodically measures cache hit rates and sends the result as a heartbeat:

# lib/my_app/cache_monitor.ex
defmodule MyApp.CacheMonitor do
  use GenServer

  require Logger

  @interval :timer.minutes(1)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{hits: 0, misses: 0}, name: __MODULE__)

  def record_hit, do: GenServer.cast(__MODULE__, :hit)
  def record_miss, do: GenServer.cast(__MODULE__, :miss)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_cast(:hit, state), do: {:noreply, Map.update!(state, :hits, &(&1 + 1))}
  def handle_cast(:miss, state), do: {:noreply, Map.update!(state, :misses, &(&1 + 1))}

  def handle_info(:report, %{hits: hits, misses: misses} = state) do
    total = hits + misses
    hit_rate = if total > 0, do: Float.round(hits / total * 100, 1), else: 0.0

    Logger.info("Cache hit rate: #{hit_rate}% (#{hits} hits, #{misses} misses)")

    if total > 0 do
      ping_heartbeat()
    end

    schedule()
    {:noreply, %{state | hits: 0, misses: 0}}
  end

  defp ping_heartbeat do
    url = Application.get_env(:my_app, :vigilmon)[:cache_heartbeat_url]
    if url, do: Req.get(url, receive_timeout: 5_000)
  end

  defp schedule, do: Process.send_after(self(), :report, @interval)
end

Instrument your cache calls:

defmodule MyApp.Cache do
  def get(key) do
    case ConCache.get(:my_cache, key) do
      nil ->
        MyApp.CacheMonitor.record_miss()
        nil
      value ->
        MyApp.CacheMonitor.record_hit()
        value
    end
  end

  def put(key, value, opts \\ []) do
    ConCache.put(:my_cache, key, value, opts)
  end
end

In Vigilmon, create a Heartbeat Monitor:

1. Click New Monitor → Heartbeat
2. Set expected interval: 5 minutes (5x your 1-minute reporting interval for buffer)
3. Copy the ping URL
4. Set it as VIGILMON_CACHE_HEARTBEAT_URL in your environment

If the cache stops being used (e.g., all requests are cold misses because an upstream change broke cache population), the heartbeat stops — and Vigilmon alerts you.

Step 5: Alert on cache memory pressure

ConCache uses ETS, which allocates from the Erlang VM's memory pool. Under sustained load, ETS can grow unexpectedly if TTLs are too long or eviction is misfiring.

Extend the health check to flag memory pressure:

defp cache_check do
  try do
    ConCache.put(:my_cache, @probe_key, @probe_value)

    case ConCache.get(:my_cache, @probe_key) do
      @probe_value ->
        table = ConCache.ets(:my_cache)
        info = :ets.info(table)
        size = Keyword.get(info, :size, 0)
        memory_words = Keyword.get(info, :memory, 0)
        # 1 word = 8 bytes on 64-bit; warn above 500MB
        memory_mb = memory_words * 8 / 1_048_576

        status = cond do
          memory_mb > 500 -> "warning"
          true -> "ok"
        end

        %{status: status, size: size, memory_mb: Float.round(memory_mb, 2)}
      _ ->
        %{status: "error", reason: "probe mismatch"}
    end
  catch
    kind, reason ->
      %{status: "error", reason: "#{kind}: #{inspect(reason)}"}
  end
end

Set a Vigilmon keyword alert for "status":"warning" to catch memory pressure before it causes VM-level OOM kills.

Step 6: Slack alerts and status page

Slack alerts:

1. In Vigilmon, go to Notifications → New Channel → Slack
2. Paste your Slack webhook URL
3. Enable it on your HTTP monitor and heartbeat monitor

When ConCache is down:

🔴 DOWN: yourdomain.com/health
Keyword "status":"ok" not found
Detected: EU-West, US-East
3 minutes ago

When the cache heartbeat times out:

🔴 HEARTBEAT MISSED: Cache Liveness Monitor
Expected ping within 5 minutes — none received
Last seen: 8 minutes ago

Status page:

Go to Status Pages → New Status Page, add your monitors, and share the URL with your team.

What you've built

| What | How | |------|-----| | Cache probe | Write + read-back via health endpoint | | ETS memory check | ets:info/1 memory_words with MB threshold | | HTTP uptime monitoring | Vigilmon HTTP monitor + keyword check | | Liveness heartbeat | CacheMonitor GenServer + heartbeat monitor | | Cache hit tracking | Instrumented wrapper with miss/hit counters | | Slack alerts | Vigilmon Slack notification channel | | Status page | Vigilmon public status page |

ConCache keeps your data fast. Vigilmon tells you when it stops.

Next steps

• Expose per-cache-name metrics if you run multiple ConCache instances
• Add response time trending to catch gradual ETS slowdowns before they cause timeouts
• Set up a separate Vigilmon heartbeat per application subsystem that depends on cache (e.g., separate beats for user sessions vs. API rate limits)

Get started free at vigilmon.online.