tutorial

How to Monitor Nebulex with Vigilmon

Monitor Nebulex cache hit rates, adapter health, multi-level cache consistency, and eviction pressure across ETS, Redis, and Memcached backends with Vigilmon.

How to Monitor Nebulex with Vigilmon

Nebulex is a flexible, composable caching framework for Elixir that unifies local caching (ETS), distributed caching (Redis, Memcached), and multi-level hierarchical caches under a single API. Rather than hard-coding a caching backend, you define adapters — and swap or compose them without changing application code.

This flexibility is its strength and its observability challenge. A Nebulex cache can fail silently: the ETS layer is fine while the Redis adapter is refusing connections, the multi-level cache is serving stale data from L1 while L2 is down, or the distributed cache is partitioned and two nodes are serving divergent values. Your application code returns values; it just returns wrong or stale ones.

Vigilmon lets you verify from outside that cache reads are returning fresh data and adapter backends are reachable, not just that the Elixir process tree is alive.

Why Monitor Nebulex?

| Failure mode | Why it's silent | |---|---| | Redis adapter disconnection | L1 ETS continues serving stale data; no error is raised | | L2 miss spike | Every request falls through to the database; latency increases gradually | | Eviction under memory pressure | Keys disappear; application recomputes on every call | | Multi-level inconsistency | L1 has stale value after L2 write by another node | | Adapter config error | Wrong TTL or serializer returns unexpected types |

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Cache hit rate | Low hit rate = keys not being written or evicting early | | L1 vs L2 hit ratio | Imbalance indicates multi-level cache configuration issue | | Eviction count | Rising evictions = memory pressure on ETS or backend | | Adapter round-trip latency | Slow L2 backend degrading overall cache response time | | Miss-to-database ratio | How often misses trigger expensive database queries | | TTL distribution | Unexpected TTL values causing premature expiry |

Step 1: Add Nebulex to Your Application

# mix.exs
defp deps do
  [
    {:nebulex, "~> 2.6"},
    # Optional: Redis adapter
    {:nebulex_redis_adapter, "~> 2.4"},
    # Optional: stats support
    {:shards, "~> 1.1"}
  ]
end

Define your cache module:

# lib/my_app/cache.ex
defmodule MyApp.Cache do
  use Nebulex.Cache,
    otp_app: :my_app,
    adapter: Nebulex.Adapters.Multilevel

  defmodule L1 do
    use Nebulex.Cache,
      otp_app: :my_app,
      adapter: Nebulex.Adapters.Local
  end

  defmodule L2 do
    use Nebulex.Cache,
      otp_app: :my_app,
      adapter: NebulexRedisAdapter
  end
end

# config/config.exs
config :my_app, MyApp.Cache,
  model: :inclusive,
  levels: [
    {MyApp.Cache.L1, gc_interval: :timer.hours(1), backend: :shards, partitions: 2},
    {MyApp.Cache.L2, []}
  ]

config :my_app, MyApp.Cache.L1,
  stats: true

config :my_app, MyApp.Cache.L2,
  conn_opts: [
    host: System.get_env("REDIS_HOST", "localhost"),
    port: 6379
  ]

# lib/my_app/application.ex
children = [
  MyApp.Cache,
  # ...
]

Step 2: Enable Stats and Build a Health Check

# lib/my_app/nebulex_health.ex
defmodule MyApp.NebulexHealth do
  require Logger

  @probe_key "vigilmon:health:probe"
  @probe_value "ok"

  def check do
    with {:write, :ok} <- {:write, write_probe()},
         {:read, {:ok, @probe_value}} <- {:read, read_probe()},
         {:l1_stats, l1_stats} <- {:l1_stats, fetch_l1_stats()},
         {:delete, :ok} <- {:delete, delete_probe()} do
      %{
        status: :ok,
        write_ok: true,
        read_ok: true,
        l1_stats: l1_stats
      }
    else
      {:read, {:ok, unexpected}} ->
        %{status: :error, reason: "probe read returned unexpected value: #{inspect(unexpected)}"}

      {step, {:error, reason}} ->
        Logger.warning("NebulexHealth failed at #{step}: #{inspect(reason)}")
        %{status: :error, failed_step: step, reason: inspect(reason)}

      {step, other} ->
        %{status: :error, failed_step: step, reason: inspect(other)}
    end
  end

  defp write_probe do
    case MyApp.Cache.put(@probe_key, @probe_value, ttl: :timer.seconds(30)) do
      :ok -> :ok
      error -> {:error, error}
    end
  end

  defp read_probe do
    case MyApp.Cache.get(@probe_key) do
      nil -> {:error, :key_not_found}
      value -> {:ok, value}
    end
  end

  defp delete_probe do
    MyApp.Cache.delete(@probe_key)
    :ok
  end

  defp fetch_l1_stats do
    try do
      stats = MyApp.Cache.L1.stats()
      %{
        hits: get_in(stats, [:measurements, :hits]) || 0,
        misses: get_in(stats, [:measurements, :misses]) || 0,
        writes: get_in(stats, [:measurements, :writes]) || 0,
        evictions: get_in(stats, [:measurements, :evictions]) || 0
      }
    rescue
      _ -> %{hits: nil, misses: nil, writes: nil, evictions: nil}
    end
  end
end

Step 3: Expose a Health Endpoint

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    health = MyApp.NebulexHealth.check()
    hit_rate = compute_hit_rate(health)

    status = if health.status == :ok, do: 200, else: 503

    conn
    |> put_status(status)
    |> json(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      cache: health,
      hit_rate_pct: hit_rate
    })
  end

  defp compute_hit_rate(%{l1_stats: %{hits: hits, misses: misses}})
       when is_integer(hits) and is_integer(misses) and hits + misses > 0 do
    Float.round(hits / (hits + misses) * 100, 1)
  end

  defp compute_hit_rate(_), do: nil
end

# lib/my_app_web/router.ex
scope "/", MyAppWeb do
  get "/health", HealthController, :index
end

Step 4: Emit Cache Telemetry

# lib/my_app/nebulex_poller.ex
defmodule MyApp.NebulexPoller do
  use GenServer
  require Logger

  @interval :timer.seconds(30)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:poll, state) do
    report_metrics()
    schedule()
    {:noreply, state}
  end

  defp report_metrics do
    try do
      stats = MyApp.Cache.L1.stats()
      measurements = stats[:measurements] || %{}

      hits = Map.get(measurements, :hits, 0)
      misses = Map.get(measurements, :misses, 0)
      evictions = Map.get(measurements, :evictions, 0)
      writes = Map.get(measurements, :writes, 0)

      hit_rate =
        if hits + misses > 0, do: hits / (hits + misses) * 100, else: 0.0

      :telemetry.execute(
        [:my_app, :cache, :stats],
        %{
          hits: hits,
          misses: misses,
          evictions: evictions,
          writes: writes,
          hit_rate: Float.round(hit_rate, 2)
        },
        %{level: :l1}
      )
    rescue
      e -> Logger.warning("NebulexPoller: stats error: #{inspect(e)}")
    end
  end

  defp schedule, do: Process.send_after(self(), :poll, @interval)
end

Add it to your supervision tree:

children = [
  MyApp.Cache,
  MyApp.NebulexPoller,
  # ...
]

Step 5: Create Monitors in Vigilmon

HTTP monitor for the health endpoint:

  1. Sign in at vigilmon.online
  2. Click New Monitor → HTTP
  3. URL: https://your-app.example.com/health
  4. Interval: 60 seconds
  5. Alert condition: non-200 or body contains "status":"degraded"

Heartbeat monitor for cache liveness:

# lib/my_app/nebulex_heartbeat.ex
defmodule MyApp.NebulexHeartbeat do
  use GenServer

  @interval :timer.minutes(2)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:ping, state) do
    health = MyApp.NebulexHealth.check()

    if health.status == :ok do
      url = System.get_env("VIGILMON_NEBULEX_HEARTBEAT_URL")

      if url do
        :httpc.request(:get, {String.to_charlist(url), []}, [{:timeout, 5_000}], [])
      end
    end

    schedule()
    {:noreply, state}
  end

  defp schedule, do: Process.send_after(self(), :ping, @interval)
end

In Vigilmon, create a Heartbeat monitor with a 5-minute expected interval. Set VIGILMON_NEBULEX_HEARTBEAT_URL to the heartbeat URL from the Vigilmon dashboard.

Step 6: Alerting

In Vigilmon, go to Notifications → New Channel and configure:

Slack for cache degradation:

🔴 DOWN: Nebulex Cache — MyApp
Monitor: /health returning cache.status = "degraded"
Step failed: {failed_step}
Action: Check Redis adapter connection, L1 ETS process, multi-level config

Slack for low hit rate alert (configured via health check threshold):

⚠️ WARN: Nebulex Cache Hit Rate Low — MyApp
Hit rate: {hit_rate_pct}% (threshold: 70%)
Action: Check for key eviction, TTL misconfiguration, or cache warm-up needed

PagerDuty for heartbeat miss — the cache layer is completely unreachable; database will absorb full load.

Common Nebulex Issues and Fixes

L2 Redis adapter unavailable while L1 serves stale data:

# Add an L2 health check to your health module
defp check_l2_reachable do
  case MyApp.Cache.L2.get("__l2_probe__") do
    nil ->
      MyApp.Cache.L2.put("__l2_probe__", "ok", ttl: :timer.seconds(60))
      {:ok, :reachable}

    _value ->
      {:ok, :reachable}
  end
rescue
  e -> {:error, inspect(e)}
end

Low hit rate after deployment:

# Warm the cache after deployment using a startup task
defmodule MyApp.CacheWarmer do
  def warm do
    critical_keys = MyApp.CriticalData.list_keys()
    Enum.each(critical_keys, fn key ->
      value = MyApp.CriticalData.fetch(key)
      MyApp.Cache.put(key, value, ttl: :timer.hours(1))
    end)
  end
end

Multi-level inconsistency after node partition:

# Force L1 invalidation on write to ensure consistency
def put_through(key, value, opts \\ []) do
  MyApp.Cache.put(key, value, opts)
  # Explicitly invalidate L1 on other nodes via your pub/sub
  Phoenix.PubSub.broadcast(MyApp.PubSub, "cache:invalidate", {:invalidate, key})
end

What You've Built

| What | How | |------|-----| | Cache write/read health | Probe key written and read back through full multi-level stack | | L1 stats collection | Hits, misses, evictions, writes from Nebulex stats API | | Hit rate computation | Derived metric exposed in health endpoint and telemetry | | Structured health endpoint | JSON response with cache status and hit rate | | Liveness heartbeat | GenServer pinging Vigilmon only when cache is healthy | | Real-time alerting | HTTP monitor + PagerDuty on heartbeat miss |

Nebulex gives you a unified caching API across adapters. Vigilmon gives you unified visibility into whether any of those adapters is silently failing.

Next Steps

  • Add per-adapter latency tracking so you can distinguish L1 from L2 slowdowns
  • Alert on eviction rate spike as a leading indicator of memory pressure before keys disappear
  • Use Vigilmon response time history to track cache health endpoint latency trends
  • Set up a status page that combines cache health with your database and Redis monitors

Get started free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles