How to Monitor Bumblebee with Vigilmon

Bumblebee brings Hugging Face's ecosystem of pre-trained transformer models — BERT, GPT-2, Whisper, CLIP, and more — directly into Elixir via Axon and Nx. A single Bumblebee.load_model({:hf, "bert-base-uncased"}) call gives you a production-ready text classifier or embedding model running on BEAM. But model serving is not the same as web serving: models can fail to load from the HF Hub, tokenizers can crash on out-of-vocabulary input, and GPU memory can quietly OOM between requests.

This tutorial wires up complete monitoring for a Bumblebee model serving stack:

• A health endpoint that validates the loaded model and tokenizer
• HTTP uptime monitoring with Vigilmon
• Heartbeat monitoring to detect model reload failures
• Inference latency alerting
• Slack alerts on serving degradation

Why Monitor Bumblebee?

| Signal | What it catches | |---|---| | Model availability | HF Hub download failures, corrupted cache, EXLA compilation errors | | Tokenizer health | Out-of-vocabulary panics, sequence length overflows | | Inference latency | Model size regression, batch size misconfiguration, GPU contention | | Memory utilization | GPU OOM before process crash, leaked model parameters | | Serving pipeline | Nx.Serving queue backlog, timeout on large inputs |

A basic process check won't catch a model that loaded successfully but hangs on every request longer than 512 tokens.

Step 1: Define and Start Your Serving Pipeline

Structure your Bumblebee model as a supervised Nx.Serving in your application:

# lib/my_app/application.ex
defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    children = [
      MyApp.BumblebeeServing,
      # ...
    ]
    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

# lib/my_app/bumblebee_serving.ex
defmodule MyApp.BumblebeeServing do
  use GenServer

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, nil, name: __MODULE__)
  end

  def predict(text) do
    Nx.Serving.batched_run(MyApp.TextClassifier, text)
  end

  def init(_) do
    {:ok, model_info} = Bumblebee.load_model({:hf, "distilbert-base-uncased-finetuned-sst-2-english"})
    {:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "distilbert-base-uncased"})
    {:ok, featurizer} = Bumblebee.load_featurizer({:hf, "distilbert-base-uncased"})

    serving =
      Bumblebee.Text.text_classification(model_info, tokenizer,
        top_k: 1,
        compile: [batch_size: 4, sequence_length: 512],
        defn_options: [compiler: EXLA]
      )

    {:ok, _pid} =
      Nx.Serving.start_link(serving: serving, name: MyApp.TextClassifier, batch_size: 4)

    {:ok, %{model_info: model_info, tokenizer: tokenizer, featurizer: featurizer}}
  end
end

Step 2: Add a Health Endpoint

# lib/my_app_web/plugs/bumblebee_health.ex
defmodule MyAppWeb.Plugs.BumblebeeHealth do
  import Plug.Conn

  def init(opts), do: opts

  def call(%Plug.Conn{request_path: "/health/bumblebee"} = conn, _opts) do
    {status, body} = case run_health_inference() do
      {:ok, result} ->
        {200, %{status: "ok", sample_prediction: result}}
      {:error, reason} ->
        {503, %{status: "error", reason: inspect(reason)}}
    end

    conn
    |> put_resp_content_type("application/json")
    |> send_resp(status, Jason.encode!(body))
    |> halt()
  end

  def call(conn, _opts), do: conn

  defp run_health_inference do
    try do
      # Run a fixed health-check sentence through the classifier
      %{predictions: [%{label: label}]} =
        Nx.Serving.batched_run(MyApp.TextClassifier, "The system is healthy.")
      {:ok, label}
    rescue
      e -> {:error, e}
    catch
      :exit, reason -> {:error, {:exit, reason}}
    end
  end
end

Register in your endpoint:

# lib/my_app_web/endpoint.ex
plug MyAppWeb.Plugs.BumblebeeHealth
plug MyAppWeb.Router

Test it:

curl http://localhost:4000/health/bumblebee
# => {"status":"ok","sample_prediction":"POSITIVE"}

Step 3: Add Vigilmon Uptime Monitoring

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your health endpoint URL: https://ml.yourapp.com/health/bumblebee.
4. Set Check interval to 2 minutes (model inference is slower than a plain HTTP check).
5. Set Expected HTTP status to 200.
6. Set Response timeout to 15 seconds (EXLA-compiled models need time to warm up).
7. Under Advanced, add a Response body contains check: "status":"ok".
8. Click Save.

Step 4: Heartbeat for Model Reload Monitoring

Bumblebee models are sometimes reloaded from the HF Hub or local cache on startup. Add a heartbeat that confirms the model loaded successfully:

# lib/my_app/bumblebee_heartbeat.ex
defmodule MyApp.BumblebeeHeartbeat do
  use GenServer

  @interval :timer.minutes(5)
  @vigilmon_heartbeat_url "https://vigilmon.online/api/heartbeat/YOUR_HEARTBEAT_ID"

  def start_link(_), do: GenServer.start_link(__MODULE__, nil, name: __MODULE__)

  def init(_) do
    schedule()
    {:ok, nil}
  end

  def handle_info(:ping, state) do
    case run_health_check() do
      {:ok, _} ->
        HTTPoison.get!(@vigilmon_heartbeat_url)
      {:error, reason} ->
        require Logger
        Logger.error("Bumblebee health check failed: #{inspect(reason)}")
    end
    schedule()
    {:noreply, state}
  end

  defp schedule, do: Process.send_after(self(), :ping, @interval)

  defp run_health_check do
    try do
      Nx.Serving.batched_run(MyApp.TextClassifier, "health check")
      {:ok, :healthy}
    rescue
      e -> {:error, e}
    end
  end
end

Add to your supervision tree:

children = [
  MyApp.BumblebeeServing,
  MyApp.BumblebeeHeartbeat,
  # ...
]

Create the heartbeat in Vigilmon:

1. Click Add Monitor → Cron / Heartbeat.
2. Set expected interval to 5 minutes.
3. Copy the generated heartbeat URL into @vigilmon_heartbeat_url.

Step 5: Monitor Inference Latency

Track per-request inference time and alert when it degrades:

# lib/my_app_web/controllers/inference_controller.ex
defmodule MyAppWeb.InferenceController do
  use MyAppWeb, :controller

  def classify(conn, %{"text" => text}) do
    start = System.monotonic_time(:millisecond)

    result = Nx.Serving.batched_run(MyApp.TextClassifier, text)
    latency = System.monotonic_time(:millisecond) - start

    :telemetry.execute(
      [:my_app, :bumblebee, :inference],
      %{latency_ms: latency},
      %{model: "distilbert-sst2"}
    )

    json(conn, %{result: result, latency_ms: latency})
  end
end

In Vigilmon, set latency thresholds on your health monitor:

1. Open the monitor settings → Alerting.
2. Set warning at 2000ms, critical at 5000ms.

Step 6: Set Up Alerts

1. Go to Alert Channels → Add Channel.
2. Choose Slack, Email, or PagerDuty.
3. For the HTTP monitor: alert after 2 consecutive failures (avoids false positives on cold JIT compilations).
4. For the heartbeat: alert immediately on first miss.
5. Assign both channels to your Bumblebee monitors.

Key Metrics to Watch

| Metric | Vigilmon feature | What to alert on | |---|---|---| | Model serving availability | HTTP monitor on /health/bumblebee | Any 5xx or timeout | | Model reload liveness | Heartbeat monitor | Missing for > 10 minutes | | Inference latency | Response time chart | P95 > 2s | | SSL certificate | Cert expiry monitor | Expires in < 14 days | | Serving pipeline | HTTP monitor response time | > 15s (EXLA warm-up) |

Conclusion

Bumblebee makes it easy to run transformer models in Elixir, but production model serving needs more than process monitoring. A health endpoint that actually runs inference, a heartbeat that detects serving failures, and latency alerting give you the observability layer that NaN errors and GPU OOM events demand. Wire up these three Vigilmon monitors and catch Bumblebee failures before they reach your users.

Get started free at vigilmon.online.