How to Monitor Witchcraft with Vigilmon

Witchcraft is a Haskell-inspired functional programming library for Elixir that brings typeclasses — Functor, Applicative, Monad, Foldable, Traversable, and more — into the Elixir ecosystem. It provides algebraic structures and composable abstractions that let you write purely functional code with the same rigor as Haskell while staying in the Elixir runtime. Where Elixir's standard library gives you Enum.map/2, Witchcraft gives you a unified map/2 that works over any structure implementing the Functor typeclass — lists, maybes, continuations, or your own custom types.

Code written with algebraic abstractions can be harder to observe than imperative code. A monadic chain of computations composes cleanly in the source, but when something goes wrong — a traversal that degrades under large inputs, a monad bind that short-circuits more often than expected — the problem is invisible without instrumentation. Functional purity does not imply performance or reliability.

Vigilmon gives you the observability layer: measure composition chain throughput, track algebraic operation latency, and alert when your functional pipelines behave unexpectedly.

Why Monitor Witchcraft-Based Code?

Functional abstractions in production present unique observability challenges:

• Composition chain opacity — a monadic pipeline of 6 bind operations looks like a single expression; without instrumentation you don't know which bind is slow or failing
• Lazy evaluation surprises — Witchcraft structures that defer computation may accumulate work invisibly until a final fold forces evaluation, causing unpredictable latency spikes
• Traversal performance degradation — traverse/2 operations that sequence effects over large collections can take exponentially longer than expected as collection size grows
• Typeclass dispatch overhead — Witchcraft uses protocol dispatch for typeclass operations; measuring this overhead ensures your abstraction layer isn't dominating CPU budget in hot paths
• Algebraic law violation risk — custom typeclass implementations that break functor or monad laws (identity, composition, associativity) produce incorrect results silently; catching this requires monitoring actual output shapes

Key Metrics to Track

| Metric | What it tells you | |--------|-------------------| | Operation count by typeclass and operation | Which functors, monads, and traversals are called most often | | Operation duration distribution | Latency profile of map, bind, traverse, fold operations | | Traversal input size vs. duration | Whether traversal performance scales as expected | | Short-circuit rate in monadic chains | How often monad binds return failure values that halt the chain | | Memory allocation per composition chain | Whether algebraic structures accumulate excessive intermediate data | | Error rate in applicative validation | What fraction of applicative-validated inputs fail |

Step 1: Add Witchcraft to Your Project

# mix.exs
defp deps do
  [
    {:witchcraft, "~> 1.0"},
    {:algae, "~> 1.2"}  # Algebraic data types that implement Witchcraft typeclasses
  ]
end

Basic Functor and Monad usage:

# lib/my_app/data_pipeline.ex
defmodule MyApp.DataPipeline do
  use Witchcraft

  alias Algae.Maybe
  alias Algae.Either

  # Functor: transform values inside a structure
  def transform_users(users) do
    users
    |> map(fn user -> Map.update!(user, :email, &String.downcase/1) end)
    |> map(fn user -> Map.put(user, :processed_at, DateTime.utc_now()) end)
  end

  # Monad: sequence computations that may fail
  def process_order(order_id) do
    monad %Maybe{} do
      order <- fetch_order(order_id)
      customer <- fetch_customer(order.customer_id)
      inventory <- check_inventory(order.product_id, order.quantity)
      return(complete_order(order, customer, inventory))
    end
  end

  defp fetch_order(id) do
    case MyApp.Repo.get(MyApp.Order, id) do
      nil -> %Maybe.Nothing{}
      order -> %Maybe.Just{just: order}
    end
  end

  defp fetch_customer(id) do
    case MyApp.Repo.get(MyApp.Customer, id) do
      nil -> %Maybe.Nothing{}
      customer -> %Maybe.Just{just: customer}
    end
  end

  defp check_inventory(product_id, qty) do
    if MyApp.Inventory.available?(product_id, qty) do
      %Maybe.Just{just: :available}
    else
      %Maybe.Nothing{}
    end
  end

  defp complete_order(order, customer, _inventory) do
    %{order: order, customer: customer, status: :complete}
  end
end

Step 2: Instrument Typeclass Operations

Wrap Witchcraft operations to emit telemetry:

# lib/my_app/witchcraft_telemetry.ex
defmodule MyApp.WitchcraftTelemetry do
  @moduledoc """
  Instrumentation wrappers for Witchcraft typeclass operations.
  Use these instead of raw Witchcraft operations in hot paths
  and at module boundaries where you want observability.
  """

  require Logger

  @doc """
  Instrument a functor map operation.
  """
  def measured_map(structure, typeclass, fun) do
    start = System.monotonic_time()
    result = Witchcraft.Functor.map(structure, fun)
    duration = System.monotonic_time() - start

    :telemetry.execute(
      [:my_app, :witchcraft, :operation],
      %{duration: duration, count: 1},
      %{typeclass: :functor, operation: :map, structure: typeclass}
    )

    result
  end

  @doc """
  Instrument a monad bind chain.
  """
  def measured_bind(monad_value, typeclass, fun) do
    start = System.monotonic_time()
    result = Witchcraft.Chain.chain(monad_value, fun)
    duration = System.monotonic_time() - start

    short_circuited = is_nothing?(result)

    :telemetry.execute(
      [:my_app, :witchcraft, :operation],
      %{duration: duration, count: 1},
      %{
        typeclass: :monad,
        operation: :bind,
        structure: typeclass,
        short_circuited: short_circuited
      }
    )

    result
  end

  @doc """
  Instrument a traversal with input size tracking.
  """
  def measured_traverse(collection, typeclass, fun) do
    size = length(collection)
    start = System.monotonic_time()
    result = Witchcraft.Traversable.traverse(collection, fun)
    duration = System.monotonic_time() - start

    :telemetry.execute(
      [:my_app, :witchcraft, :traversal],
      %{duration: duration, count: 1, input_size: size},
      %{typeclass: typeclass}
    )

    if size > 0 do
      per_item_us = div(System.convert_time_unit(duration, :native, :microsecond), size)

      if per_item_us > 1000 do
        Logger.warning("Slow traversal detected",
          typeclass: typeclass,
          size: size,
          per_item_microseconds: per_item_us
        )
      end
    end

    result
  end

  @doc """
  Instrument a fold operation.
  """
  def measured_fold(foldable, typeclass, initial, fun) do
    start = System.monotonic_time()
    result = Witchcraft.Foldable.fold_right(foldable, initial, fun)
    duration = System.monotonic_time() - start

    :telemetry.execute(
      [:my_app, :witchcraft, :operation],
      %{duration: duration, count: 1},
      %{typeclass: :foldable, operation: :fold, structure: typeclass}
    )

    result
  end

  defp is_nothing?(%Algae.Maybe.Nothing{}), do: true
  defp is_nothing?(%Algae.Either.Left{}), do: true
  defp is_nothing?(_), do: false
end

Step 3: Measure Monadic Pipeline Short-Circuits

Track how often monadic chains short-circuit vs. complete:

# lib/my_app/monad_pipeline_monitor.ex
defmodule MyApp.MonadPipelineMonitor do
  @moduledoc """
  Wraps complete monadic computation chains and reports whether the
  chain completed (final monad holds a value) or short-circuited
  (final monad is Nothing/Left/empty).
  """

  require Logger

  def run(pipeline_name, monad_type, pipeline_fun) do
    start = System.monotonic_time()
    result = pipeline_fun.()
    duration = System.monotonic_time() - start

    completed = value_present?(result)

    :telemetry.execute(
      [:my_app, :witchcraft, :pipeline],
      %{duration: duration, count: 1},
      %{
        pipeline: pipeline_name,
        monad: monad_type,
        completed: completed,
        short_circuited: not completed
      }
    )

    unless completed do
      Logger.info("Monad pipeline short-circuited",
        pipeline: pipeline_name,
        monad: monad_type
      )
    end

    result
  end

  defp value_present?(%Algae.Maybe.Just{}), do: true
  defp value_present?(%Algae.Maybe.Nothing{}), do: false
  defp value_present?(%Algae.Either.Right{}), do: true
  defp value_present?(%Algae.Either.Left{}), do: false
  defp value_present?([]), do: false
  defp value_present?(list) when is_list(list), do: true
  defp value_present?(_), do: true
end

Usage:

defmodule MyApp.OrderProcessor do
  def process(order_id) do
    MyApp.MonadPipelineMonitor.run(:order_processing, :maybe, fn ->
      MyApp.DataPipeline.process_order(order_id)
    end)
  end
end

Step 4: Define Telemetry Metrics

# lib/my_app/telemetry.ex
def metrics do
  [
    # Per-operation metrics
    counter("my_app.witchcraft.operation.count",
      tags: [:typeclass, :operation, :structure, :short_circuited]
    ),
    distribution("my_app.witchcraft.operation.duration",
      unit: {:native, :microsecond},
      reporter_options: [buckets: [10, 50, 200, 1000, 5000, 20_000]],
      tags: [:typeclass, :operation, :structure]
    ),

    # Traversal metrics with size tracking
    counter("my_app.witchcraft.traversal.count",
      tags: [:typeclass]
    ),
    distribution("my_app.witchcraft.traversal.duration",
      unit: {:native, :millisecond},
      reporter_options: [buckets: [1, 10, 100, 1000, 10_000]],
      tags: [:typeclass]
    ),
    distribution("my_app.witchcraft.traversal.input_size",
      reporter_options: [buckets: [1, 10, 100, 1000, 10_000]]
    ),

    # Pipeline-level metrics
    counter("my_app.witchcraft.pipeline.count",
      tags: [:pipeline, :monad, :completed, :short_circuited]
    ),
    distribution("my_app.witchcraft.pipeline.duration",
      unit: {:native, :millisecond},
      reporter_options: [buckets: [1, 10, 100, 500, 2000]],
      tags: [:pipeline, :monad]
    )
  ]
end

Step 5: Detect Composition Chain Performance Regression

# lib/my_app/composition_perf_monitor.ex
defmodule MyApp.CompositionPerfMonitor do
  use GenServer
  require Logger

  @window_seconds 300
  @slow_threshold_ms 500
  @poll_interval :timer.seconds(30)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{slow_ops: []}, name: __MODULE__)

  def init(state) do
    :telemetry.attach(
      "witchcraft-perf-tracker",
      [:my_app, :witchcraft, :pipeline],
      &handle_pipeline/4,
      nil
    )

    schedule()
    {:ok, state}
  end

  def handle_pipeline(_event, %{duration: duration, count: 1}, %{pipeline: pipeline}, _) do
    ms = System.convert_time_unit(duration, :native, :millisecond)

    if ms > @slow_threshold_ms do
      GenServer.cast(__MODULE__, {:record_slow, pipeline, ms, System.system_time(:second)})
    end
  end

  def handle_cast({:record_slow, pipeline, ms, ts}, state) do
    entry = {pipeline, ms, ts}
    {:noreply, %{state | slow_ops: [entry | state.slow_ops]}}
  end

  def handle_info(:check, state) do
    cutoff = System.system_time(:second) - @window_seconds

    recent = Enum.filter(state.slow_ops, fn {_, _, ts} -> ts > cutoff end)

    recent
    |> Enum.group_by(fn {pipeline, _, _} -> pipeline end)
    |> Enum.each(fn {pipeline, ops} ->
      count = length(ops)
      avg_ms = Enum.sum(Enum.map(ops, fn {_, ms, _} -> ms end)) / count

      :telemetry.execute(
        [:my_app, :witchcraft, :slow_pipeline_rate],
        %{count: count, avg_ms: avg_ms},
        %{pipeline: pipeline, threshold_ms: @slow_threshold_ms}
      )

      if count > 5 do
        Logger.warning("Functional pipeline repeatedly slow",
          pipeline: pipeline,
          slow_count: count,
          avg_ms: Float.round(avg_ms, 1)
        )
      end
    end)

    pruned = Enum.filter(recent, fn {_, _, ts} -> ts > cutoff end)
    schedule()
    {:noreply, %{state | slow_ops: pruned}}
  end

  defp schedule, do: Process.send_after(self(), :check, @poll_interval)
end

Step 6: Health Endpoint

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    functional_check = check_functional_pipeline()

    checks = %{
      functional_pipeline: functional_check,
      database: check_db()
    }

    status = if functional_check.status == "ok", do: 200, else: 503

    conn
    |> put_status(status)
    |> json(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      checks: checks
    })
  end

  defp check_functional_pipeline do
    start = System.monotonic_time()

    result =
      MyApp.MonadPipelineMonitor.run(:health_probe, :maybe, fn ->
        [1, 2, 3]
        |> MyApp.WitchcraftTelemetry.measured_map(:list, &(&1 * 2))
        |> then(fn list ->
          if Enum.sum(list) == 12 do
            %Algae.Maybe.Just{just: :pipeline_healthy}
          else
            %Algae.Maybe.Nothing{}
          end
        end)
      end)

    duration_ms = System.convert_time_unit(System.monotonic_time() - start, :native, :millisecond)

    case result do
      %Algae.Maybe.Just{} -> %{status: "ok", latency_ms: duration_ms}
      %Algae.Maybe.Nothing{} -> %{status: "error", reason: "pipeline smoke test returned Nothing"}
    end
  rescue
    e -> %{status: "error", reason: Exception.message(e)}
  end

  defp check_db do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", []) do
      {:ok, _} -> %{status: "ok"}
      _ -> %{status: "error"}
    end
  rescue
    _ -> %{status: "error"}
  end
end

Step 7: Create Monitors in Vigilmon

HTTP monitor for the health endpoint:

1. Sign in at vigilmon.online
2. Click New Monitor → HTTP
3. Set URL to https://your-app.example.com/health
4. Set interval to 60 seconds
5. Alert condition: status 200 and body contains "functional_pipeline":{"status":"ok"}

Heartbeat monitor for the functional pipeline subsystem:

# lib/my_app/witchcraft_heartbeat.ex
defmodule MyApp.WitchcraftHeartbeat do
  use GenServer

  @heartbeat_url System.get_env("VIGILMON_WITCHCRAFT_HEARTBEAT_URL")
  @interval :timer.minutes(5)

  def start_link(_), do: GenServer.start_link(__MODULE__, %{}, name: __MODULE__)

  def init(state) do
    schedule()
    {:ok, state}
  end

  def handle_info(:ping, state) do
    probe_result =
      MyApp.MonadPipelineMonitor.run(:heartbeat_probe, :maybe, fn ->
        %Algae.Maybe.Just{just: :heartbeat_ok}
      end)

    if match?(%Algae.Maybe.Just{}, probe_result) do
      ping_vigilmon()
    end

    schedule()
    {:noreply, state}
  end

  defp ping_vigilmon do
    if @heartbeat_url do
      :httpc.request(:get, {String.to_charlist(@heartbeat_url), []}, [{:timeout, 3000}], [])
    end
  end

  defp schedule, do: Process.send_after(self(), :ping, @interval)
end

Create a Heartbeat monitor in Vigilmon with an 8-minute expected interval.

Step 8: Alerting

In Vigilmon, configure Notifications → New Channel:

Slack for high short-circuit rate:

⚠️ DEGRADED: Monad Pipeline Short-Circuits — MyApp
Pipeline: order_processing
Short-circuit rate: 43% over last 5 minutes
Action: Check order_processing Maybe chain — fetch_order or fetch_customer likely returning Nothing

PagerDuty for health endpoint failure:

Page on-call if the health endpoint returns 503 for two consecutive 60-second checks.

Grafana alert for traversal latency:

- alert: WitchcraftTraversalSlow
  expr: |
    histogram_quantile(0.95,
      rate(my_app_witchcraft_traversal_duration_bucket[5m])
    ) > 2000
  for: 3m
  annotations:
    summary: "Witchcraft traversal p95 latency {{ $value }}ms — possible large collection or performance regression"

Common Witchcraft Issues and Fixes

Monad chain short-circuits silently on missing data:

# Replace Maybe with Either to capture the short-circuit reason
def process_order(order_id) do
  monad %Algae.Either{} do
    order <- fetch_order_either(order_id)
    customer <- fetch_customer_either(order.customer_id)
    return(complete_order(order, customer))
  end
end

defp fetch_order_either(id) do
  case MyApp.Repo.get(MyApp.Order, id) do
    nil -> %Algae.Either.Left{left: {:not_found, :order, id}}
    order -> %Algae.Either.Right{right: order}
  end
end

Traversal over large collections causes latency spikes:

# Stream large collections instead of loading all into memory
def process_large_batch(ids) do
  ids
  |> Stream.chunk_every(100)
  |> Enum.flat_map(fn chunk ->
    MyApp.WitchcraftTelemetry.measured_traverse(chunk, :list_chunk, fn id ->
      process_single(id)
    end)
  end)
end

Protocol dispatch overhead in tight loops:

# For performance-critical inner loops, use concrete Enum functions
# Reserve Witchcraft abstractions for module boundaries and composition points
defmodule MyApp.HighThroughputProcessor do
  def process(items) when is_list(items) do
    # Direct Enum in the hot path
    Enum.map(items, &transform/1)
    |> Enum.filter(&valid?/1)
  end

  # Witchcraft abstractions at the module boundary for composability
  def lift_into_maybe(result) do
    import Witchcraft.Applicative
    of(%Algae.Maybe{}, result)
  end
end

Custom typeclass implementation breaking algebraic laws:

# Test that your Functor implementation satisfies identity law: map(f, id) == f
defmodule MyApp.MyFunctorTest do
  use ExUnit.Case
  use Witchcraft

  test "satisfies functor identity law" do
    f = %MyApp.MyFunctor{value: 42}
    assert map(f, &Function.identity/1) == f
  end

  test "satisfies functor composition law" do
    f = %MyApp.MyFunctor{value: 5}
    g = &(&1 * 2)
    h = &(&1 + 1)
    assert map(map(f, g), h) == map(f, fn x -> h.(g.(x)) end)
  end
end

What You've Built

| What | How | |------|-----| | Instrumented typeclass operations | Telemetry on map, bind, traverse, fold with duration | | Monad pipeline monitor | Short-circuit rate tracking for Maybe/Either chains | | Traversal size tracker | Correlates input size with latency to detect scaling issues | | Slow pipeline detector | Rolling-window GenServer alerting on latency regression | | Functional health endpoint | JSON health with composition chain smoke test | | Liveness heartbeat | GenServer pinging Vigilmon after monad probe passes | | Real-time alerting | Vigilmon HTTP monitor + PagerDuty on pipeline failure |

Witchcraft makes Elixir code compositional and algebraically rigorous. Vigilmon makes the runtime behavior of those compositions visible — so you catch performance regressions, short-circuit clusters, and traversal scaling issues before they reach your users.

Next Steps

• Add OpenTelemetry spans to monadic pipelines for distributed tracing across service boundaries
• Build Grafana dashboards showing short-circuit rates per pipeline alongside the total completion rates to visualize your functional pipeline funnel
• Use Witchcraft's Traversable with Task to parallelize effects and instrument the resulting concurrency profile
• Set up Vigilmon monitors for background jobs that use functional pipelines to ensure they remain healthy between user-visible requests

Get started free at vigilmon.online.