How to Monitor Domo with Vigilmon

Domo is an Elixir library that enforces struct typespecs at both compile time and runtime, ensuring data flowing through your application conforms to the types you declared. It generates validation functions directly from your @type t specs, so a User.new/1 call that passes an integer where a string is expected raises immediately rather than failing silently ten modules later. When Domo validation failures spike — because an upstream API changed its payload shape, a migration introduced a schema mismatch, or an external vendor started sending unexpected types — you need external visibility to catch it before your end users do. Vigilmon adds uptime monitoring, heartbeat checks, and alerting so struct contract violations surface as operational alerts, not buried log lines.

What You'll Set Up

• HTTP health endpoint that reports Domo validation error rates
• Heartbeat monitor for background jobs that process Domo-validated structs
• Slack alerts on validation failure spikes
• SSL monitoring for services your validated data pipeline calls

Prerequisites

• Elixir 1.14+ with domo in mix.exs
• A Phoenix or Plug application using Domo-validated structs
• A free Vigilmon account

Step 1: Add Runtime Validation Tracking to Your Structs

Domo's use_type_ensure!/1 and ensure_type!/1 guards surface type mismatches at call sites. To make these failures observable, add a lightweight counter that your health endpoint can read:

# lib/my_app/validation_tracker.ex
defmodule MyApp.ValidationTracker do
  use Agent

  def start_link(_opts) do
    Agent.start_link(fn -> %{failures: 0, last_failure: nil, last_struct: nil} end,
      name: __MODULE__)
  end

  def record_failure(struct_module, reason) do
    Agent.update(__MODULE__, fn state ->
      %{state |
        failures: state.failures + 1,
        last_failure: DateTime.utc_now(),
        last_struct: struct_module
      }
    end)
  end

  def stats do
    Agent.get(__MODULE__, & &1)
  end

  def reset do
    Agent.update(__MODULE__, fn state -> %{state | failures: 0} end)
  end
end

Add it to your supervision tree:

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  MyAppWeb.Endpoint,
  MyApp.ValidationTracker,
  # ...
]

Wrap Domo validation calls in your domain layer to capture failures:

# lib/my_app/domain/user.ex
defmodule MyApp.Domain.User do
  use Domo

  @enforce_keys [:id, :email, :role]
  defstruct [:id, :email, :role, :name]

  @type t :: %__MODULE__{
    id: pos_integer(),
    email: String.t(),
    role: :admin | :member | :viewer,
    name: String.t() | nil
  }
end

# lib/my_app/user_factory.ex
defmodule MyApp.UserFactory do
  alias MyApp.Domain.User
  alias MyApp.ValidationTracker

  def build(attrs) do
    case User.new(attrs) do
      {:ok, user} ->
        {:ok, user}

      {:error, reason} ->
        ValidationTracker.record_failure(User, reason)
        {:error, {:validation_failed, reason}}
    end
  end
end

Step 2: Add a Health Endpoint That Exposes Validation State

# lib/my_app_web/controllers/health_controller.ex
defmodule MyAppWeb.HealthController do
  use MyAppWeb, :controller

  alias MyApp.ValidationTracker

  def index(conn, _params) do
    stats = ValidationTracker.stats()

    # Consider unhealthy if more than 10 validation failures since last reset
    validation_ok = stats.failures < 10

    checks = %{
      validation_failures: stats.failures,
      last_failure_at: stats.last_failure,
      last_failed_struct: stats.last_struct && inspect(stats.last_struct),
      database: check_database(),
      domo_validation: if(validation_ok, do: :ok, else: :degraded)
    }

    status = if validation_ok and checks.database == :ok, do: 200, else: 503

    conn
    |> put_status(status)
    |> json(%{
      status: if(status == 200, do: "ok", else: "degraded"),
      checks: checks
    })
  end

  defp check_database do
    case Ecto.Adapters.SQL.query(MyApp.Repo, "SELECT 1", []) do
      {:ok, _} -> :ok
      _ -> :error
    end
  end
end

Register the route:

# lib/my_app_web/router.ex
scope "/", MyAppWeb do
  get "/health", HealthController, :index
end

Test it locally:

mix phx.server
curl -s http://localhost:4000/health | jq .
# {
#   "status": "ok",
#   "checks": {
#     "validation_failures": 0,
#     "domo_validation": "ok",
#     "database": "ok"
#   }
# }

Step 3: Set Up HTTP Monitoring in Vigilmon

1. Sign in at vigilmon.online
2. Click New Monitor → HTTP
3. URL: https://yourdomain.com/health
4. Expected status: 200
5. Add a keyword check: "domo_validation":"ok" to confirm validation state is healthy
6. Interval: 1 minute
7. Save

The keyword check is essential — a 200 response with "domo_validation":"degraded" inside indicates a struct contract violation wave is in progress and you need to investigate the upstream data source.

Step 4: Heartbeat Monitoring for Struct-Processing Background Jobs

If you use Oban or a GenServer to batch-process external payloads through Domo-validated structs, add a heartbeat so Vigilmon detects when that pipeline stalls:

# lib/my_app/workers/payload_ingestion_worker.ex
defmodule MyApp.Workers.PayloadIngestionWorker do
  use Oban.Worker, queue: :ingestion, max_attempts: 3

  require Logger

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"batch_id" => batch_id}}) do
    with {:ok, payloads} <- fetch_payloads(batch_id),
         {:ok, _structs} <- validate_and_build(payloads) do
      ping_heartbeat()
      :ok
    else
      {:error, {:validation_failed, reason}} ->
        Logger.error("Domo validation failed for batch #{batch_id}: #{inspect(reason)}")
        {:error, :validation_failed}

      {:error, reason} ->
        Logger.error("Ingestion failed for batch #{batch_id}: #{inspect(reason)}")
        {:error, reason}
    end
  end

  defp fetch_payloads(batch_id) do
    # Fetch raw payloads from external source
    {:ok, []}
  end

  defp validate_and_build(payloads) do
    results = Enum.map(payloads, &MyApp.UserFactory.build/1)
    errors = Enum.filter(results, &match?({:error, _}, &1))

    if Enum.empty?(errors) do
      {:ok, Enum.map(results, fn {:ok, s} -> s end)}
    else
      {:error, {:validation_failed, length(errors)}}
    end
  end

  defp ping_heartbeat do
    url = Application.get_env(:my_app, :vigilmon)[:ingestion_heartbeat_url]

    if url do
      Req.get(url, receive_timeout: 5_000)
    end
  end
end

In Vigilmon:

1. Click New Monitor → Heartbeat
2. Name: payload-ingestion-worker
3. Expected interval: match your Oban schedule (e.g. 10 minutes)
4. Copy the ping URL and set it as VIGILMON_INGESTION_HEARTBEAT_URL

Step 5: Alert on Validation Failures via Slack

1. In Vigilmon: Notifications → New Channel → Slack
2. Paste your Slack incoming webhook URL
3. Enable it on your /health monitor and the ingestion heartbeat

Alert messages look like:

🔴 DOWN: yourdomain.com/health
Keyword "domo_validation":"ok" not found
Detected: EU-West, US-East

Add the Slack channel to your notification group, then attach that group to both monitors so both alert to the same channel.

Step 6: SSL Monitoring for Upstream APIs

Domo validates the shape of data you receive — but if the upstream API's TLS certificate expires, you stop receiving data entirely. Add SSL monitors for each external source:

1. New Monitor → SSL Certificate → https://api.upstream-vendor.com
2. Alert threshold: 14 days before expiry

What You've Built

| What | How | |------|-----| | Validation failure tracking | ValidationTracker Agent counting Domo errors | | Active health check | /health endpoint with domo_validation keyword | | Background job monitoring | Heartbeat on payload ingestion Oban worker | | Validation spike alerts | Slack notification when health degrades | | Upstream SSL monitoring | Certificate expiry alerts for data sources |

Domo eliminates an entire class of runtime type bugs at the Elixir level. Vigilmon ensures the data pipelines feeding your Domo-validated structs stay healthy at the infrastructure level.

Next Steps

• Reset ValidationTracker counters on a schedule so spikes reflect recent activity, not all-time counts
• Add a Vigilmon response-time monitor on your ingestion endpoint to track payload processing latency
• Use Domo's ensure_type!/2 with custom error messages to make Slack alert bodies more actionable
• Add a second heartbeat for each Oban queue that processes validated structs

Get started free at vigilmon.online.