How to Monitor LangWatch with Vigilmon

LangWatch is an open-source LLM monitoring and evaluation platform that gives you distributed traces for LLM chains, customizable quality metrics, PII detection, cost tracking, and alerting for latency or quality regressions. If your team runs LangWatch to observe a production AI application, LangWatch itself becomes a critical dependency — and like any infrastructure component, it can go down or degrade.

This guide covers how to monitor LangWatch with Vigilmon so you catch ingestion failures, API outages, and dashboard unavailability before they create blind spots in your LLM observability pipeline.

Why Monitor LangWatch?

When LangWatch is healthy you get traces, evaluations, and cost data in real time. When it's not, you lose visibility silently:

• Trace ingestion fails — your LLM calls complete normally, but traces are dropped or queued. You won't know until you look at the dashboard and notice the gap.
• Evaluation pipeline stalls — quality metrics stop updating, so regressions go undetected while the underlying model degrades.
• Dashboard goes unavailable — on-call engineers can't investigate an active AI incident because the observability tool is also down.
• Self-hosted LangWatch goes offline — if you run LangWatch on your own infrastructure, a config change or resource exhaustion can take it down entirely.

External monitoring from Vigilmon catches all of these from outside your application stack.

Key Metrics to Monitor

| Metric | Why it matters | |--------|---------------| | API server availability | Core ingestion and query endpoints must respond | | Trace ingestion latency | Slow ingest means delayed observability | | Dashboard HTTP response | Engineers need dashboard access during incidents | | Evaluation endpoint health | Quality regressions go undetected if evals stall | | Database connectivity | LangWatch stores traces and evals in a backing DB | | Queue depth (self-hosted) | A growing queue means ingestion is falling behind |

Setting Up Vigilmon Monitors for LangWatch

1. API Health Endpoint

LangWatch exposes a health endpoint you can probe directly:

For LangWatch Cloud:

1. Log in to Vigilmon → Monitors → New Monitor
2. Type: HTTP
3. Method: GET
4. URL: https://app.langwatch.ai/api/health
5. Keyword check: ok or healthy
6. Interval: 2 minutes
7. Response timeout: 10 seconds
8. Save

For self-hosted LangWatch:

Replace the URL with your own instance: https://langwatch.yourdomain.com/api/health

2. Trace Ingestion Endpoint

The ingestion API is the most critical endpoint — if it's down, all LLM traces are lost:

1. Type: HTTP
2. Method: POST
3. URL: https://app.langwatch.ai/api/collector (or your self-hosted equivalent)
4. Custom headers:
   • X-Auth-Token: your-langwatch-api-key
   • Content-Type: application/json
5. Body: {"traces":[]}
6. Expected status: 200 or 202 (empty payload should be accepted or rejected gracefully — watch for 5xx)
7. Interval: 5 minutes
8. Save

This is a synthetic ingest check. The empty payload won't create real traces, but a 5xx response signals the ingestion pipeline is broken.

3. Dashboard Availability

Your dashboard being up matters independently of the API:

1. Type: HTTP
2. Method: GET
3. URL: https://app.langwatch.ai (or your self-hosted URL)
4. Keyword check: LangWatch
5. Interval: 5 minutes
6. Save

Monitoring Self-Hosted LangWatch

If you run LangWatch on your own infrastructure (Docker Compose or Kubernetes), add monitors for the backing services too.

PostgreSQL / ClickHouse connectivity check

Expose a lightweight database health endpoint from your LangWatch instance and probe it:

# health.py — add to your LangWatch deployment
from fastapi import FastAPI
import asyncpg
import os

app = FastAPI()

@app.get("/health/db")
async def db_health():
    conn = await asyncpg.connect(os.environ["DATABASE_URL"])
    await conn.fetchval("SELECT 1")
    await conn.close()
    return {"db": "ok"}

Point Vigilmon at https://langwatch.yourdomain.com/health/db with keyword check "db":"ok".

Redis / queue health

If LangWatch uses Redis for trace queuing:

@app.get("/health/queue")
async def queue_health():
    import redis
    r = redis.from_url(os.environ["REDIS_URL"])
    r.ping()
    queue_len = r.llen("trace_queue")
    return {"queue": "ok", "depth": queue_len}

Monitor this endpoint and set a keyword absence alert if depth exceeds a threshold — a growing queue means ingestion is falling behind processing capacity.

Instrumenting Your Application to Detect LangWatch Failures

Add a heartbeat sender to your application that signals when traces are successfully ingested. If LangWatch goes down, the heartbeat stops and Vigilmon alerts you:

// lib/langwatch-heartbeat.ts
import LangWatch from 'langwatch';

export async function sendLangWatchHeartbeat(): Promise<void> {
  const client = new LangWatch({
    apiKey: process.env.LANGWATCH_API_KEY,
  });

  const trace = client.getTrace();

  try {
    await trace.update({
      metadata: { type: 'heartbeat', source: 'monitoring' },
    });

    // Ping Vigilmon heartbeat URL to confirm successful ingest
    await fetch(process.env.VIGILMON_HEARTBEAT_URL!, { method: 'POST' });
  } catch (err) {
    console.error('[LangWatch heartbeat] failed:', err);
  }
}

In Vigilmon, create a Heartbeat monitor and configure it to alert if no ping arrives within 10 minutes. Run the heartbeat on a 5-minute cron.

Alerting Configuration

Structure alerts to match the severity of each failure:

| Monitor | Severity | Channel | |---------|----------|---------| | LangWatch API health | Critical | PagerDuty + Slack #ai-ops | | Trace ingestion endpoint | Critical | PagerDuty + Slack #ai-ops | | Dashboard availability | High | Slack #ai-ops | | DB health (self-hosted) | Critical | PagerDuty | | Queue depth (self-hosted) | Warning | Slack #ai-ops | | Heartbeat monitor | High | Slack #ai-ops + email |

Recommended thresholds:

• Response time warning: 3000ms
• Response time critical: 10000ms
• Confirm failures after: 2 consecutive checks (avoids flapping alerts)

Building a LangWatch Observability Dashboard

Group all LangWatch monitors in Vigilmon for a single-pane view:

1. Vigilmon → Groups → New Group
2. Name: "LangWatch Observability"
3. Add: API health, ingestion endpoint, dashboard, DB health, heartbeat
4. Share the group status page with your AI engineering team

This lets on-call engineers see LangWatch health at a glance when they're investigating AI quality incidents — before they even open the LangWatch dashboard itself.

Summary

| Monitor | URL/endpoint | Check type | |---------|-------------|------------| | API health | /api/health | HTTP GET, keyword | | Trace ingestion | /api/collector | HTTP POST, status code | | Dashboard | / | HTTP GET, keyword | | DB health | /health/db | HTTP GET, keyword | | Heartbeat | Vigilmon heartbeat URL | Heartbeat timeout |

Key principles:

• Monitor the ingestion endpoint separately from the dashboard — they can fail independently
• Use a heartbeat monitor to detect silent ingest failures that don't surface as HTTP errors
• On self-hosted deployments, monitor backing services (DB, Redis) in addition to the LangWatch API
• Set latency thresholds — slow trace ingestion delays your observability feedback loop

With Vigilmon watching LangWatch, you'll know the moment your LLM observability pipeline develops a gap — so you can fix it before a model regression goes undetected.

Further Reading

• Monitoring AI/LLM API Dependencies with Vigilmon
• Monitoring PromptLayer with Vigilmon
• Best Uptime Monitoring Tools for Developers 2026