Monitoring Langtrace-Instrumented LLM Applications with Vigilmon: External Uptime, API Health & Heartbeat Monitoring

Langtrace is an open-source LLM observability and tracing platform. It captures every LLM call your application makes — token counts, latency, model parameters, vector database queries, and agent execution traces — giving you deep inside-out visibility into what your AI application is doing. What Langtrace doesn't provide is an independent external check: a synthetic probe that measures whether your AI application's public endpoints are reachable and responding correctly from the outside. Vigilmon fills that gap. It sits entirely outside your infrastructure and probes your services from multiple external locations, alerting you independently of whether Langtrace's tracing pipeline is healthy.

This tutorial covers adding external uptime monitoring to LLM applications already instrumented with Langtrace.

What You'll Build

• A /health endpoint that reflects the real state of your AI application's dependencies
• A Vigilmon HTTP monitor with response-body assertions
• A heartbeat monitor for asynchronous LLM batch jobs
• An alert routing strategy that keeps Langtrace tracing alerts and Vigilmon uptime alerts separate but correlated

Prerequisites

• A Langtrace account or self-hosted Langtrace instance with your LLM application sending traces
• At least one AI application or inference API endpoint reachable via a public HTTPS URL
• A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your AI Application

Langtrace traces your LLM calls from inside your application. Vigilmon needs an HTTP endpoint it can reach from the outside. Make the health check verify real dependencies — your LLM provider connectivity, vector store availability, and any backing database — not just a static 200 that would pass even when core dependencies are degraded.

Python (FastAPI with OpenAI and Pinecone)

# app/health.py
import os
import httpx
from fastapi import FastAPI
from fastapi.responses import JSONResponse

app = FastAPI()

@app.get("/health")
async def health():
    checks = {}
    ok = True

    # LLM provider reachability check
    try:
        async with httpx.AsyncClient(timeout=5.0) as client:
            resp = await client.get(
                "https://api.openai.com/v1/models",
                headers={"Authorization": f"Bearer {os.environ['OPENAI_API_KEY']}"},
            )
            checks["llm_provider"] = "ok" if resp.status_code == 200 else f"http_{resp.status_code}"
            if resp.status_code != 200:
                ok = False
    except Exception as exc:
        checks["llm_provider"] = f"error: {exc}"
        ok = False

    # Vector store check
    try:
        import pinecone
        pc = pinecone.Pinecone(api_key=os.environ["PINECONE_API_KEY"])
        pc.list_indexes()
        checks["vector_store"] = "ok"
    except Exception as exc:
        checks["vector_store"] = f"error: {exc}"
        ok = False

    status_code = 200 if ok else 503
    return JSONResponse(
        status_code=status_code,
        content={"status": "ok" if ok else "degraded", "checks": checks},
    )

Node.js (Express with LangChain)

// src/health.js
const express = require('express');
const router = express.Router();

router.get('/health', async (req, res) => {
  const checks = {};
  let ok = true;

  // OpenAI connectivity check
  try {
    const response = await fetch('https://api.openai.com/v1/models', {
      headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
      signal: AbortSignal.timeout(5000),
    });
    checks.llm_provider = response.ok ? 'ok' : `http_${response.status}`;
    if (!response.ok) ok = false;
  } catch (err) {
    checks.llm_provider = `error: ${err.message}`;
    ok = false;
  }

  // Redis / semantic cache check
  try {
    await redisClient.ping();
    checks.cache = 'ok';
  } catch (err) {
    checks.cache = `error: ${err.message}`;
    ok = false;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    checks,
  });
});

module.exports = router;

Verify the endpoint before configuring Vigilmon:

curl -s https://your-ai-app.example.com/health | jq .
# {"status":"ok","checks":{"llm_provider":"ok","vector_store":"ok"}}

Step 2: Configure Vigilmon HTTP Monitor

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: https://your-ai-app.example.com/health
3. Check interval: 60 seconds — appropriate for production AI APIs.
4. Response timeout: 15 seconds — LLM health checks can be slower due to provider round-trips.
5. Expected status code: 200.
6. JSON body assertion:
   • Path: status
   • Expected value: ok
7. Click Save.

Vigilmon immediately begins probing from multiple external probe locations. A non-200 response or a failed JSON assertion triggers an alert independently of Langtrace's own health.

Multiple endpoints and environments

AI applications commonly expose multiple endpoints: an inference API, an embeddings endpoint, and an admin API. Mirror each in Vigilmon:

| Endpoint | Vigilmon monitor URL | Interval | |---|---|---| | Inference API | https://api.example.com/health | 60 s | | Embeddings service | https://embeddings.example.com/health | 60 s | | RAG retrieval API | https://rag.example.com/health | 60 s | | Staging inference | https://api-staging.example.com/health | 120 s |

Group monitors under a single Monitor group in Vigilmon to keep the dashboard organized.

Step 3: Heartbeat Monitor for LLM Batch Jobs

Langtrace can trace synchronous LLM calls in real time, but asynchronous batch jobs — nightly embeddings refreshes, daily fine-tuning runs, scheduled RAG index rebuilds — run outside your request/response lifecycle. Use Vigilmon heartbeats to guarantee these jobs complete on schedule.

# jobs/refresh_embeddings.py
import os
import requests

def refresh_embeddings():
    """Rebuild the RAG vector index from updated source documents."""
    load_documents_from_store()
    generate_embeddings_with_openai()
    upsert_to_pinecone()

if __name__ == "__main__":
    try:
        refresh_embeddings()
        requests.post(os.environ["VIGILMON_HEARTBEAT_URL"], timeout=5)
        print("Embeddings refresh complete — heartbeat sent")
    except Exception as exc:
        # Deliberate silence — missed heartbeat fires Vigilmon alert
        print(f"Embeddings refresh failed: {exc}")
        raise

In Vigilmon, create a Heartbeat monitor and set the grace period slightly above your cron interval. For a nightly job, use 25 hours. Store the heartbeat URL in your secrets manager and inject it as VIGILMON_HEARTBEAT_URL.

Kubernetes CronJob for embeddings refresh

# k8s/embeddings-refresh.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: embeddings-refresh
spec:
  schedule: "0 3 * * *"  # 3 AM daily
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: refresh
            image: your-registry/embeddings-job:latest
            env:
            - name: VIGILMON_HEARTBEAT_URL
              valueFrom:
                secretKeyRef:
                  name: vigilmon-secrets
                  key: heartbeat-url
            - name: OPENAI_API_KEY
              valueFrom:
                secretKeyRef:
                  name: openai-secrets
                  key: api-key
          restartPolicy: Never

Step 4: Alert Routing Strategy

Langtrace alerts on LLM-layer signals: rising token costs, increased latency percentiles, high error rates from specific models, hallucination spikes. Vigilmon alerts on infrastructure-layer availability: the endpoint is down, the health check is failing, the batch job missed its window. These are distinct failure modes.

| Failure mode | Source | Route to | |---|---|---| | LLM latency spike (p95 > threshold) | Langtrace | Slack #ai-ops + ticket | | Token cost anomaly | Langtrace | Engineering lead + billing alert | | External API endpoint down | Vigilmon | PagerDuty on-call + Slack #prod-alerts | | Embeddings refresh job missed | Vigilmon | Slack #ai-ops-critical | | TLS certificate expiring | Vigilmon | DevOps team |

Configure Vigilmon alert channels under Alerts → Add channel:

• Email: your on-call distribution list.
• PagerDuty webhook: wire to your primary on-call rotation for endpoint outages.
• Slack webhook: a dedicated #vigilmon-alerts channel separate from Langtrace dashboards.

Step 5: Correlating Langtrace Traces and Vigilmon Alerts During Incidents

When Vigilmon fires a downtime alert for an AI application endpoint, use this checklist to determine root cause:

1. Check Vigilmon probe regions — did all probe locations fail or just one? All failing points to your application or LLM provider. One region failing suggests a network routing issue.
2. Open Langtrace → check the LLM call error rate for the same time window. A sudden spike in provider errors often precedes an availability failure.
3. Check token quota and rate limits — LLM provider rate-limit exhaustion can cause your health endpoint to return 503 even when your application code is healthy.
4. Compare timestamps — did Langtrace show rising errors before Vigilmon fired? If so, the LLM provider issue likely caused the downtime.

Step 6: Public Status Page

Langtrace traces are private internal telemetry. Give customers and API consumers a public-facing status page using Vigilmon.

1. In Vigilmon, go to Status Pages → Create.
2. Add your production inference API monitor and any critical heartbeat monitors.
3. Configure a custom domain (e.g., status.example.com) or use the Vigilmon subdomain.
4. Embed the status badge in your developer documentation:

<img src="https://vigilmon.online/badge/<monitor-id>.svg" alt="API Status" />

Your API consumers get real-time uptime visibility without requiring access to Langtrace or your internal infrastructure.

What Vigilmon Adds to a Langtrace Stack

| Capability | Langtrace | Vigilmon | |---|---|---| | LLM call traces and token counts | Yes | No | | Model latency percentiles | Yes | No | | Vector DB query observability | Yes | No | | Agent execution step tracing | Yes | No | | External synthetic HTTP probing | No | Yes | | DNS resolution failure detection | No | Yes | | TLS/SSL certificate expiry alerts | No | Yes | | Scheduled job heartbeat monitoring | No | Yes | | Public status page | No | Yes | | Independent of LLM provider health | No | Yes |

Langtrace gives you deep visibility into every LLM call your application makes. Vigilmon gives you the external, synthetic view that tells you what your users actually experience when they try to reach your AI application. Together they cover every dimension of AI service reliability: from model-level traces to customer-facing uptime.

Add external uptime monitoring to your Langtrace-instrumented application today — register free at vigilmon.online.