How to Monitor Guardrails AI with Vigilmon

Guardrails AI wraps your LLM calls with composable validators — PII detection, toxicity filters, JSON format checks, factuality guards, and more — enforcing safety and quality constraints on both inputs and outputs in real time. In production, a Guardrails AI guard server becomes a critical gateway between your users and your LLMs. If the guard server goes down, your app either passes unvalidated content to the LLM or blocks all requests entirely. Vigilmon gives you HTTP health checks for your Guardrails AI server, heartbeat monitoring for asynchronous validation pipelines, SSL alerts for your guard endpoints, and instant notifications when the safety layer goes offline.

What You'll Set Up

• HTTP uptime monitor for your Guardrails AI guard server
• Cron heartbeat for async validation and batch audit jobs
• SSL certificate expiry alerts for your guard endpoint
• Alert channels for immediate notification on guard failures

Prerequisites

• Guardrails AI installed (pip install guardrails-ai) and the guard server running
• At least one guard configured and deployed
• A free Vigilmon account

Step 1: Verify the Guardrails AI Server Health Endpoint

Guardrails AI ships with a built-in server (guardrails start) that exposes a REST API for guard execution. The server includes a health endpoint out of the box:

# Start the Guardrails AI server
guardrails start --port 8000

The server exposes:

• GET /health — server liveness check
• POST /guards/{guard_name}/validate — run a guard

Test it before setting up monitoring:

curl http://localhost:8000/health
# {"status": "healthy"}

If you're wrapping Guardrails AI in a custom FastAPI app, add your own health endpoint that also confirms your guards are registered:

from fastapi import FastAPI
from fastapi.responses import JSONResponse
import guardrails as gd
from guardrails.hub import DetectPII, ToxicLanguage

app = FastAPI()

# Initialize guards at startup
pii_guard = gd.Guard().use(DetectPII, pii_entities=["EMAIL_ADDRESS", "PHONE_NUMBER"])
toxicity_guard = gd.Guard().use(ToxicLanguage, threshold=0.5)

GUARDS = {
    "pii": pii_guard,
    "toxicity": toxicity_guard,
}

@app.get("/health")
async def health():
    return {
        "status": "ok",
        "guards_loaded": list(GUARDS.keys()),
        "guard_count": len(GUARDS),
    }

Step 2: Monitor Your Guardrails AI Endpoint

With the health endpoint confirmed, add a Vigilmon monitor:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your guard server URL: https://guards.yourdomain.com/health
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

Because the guard server sits between your users and your LLMs, a 1-minute check interval means you know within 60 seconds if the safety layer is unavailable — before too many unvalidated requests reach your LLM or too many users see errors.

Step 3: Heartbeat Monitoring for Async Validation Jobs

Guardrails AI supports asynchronous validation for streaming responses and batch audit pipelines. These background jobs — running compliance checks over conversation logs, auditing model outputs for policy violations, or revalidating historical data — don't serve HTTP traffic. Use a Vigilmon cron heartbeat to confirm they complete on schedule:

1. In Vigilmon, click Add Monitor → Cron Heartbeat.
2. Set the expected ping interval to match your audit schedule (e.g. 1440 minutes for daily).
3. Copy the heartbeat URL (e.g. https://vigilmon.online/heartbeat/abc123).

Add the ping to your audit job after successful completion:

import httpx
import guardrails as gd
from guardrails.hub import DetectPII, FactualityEval

def run_daily_audit(conversation_logs: list[dict]):
    pii_guard = gd.Guard().use(DetectPII)
    factuality_guard = gd.Guard().use(FactualityEval)

    violations = []
    for log in conversation_logs:
        # Check for PII in responses
        try:
            pii_guard.validate(log["response"])
        except gd.errors.ValidationError as e:
            violations.append({"type": "pii", "log_id": log["id"], "detail": str(e)})

        # Check factuality
        try:
            factuality_guard.validate(log["response"], metadata={"sources": log["sources"]})
        except gd.errors.ValidationError as e:
            violations.append({"type": "factuality", "log_id": log["id"], "detail": str(e)})

    save_audit_report(violations)

    # Signal success to Vigilmon
    httpx.get("https://vigilmon.online/heartbeat/abc123", timeout=5)

if __name__ == "__main__":
    logs = load_conversation_logs()
    run_daily_audit(logs)

If the audit job crashes mid-run or a guard throws an unhandled exception, Vigilmon alerts after the expected interval — keeping your compliance records current.

Step 4: Monitor Guard-Specific Endpoints

When running multiple guards as separate services (e.g., separate deployments for PII, toxicity, and factuality), add individual monitors for each guard endpoint:

https://guards.yourdomain.com/health/pii
https://guards.yourdomain.com/health/toxicity
https://guards.yourdomain.com/health/factuality

Expose per-guard readiness in your app:

@app.get("/health/{guard_name}")
async def guard_health(guard_name: str):
    if guard_name not in GUARDS:
        return JSONResponse(status_code=404, content={"error": "Unknown guard"})
    try:
        # Run a no-op validation to confirm the guard is functional
        GUARDS[guard_name].validate("health check")
        return {"status": "ok", "guard": guard_name}
    except Exception as e:
        return JSONResponse(
            status_code=503,
            content={"status": "error", "guard": guard_name, "detail": str(e)},
        )

Monitoring guards individually lets you identify which validator failed when a composite guard pipeline throws errors.

Step 5: SSL Certificate Alerts

The Guardrails AI guard server is a security-critical component — SSL expiry can break TLS verification for clients that enforce certificate validation:

1. Open the HTTP monitor for your guard server (created in Step 2).
2. Enable Monitor SSL certificate under the SSL section.
3. Set Alert when certificate expires in less than 21 days.
4. Click Save.

A 21-day window gives enough runway to renew before clients start rejecting the certificate and safety validation is bypassed due to connection errors.

Step 6: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook endpoint.
2. Set Consecutive failures before alert to 2 to avoid false positives from momentary network blips during guard validator hot-reloading.
3. Use Maintenance windows when deploying updated guard configurations:

# Before deploying updated guards or validator models
curl -X POST https://vigilmon.online/api/maintenance \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{"monitor_id": "abc123", "duration_minutes": 5}'

# Restart the guard server with updated config
guardrails start --config updated_guards.py --port 8000

# Maintenance window expires automatically

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP endpoint | /health on guard server | Guard server down, validator init failure | | Per-guard health | /health/{guard_name} | Individual validator failure | | Cron heartbeat | Heartbeat URL | Audit job crash, missed compliance run | | SSL certificate | Guard server domain | Certificate expiry |

Guardrails AI keeps your LLM applications safe with composable validators — but the guard server itself is now a critical piece of production infrastructure. With Vigilmon watching your guard endpoints, audit heartbeats, and SSL certificates, you know immediately when your safety layer needs attention and can restore it before unvalidated content reaches your users.