How to Monitor CrewAI with Vigilmon

CrewAI is an open-source multi-agent orchestration framework that lets you build teams of autonomous AI agents, each with defined roles, tools, and goals, working together on complex tasks. Research pipelines, content workflows, and automated business processes built on CrewAI involve multiple LLM calls, tool invocations, and handoffs between agents — any of which can fail silently.

Without monitoring, a crew that's supposed to run every hour might be silently failing for days. This guide covers how to use Vigilmon to keep your CrewAI deployments healthy.

Why Monitor CrewAI Deployments

CrewAI crews are long-running, multi-step processes. The failure modes are different from a simple HTTP endpoint:

1. Silent task failures — an agent might return a partial result instead of failing loudly, and the crew completes with incorrect output
2. Tool call failures — a web search tool, code execution tool, or file I/O tool fails mid-execution and the crew retries or skips silently
3. LLM provider degradation — if the underlying LLM (OpenAI, Anthropic, Groq) is slow, your crew takes 10× longer than expected
4. Memory and context overflow — long-running agents can hit context limits, causing truncated reasoning or dropped task context
5. Scheduled crew skips — if you run crews on a schedule and the scheduler fails, you may not notice for hours

Vigilmon gives you heartbeat monitoring (crews phone home on success), HTTP monitoring of any health endpoints you expose, and alerting when things stop as expected.

Key Metrics to Monitor

| Metric | What it indicates | |--------|------------------| | Crew execution time | LLM latency spike or stuck task | | Heartbeat regularity | Scheduled crew is running on time | | Agent task success rate | Individual agent failures within a crew | | Tool invocation errors | External tool (search, code exec) is failing | | LLM API latency | Upstream provider degradation | | Output validation pass rate | Crew returning malformed or incomplete results |

Setup Guide

1. Add a Heartbeat Monitor in Vigilmon

The simplest and most reliable way to monitor a scheduled CrewAI crew is a heartbeat (dead man's switch). Vigilmon alerts you if the crew stops checking in.

In Vigilmon:

1. Monitors → New Monitor
2. Type: Heartbeat
3. Name: CrewAI Research Crew — Hourly
4. Expected interval: 60 minutes
5. Grace period: 10 minutes
6. Save — copy the heartbeat ping URL (e.g. https://vigilmon.online/ping/abc123)

Now add the ping to your crew's kickoff script:

import requests
from crewai import Crew, Agent, Task

VIGILMON_HEARTBEAT_URL = "https://vigilmon.online/ping/abc123"

def run_research_crew():
    researcher = Agent(
        role="Senior Research Analyst",
        goal="Find and synthesize the latest information on the topic",
        backstory="You are an expert researcher with access to web search tools.",
        tools=[search_tool],
        verbose=True,
    )

    research_task = Task(
        description="Research the latest developments in {topic} and write a summary.",
        expected_output="A concise 3-paragraph summary with key findings.",
        agent=researcher,
    )

    crew = Crew(
        agents=[researcher],
        tasks=[research_task],
        verbose=True,
    )

    result = crew.kickoff(inputs={"topic": "AI agent frameworks 2026"})

    # Only ping Vigilmon after successful completion
    requests.get(VIGILMON_HEARTBEAT_URL, timeout=5)
    return result

if __name__ == "__main__":
    run_research_crew()

If the crew throws an exception, the heartbeat ping never fires and Vigilmon alerts you after the grace period expires.

2. Expose a Health Endpoint

For crews running inside a web application or API server, expose a /health/crewai endpoint that reports recent crew execution status:

# health/crewai_health.py
import time
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

# Track last successful crew run in your persistence layer
last_crew_success: dict = {"timestamp": None, "duration_ms": None}

class CrewHealth(BaseModel):
    ok: bool
    last_success_timestamp: float | None
    last_duration_ms: int | None
    seconds_since_last_run: float | None

@app.get("/health/crewai", response_model=CrewHealth)
def crewai_health():
    ts = last_crew_success.get("timestamp")
    seconds_since = (time.time() - ts) if ts else None
    ok = seconds_since is not None and seconds_since < 3900  # fail if >65 min

    return CrewHealth(
        ok=ok,
        last_success_timestamp=ts,
        last_duration_ms=last_crew_success.get("duration_ms"),
        seconds_since_last_run=seconds_since,
    )

Point Vigilmon at this endpoint:

1. Monitors → New Monitor
2. Type: HTTP
3. URL: https://yourapp.com/health/crewai
4. Keyword check: "ok":true
5. Interval: 10 minutes
6. Save

3. Monitor the Underlying LLM Provider

CrewAI crews depend on the LLM API you configure. Set up a separate Vigilmon monitor for your provider:

# For OpenAI-backed crews
https://api.openai.com/v1/models  (Authorization: Bearer sk-...)

# For Anthropic-backed crews
https://api.anthropic.com/v1/models  (x-api-key: sk-ant-...)

# For Groq-backed crews
https://api.groq.com/openai/v1/models  (Authorization: Bearer gsk_...)

See the AI API monitoring guide for detailed setup.

4. Instrument Individual Agent Tasks

For production crews, instrument each task completion with execution timing:

import time
import requests
from crewai import Crew, Agent, Task
from crewai.callbacks import BaseCallback

VIGILMON_METRICS_WEBHOOK = "https://vigilmon.online/webhook/your-webhook-id"

class MonitoringCallback(BaseCallback):
    def __init__(self):
        self.task_timings = {}

    def on_task_start(self, task, **kwargs):
        self.task_timings[id(task)] = time.time()

    def on_task_end(self, task, output, **kwargs):
        start = self.task_timings.pop(id(task), time.time())
        duration_ms = int((time.time() - start) * 1000)

        # Log to your observability stack
        print(f"[crewai] task completed in {duration_ms}ms: {task.description[:60]}")

        # Alert if a single task takes more than 3 minutes
        if duration_ms > 180_000:
            requests.post(VIGILMON_METRICS_WEBHOOK, json={
                "event": "slow_task",
                "task": task.description[:100],
                "duration_ms": duration_ms,
            }, timeout=5)

crew = Crew(
    agents=[...],
    tasks=[...],
    callbacks=[MonitoringCallback()],
)

Alerting

Configure Vigilmon alerts for your CrewAI monitors:

1. Open each monitor → Alerts
2. Add notification channels:
   • Slack: #ai-ops channel for immediate crew failure notification
   • Email: On-call engineer for heartbeat misses
   • PagerDuty: Only if the crew output feeds a customer-facing feature

Recommended thresholds:

| Monitor | Alert after | Escalate after | |---------|------------|----------------| | Heartbeat (hourly crew) | 1 missed heartbeat | 2 consecutive misses | | HTTP health endpoint | 2 consecutive failures | 3 consecutive failures | | LLM API probe | 1 failure | 15-minute sustained failure |

Set the response time warning threshold on your HTTP health endpoint to 5000ms — a slow health endpoint often signals a crew is mid-execution and taking longer than expected.

Monitoring Multi-Crew Pipelines

If you run multiple crews in sequence (one crew's output feeds the next), create a monitor group:

1. Groups → New Group
2. Name: "CrewAI Pipeline"
3. Add all crew heartbeat and health monitors
4. Set group alert policy: alert if any monitor fails

This gives you a single dashboard view and a single alert for the entire pipeline.

Conclusion

CrewAI makes it easy to build powerful multi-agent workflows, but that power comes with operational complexity. Heartbeat monitors catch scheduled crews that stop running. Health endpoint monitors catch crews that are alive but returning bad data. LLM provider monitors catch upstream degradation before your users notice AI features breaking.

Set up a Vigilmon heartbeat for every scheduled crew today — it takes five minutes and catches the most common failure mode: a crew that silently stops running.

Further Reading

• Monitoring AI/LLM API Dependencies with Vigilmon
• Monitoring AutoGen Multi-Agent Workflows with Vigilmon
• Monitoring Groq Inference with Vigilmon