tutorial

Monitoring SourceBot with Vigilmon

SourceBot lets developers ask questions about your codebase directly in Slack or Teams — but a stale index or downed vector database means every answer is wrong or missing. Here's how to monitor your self-hosted SourceBot with Vigilmon.

SourceBot is an open-source AI-powered codebase assistant that integrates with Slack and Microsoft Teams. Developers ask natural language questions — "what does the authentication service do?" or "find all places where we call the payments API" — and SourceBot answers in the channel, pulling context from a semantic index of your git repository.

Running SourceBot self-hosted gives you full control over which codebase gets indexed and how LLM API costs are managed. But it also means you're responsible for keeping the API server up, the vector database fresh, the code index in sync with HEAD, and the Slack/Teams bot token valid. A single broken layer produces a worse outcome than downtime: SourceBot responds, but with stale or hallucinated answers.

Vigilmon monitors every layer and alerts you the moment any component drifts out of health.

What You'll Set Up

  • HTTP uptime monitor for the SourceBot API server (port 3000)
  • Vector database health probe (Chroma, Qdrant, or Weaviate)
  • Code indexing pipeline heartbeat
  • LLM provider connectivity check
  • Query response latency heartbeat
  • Git repository sync health check

Prerequisites

  • SourceBot running as a Node.js service on port 3000
  • Vector database deployed (Chroma, Qdrant, or Weaviate)
  • LLM provider configured (OpenAI, Anthropic Claude, or local Ollama)
  • Slack App or Teams Bot integration active
  • A free Vigilmon account

Step 1: Monitor the SourceBot API Server

The API server on port 3000 handles all incoming Slack and Teams events and serves code answers. If it goes down, every developer mention of your SourceBot in Slack returns nothing — or an error from the platform saying the bot is unreachable.

Add a health endpoint to your SourceBot service:

// health.js — add to your Express/Fastify app
app.get('/health', (req, res) => {
  res.json({
    status: 'ok',
    uptime: process.uptime(),
    indexedAt: global.lastIndexTimestamp || null,
  });
});

Then configure the monitor:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter https://sourcebot.yourdomain.com/health.
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

Alert on 2 consecutive failures — developers notice immediately when bot responses stop appearing.

Step 2: Monitor the Vector Database

SourceBot stores code embeddings in a vector database (Chroma, Qdrant, or Weaviate). A database outage means semantic search returns no results — the LLM has nothing to reason about and either hallucinate or refuse to answer.

Chroma

#!/bin/bash
# probe-chroma.sh
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8000/api/v1/heartbeat)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_CHROMA_HEARTBEAT_URL" > /dev/null

Qdrant

#!/bin/bash
# probe-qdrant.sh
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:6333/healthz)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_QDRANT_HEARTBEAT_URL" > /dev/null

Weaviate

#!/bin/bash
# probe-weaviate.sh
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/v1/.well-known/live)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_WEAVIATE_HEARTBEAT_URL" > /dev/null

Schedule every 2 minutes:

*/2 * * * * /opt/sourcebot/probe-vectordb.sh

Create a Vigilmon heartbeat monitor:

  1. Add Monitor → Heartbeat / Cron.
  2. Name: SourceBot Vector DB.
  3. Expected interval: 5 minutes.

Step 3: Track Code Index Freshness

SourceBot indexes your git repository for semantic search. If the indexer falls behind HEAD — due to a crashed indexer process, a git authentication failure, or a resource constraint — developers get answers based on stale code. This is the most insidious failure mode: SourceBot keeps responding, but the answers are wrong.

Instrument your indexer to push a heartbeat after each successful run:

// indexer.js
async function indexRepository() {
  try {
    await pullLatestCommits();
    await embedNewFiles();
    await updateVectorDatabase();

    // Mark success
    global.lastIndexTimestamp = Date.now();
    await axios.get(process.env.VIGILMON_INDEX_HEARTBEAT_URL);
  } catch (err) {
    logger.error('Indexing failed', err);
    // No heartbeat — Vigilmon will alert on missed window
  }
}

Configure the Vigilmon heartbeat:

  1. Add Monitor → Heartbeat / Cron.
  2. Name: SourceBot Index Pipeline.
  3. Expected interval: set to 2× your configured indexing frequency. If you index every 30 minutes, set 60 minutes.

When this heartbeat goes silent, the index is drifting from HEAD. Developers are receiving answers about code that no longer exists.

Step 4: Check LLM Provider Connectivity

SourceBot routes developer questions through an LLM for reasoning. Without a healthy LLM connection, SourceBot can find relevant code snippets via vector search but cannot generate natural-language answers.

OpenAI

#!/bin/bash
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  https://api.openai.com/v1/models)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_LLM_HEARTBEAT_URL"

Anthropic Claude

#!/bin/bash
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  https://api.anthropic.com/v1/models)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_LLM_HEARTBEAT_URL"

Local Ollama

Add a Vigilmon HTTP monitor directly:

  1. Add Monitor → HTTP / HTTPS.
  2. URL: http://localhost:11434/api/tags.
  3. Expected status: 200.
  4. Interval: 2 minutes.

Step 5: Monitor Slack and Teams API Connectivity

SourceBot posts answers back to Slack or Teams via bot token. An expired token, a revoked OAuth scope, or a Slack API outage means answers are generated but never delivered — developers see the bot thinking and then nothing.

Probe the Slack API health and token validity:

#!/bin/bash
# probe-slack.sh
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
  https://slack.com/api/auth.test)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_SLACK_HEARTBEAT_URL"

For Microsoft Teams, probe the Graph API:

STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer $TEAMS_TOKEN" \
  https://graph.microsoft.com/v1.0/me)
[ "$STATUS" = "200" ] && curl -s "$VIGILMON_TEAMS_HEARTBEAT_URL"

Schedule every 10 minutes:

*/10 * * * * /opt/sourcebot/probe-slack.sh

Create a 20-minute heartbeat window — two probe cycles before alerting.

Step 6: Track Query Response Latency

Developers expect a response within 30 seconds of asking SourceBot a question. Latency over 30 seconds disrupts flow — they switch context and forget they asked. Latency over 2 minutes means something is wrong with the embedding search, LLM call, or message delivery.

Instrument SourceBot to push a latency heartbeat only when queries complete within the SLA:

// On successful query response
const latencyMs = Date.now() - queryStartTime;
const latencySec = latencyMs / 1000;

if (latencySec < 30) {
  await axios.get(process.env.VIGILMON_LATENCY_HEARTBEAT_URL);
}
// No heartbeat if slow — Vigilmon alerts on missed window

Set the Vigilmon heartbeat expected interval to your team's typical query frequency during business hours. A missed window during a busy session signals P95 latency has exceeded 30 seconds.

Step 7: Monitor Git Repository Sync

SourceBot must pull the latest commits before indexing. A git credential expiry, SSH key rotation, or repository move can silently block all future indexing without crashing the process.

#!/bin/bash
# probe-git-sync.sh
cd /opt/sourcebot/repo && \
git fetch --dry-run origin main 2>/dev/null && \
curl -s "$VIGILMON_GIT_SYNC_HEARTBEAT_URL" > /dev/null

Schedule every 15 minutes:

*/15 * * * * /opt/sourcebot/probe-git-sync.sh

Create a 30-minute heartbeat. A missed window means git fetch is failing — the next indexing run will use stale code.

Alerting Configuration

| Monitor | Alert condition | Suggested channel | |---|---|---| | API server | Down 2+ minutes | Slack #platform / PagerDuty | | Vector database | Heartbeat missed 5 min | Slack #platform | | Index pipeline | Heartbeat missed | Slack #platform | | LLM provider | Heartbeat missed | Slack #platform | | Slack/Teams API | Heartbeat missed 20 min | Slack #platform | | Query latency | Heartbeat missed | Slack #platform | | Git sync | Heartbeat missed 30 min | Slack #platform |

The index pipeline heartbeat is critical: a silent index failure means SourceBot answers confidently with wrong information. Route it to the same channel as API downtime alerts.

Conclusion

SourceBot has seven independently-failable layers between a developer's Slack question and a correct code answer. Vigilmon monitors each one and alerts you before developers start noticing wrong answers or bot silence. With the full monitor set in place, you catch indexing drift, vector database outages, LLM provider failures, and platform API issues within minutes — not after a flood of confused developer messages.

Get started at vigilmon.online — the free plan covers the core monitors described in this tutorial.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles