How to Monitor the OpenTelemetry Collector with Vigilmon

The OpenTelemetry Collector is the glue that holds modern observability pipelines together. It receives traces, metrics, and logs from your applications, processes them — batching, filtering, enriching — and exports them to your backends: Jaeger, Prometheus, Datadog, Grafana Cloud, or anywhere else.

Because the Collector sits in the middle of every telemetry flow, it's a single point of failure that most teams never monitor. When the Collector goes down, it does so quietly: your applications keep exporting data, but the exports return errors that most SDKs buffer and eventually drop. Your tracing dashboards go stale, your metrics stop updating, your log pipelines dry up — and you often don't notice until an incident makes you realize you have no observability data for the past hour.

In this tutorial you'll set up external uptime monitoring for the OpenTelemetry Collector using Vigilmon, covering the health check extension, zpages interface, and pipeline availability.

Why the OTel Collector needs external monitoring

The Collector has no built-in alerting for its own health. It exposes several self-diagnostic endpoints, but they require you to actively poll them. External monitoring with Vigilmon does this automatically, from multiple geographic regions, and alerts you the moment something breaks.

The failure modes that external monitoring catches:

• Collector process dies — your applications start buffering exports in-memory; buffer fills up; telemetry is dropped. Vigilmon catches this within one check interval.
• Exporter backend unreachable — the Collector is running but can't reach Jaeger or Prometheus. It starts dropping spans after the retry budget is exhausted. The health check extension can reflect this.
• Pipeline configuration error — a config reload with a typo crashes the Collector. External monitoring detects the process going down.
• Memory pressure and OOM — the Collector's memory_limiter processor starts dropping data before the process OOMs. Catching the Collector going down quickly lets you act before the memory limiter has been dropping data for an extended period.
• Resource exhaustion — the Collector's gRPC port becomes overloaded and stops accepting new connections. TCP monitoring catches this before users report missing traces.

What you'll need

• A running OpenTelemetry Collector (any distribution: core, contrib, otelcol-builder)
• The health_check extension enabled in your Collector config
• The Collector's HTTP ports accessible (default health check: 13133)
• A free Vigilmon account

Step 1: Enable the health_check extension

The health_check extension is built into the Collector contrib distribution and exposes an HTTP endpoint reporting pipeline health. Add it to your configuration:

# otel-collector-config.yaml
extensions:
  health_check:
    endpoint: 0.0.0.0:13133
    path: /health/status
    check_collector_pipeline:
      enabled: true
      interval: 5m
      exporter_failure_threshold: 5

  zpages:
    endpoint: 0.0.0.0:55679

service:
  extensions: [health_check, zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, memory_limiter]
      exporters: [jaeger, debug]
    metrics:
      receivers: [otlp, prometheus]
      processors: [batch]
      exporters: [prometheusremotewrite]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [loki]

Restart the Collector and verify:

curl -s http://localhost:13133/health/status
# Expected: {"status":"Server available","upSince":"2026-01-15T10:00:00.000Z","uptime":"2h15m30s"}

# If pipeline checking is enabled and exporters are failing:
# {"status":"Server not available","statusErr":"...","upSince":"..."}

The zpages interface provides real-time pipeline statistics:

# View pipeline throughput (spans per second, dropped spans, etc.)
curl -s http://localhost:55679/debug/tracez | head -50

# View running spans
curl -s http://localhost:55679/debug/rpcz

Step 2: Monitor the health check endpoint

This is your primary liveness signal — the endpoint reflects whether the Collector considers itself healthy and whether its configured pipelines are functioning.

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS
3. Set the URL: http://otel-collector.example.com:13133/health/status
4. Check interval: 1 minute
5. Expected response:
   • Status code: 200
   • Response body contains: "status":"Server available"
6. Save the monitor

With check_collector_pipeline enabled, the Collector will return a non-200 status when exporters are failing consistently — so this monitor catches both "process is down" and "exporters are broken."

Step 3: Monitor the zpages interface

The zpages endpoint is a secondary signal. If this port is down while the health check port is up, you have a partial process failure (unusual but it happens with port binding conflicts).

1. Monitors → New Monitor → HTTP / HTTPS
2. URL: http://otel-collector.example.com:55679/debug/tracez
3. Check interval: 2 minutes (zpages is less critical than health_check)
4. Expected response:
   • Status code: 200
5. Save the monitor

Step 4: Monitor the OTLP receiver ports

Your applications send telemetry to the Collector's OTLP receiver. If this port goes down, your instrumentation starts buffering and eventually dropping data. Add TCP monitors for both gRPC and HTTP endpoints:

OTLP gRPC (default port 4317):

1. Monitors → New Monitor → TCP Port
2. Host: otel-collector.example.com
3. Port: 4317
4. Check interval: 1 minute
5. Save the monitor

OTLP HTTP (default port 4318):

1. Monitors → New Monitor → TCP Port
2. Host: otel-collector.example.com
3. Port: 4318
4. Check interval: 1 minute
5. Save the monitor

These TCP monitors tell you whether the receiver ports are accepting connections at all, independent of whether the Collector considers itself healthy.

Step 5: Multi-Collector deployments

In production, you typically run the Collector in one of two topologies:

Agent + Gateway — a lightweight agent Collector runs on each node and forwards to a central gateway Collector. Monitor both tiers:

| Monitor | URL | What it catches | |---------|-----|-----------------| | Gateway health | http://gateway:13133/health/status | Central processing failure | | Gateway OTLP gRPC | gateway:4317 (TCP) | Agent-to-gateway routing failure | | Agent health (sample) | http://node-1:13133/health/status | Per-node collection failure |

Sidecar — a Collector sidecar runs alongside each application pod. Monitor the service fronting the sidecars:

# Kubernetes: expose a Service for Collector health checks
kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: otel-collector-health
  namespace: observability
spec:
  selector:
    app: otel-collector
  ports:
    - name: health
      port: 13133
      targetPort: 13133
EOF

Then monitor http://otel-collector-health.observability.svc:13133/health/status as a representative health check for the fleet.

Step 6: Configure alert channels

When the Collector goes down, telemetry data starts being lost immediately. Alert routing should reflect the urgency.

Email alerts

1. Alert Channels → Add Channel → Email
2. Add your on-call infrastructure address
3. Assign to all OTel Collector monitors

Webhook alerts

1. Alert Channels → Add Channel → Webhook
2. Configure your Slack or PagerDuty webhook URL
3. Assign to monitors

Sample Vigilmon alert payload:

{
  "monitor_name": "OTel Collector health_check",
  "status": "down",
  "url": "http://otel-collector.example.com:13133/health/status",
  "started_at": "2026-01-15T14:23:00Z",
  "duration_seconds": 120
}

Recommended alert severity levels

• Health check endpoint returns non-200 → P2 alert (exporters failing, data may be dropping)
• Health check endpoint unreachable → P1 alert (Collector process is down, data is definitely being lost)
• OTLP receiver port down → P1 alert (applications can't deliver telemetry at all)

Step 7: Correlating Vigilmon alerts with Collector logs

When you get a Vigilmon alert, the Collector's own logs are your next stop:

# Docker deployment
docker logs otel-collector --tail 100 --since 5m

# Kubernetes deployment
kubectl logs -n observability -l app=otel-collector --tail 100

# Look for exporter errors
kubectl logs -n observability -l app=otel-collector | grep -E 'error|failed|dropped'

Common log patterns after a Vigilmon alert:

# Exporter can't reach backend
error   exporterhelper/queued_retry.go   Exporting failed. Will retry.
{"kind": "exporter", "name": "jaeger", "error": "connection refused"}

# Memory limiter dropping data
warn    memorylimiterprocessor/memorylimiter.go   Data dropped due to high memory usage.

# Pipeline shutdown
info    service/service.go   Stopping pipeline    {"name": "traces"}

What the monitoring matrix looks like

Here's a complete monitoring setup for a production OTel Collector deployment:

| Monitor | Type | Alert on | Priority | |---------|------|----------|----------| | Health check endpoint | HTTP | Non-200 or "Server not available" | P2 | | Health check port | TCP | Port unreachable | P1 | | OTLP gRPC receiver | TCP | Port unreachable | P1 | | OTLP HTTP receiver | TCP | Port unreachable | P1 | | zpages interface | HTTP | Non-200 | P3 |

Summary

The OpenTelemetry Collector is invisible when it's working and invisible when it's broken — until you notice that your traces, metrics, and logs stopped arriving an hour ago. External monitoring with Vigilmon makes the Collector visible: you get alerted the moment it goes down, before your observability stack goes dark.

Set up OTel Collector monitoring for free at vigilmon.online — the first monitor is running in under two minutes.