How to Monitor Grafana Pyroscope with Vigilmon

Grafana Pyroscope is the open-source continuous profiling database that powers FlameGraph analysis at scale. It ingests profiling data from your applications — CPU, memory, goroutine, mutex — and stores it with high efficiency so you can query profiles across any time range without pre-aggregation.

Pyroscope is a critical component of modern observability stacks. When it goes down, profiling data is silently lost — there are no retries, no queue, no buffer. Your application instrumentation keeps pushing to /ingest, but nothing lands in the database. By the time someone notices that profiling data is missing, you've already lost hours of flamegraphs you needed for the incident you just had.

In this tutorial you'll set up external uptime monitoring for Grafana Pyroscope using Vigilmon — covering the ingestion endpoint, query frontend, and storage backend connectivity.

Why Pyroscope needs external monitoring

Pyroscope doesn't expose its own alerting for self-health. The built-in /ready and /healthz endpoints tell you the process is alive — but they don't tell you:

Ingestion backpressure — is the /ingest endpoint actually accepting data, or returning 503s that your SDK swallows silently?
Query frontend availability — can the Grafana datasource reach Pyroscope's query API, or are all profile queries timing out?
Storage backend connectivity — is Pyroscope connected to its object store (S3, GCS, Azure Blob)? A misconfigured IAM role or expired token will make Pyroscope appear healthy while writes silently fail
Component disaggregation — in microservices mode (distributor, ingester, querier running separately), each component can fail independently

External monitoring from Vigilmon catches all of these at the network boundary — before your on-call engineer opens Grafana and discovers that today's CPU flamegraphs simply don't exist.

What you'll need

A running Grafana Pyroscope instance (Docker, Kubernetes, or binary)
Pyroscope's HTTP port accessible from the internet (default: 4040)
A free Vigilmon account — no credit card required

Step 1: Verify your Pyroscope health endpoints

Pyroscope exposes several HTTP endpoints you can probe. Let's verify them first:

# Readiness probe — is Pyroscope ready to accept traffic?
curl -s http://pyroscope.example.com:4040/ready
# Expected: "ready"

# Liveness / health check
curl -s http://pyroscope.example.com:4040/healthz
# Expected: HTTP 200

# Component-level ring status (in microservices mode)
curl -s http://pyroscope.example.com:4040/ring
# Expected: HTML page listing distributor/ingester ring members

# Build info and version
curl -s http://pyroscope.example.com:4040/api/v1/status/buildinfo
# Expected: JSON with {"status":"success","data":{"version":"..."}}

For the ingestion endpoint, you can send a minimal test payload:

# Test that the ingest endpoint accepts data
curl -s -o /dev/null -w "%{http_code}" \
  -X POST "http://pyroscope.example.com:4040/ingest?name=test.cpu&from=$(date +%s)&until=$(date +%s)&format=pprof" \
  -H "Content-Type: application/octet-stream" \
  --data-binary ""
# Expected: 200 or 422 (invalid payload, but endpoint reachable)

The key signal is that the endpoint responds at all — a 4xx on an empty payload is fine, a 5xx or connection refused means the ingest path is broken.

Step 2: Monitor the readiness endpoint

The /ready endpoint is the most important endpoint to watch — Pyroscope returns non-200 here during startup, after losing storage connectivity, or during graceful shutdown.

Log in to vigilmon.online and go to Monitors → New Monitor
Choose HTTP / HTTPS
Set the URL: http://pyroscope.example.com:4040/ready
Set the check interval: 1 minute
Under Expected response:
- Status code: 200
- Response body contains: ready
Save the monitor

Vigilmon probes this from multiple geographic regions. If Pyroscope loses object storage connectivity (which doesn't affect /healthz but does affect /ready in many configurations), you'll get an alert within seconds.

Step 3: Monitor the query API

The query frontend is what Grafana actually talks to when rendering flamegraphs. Even if Pyroscope is ingesting data, a broken query path means your observability dashboards go dark.

Add a second monitor:

Monitors → New Monitor → HTTP / HTTPS
URL: http://pyroscope.example.com:4040/api/v1/status/buildinfo
Check interval: 1 minute
Expected response:
- Status code: 200
- Response body contains: "status":"success"
Save the monitor

This endpoint is served by the query frontend path, so if the query component is down (in microservices mode) or the query layer has a routing issue, this probe fails even when the ingestion path is healthy.

Step 4: Monitor the ingestion endpoint

Add a TCP monitor to verify the ingestion port is open, separate from the HTTP health check:

Monitors → New Monitor → TCP Port
Host: pyroscope.example.com
Port: 4040
Check interval: 1 minute
Save the monitor

TCP monitoring catches the case where the Pyroscope process has died completely (no port listener), which would otherwise only be detected after the /ready check times out.

For Kubernetes deployments, you can also add a monitor for each component separately:

| Monitor | URL | What it catches | |---------|-----|-----------------| | Distributor ready | http://distributor:4040/ready | Ingestion path failure | | Ingester ready | http://ingester:4040/ready | Profile storage failure | | Querier ready | http://querier:4040/ready | Query path failure | | Query frontend | http://query-frontend:4040/api/v1/status/buildinfo | Grafana datasource path |

Step 5: Configure alert channels

When Pyroscope ingestion goes down, every profiling push from your SDKs is silently dropped. You want to know immediately.

Email alerts

Go to Alert Channels → Add Channel → Email
Enter your on-call or infrastructure team address
Assign the channel to all Pyroscope monitors

Webhook alerts for Slack or PagerDuty

Alert Channels → Add Channel → Webhook
Enter your Slack incoming webhook URL or PagerDuty events URL
Assign the channel to your monitors

Vigilmon sends a payload like this when Pyroscope goes down:

{
  "monitor_name": "Pyroscope /ready",
  "status": "down",
  "url": "http://pyroscope.example.com:4040/ready",
  "started_at": "2026-01-15T14:23:00Z",
  "duration_seconds": 60
}

Recommended alert routing

For a production profiling cluster, route based on which component fails:

Distributor or ingest port down → page on-call immediately (profiling data is being lost right now)
Querier or query frontend down → alert the observability team (dashboards are broken but data is safe)
All components down → escalate to critical incident (Pyroscope cluster is fully unavailable)

Step 6: Set up a status page

If your team relies on Pyroscope for performance investigations, a status page prevents repeated "is Pyroscope down?" questions during incidents.

Go to Status Pages → New Status Page
Name it: "Observability Infrastructure"
Add your Pyroscope monitors in a group called "Continuous Profiling"
Optionally add your Prometheus, Grafana, and Loki monitors to the same page
Publish the page and share the URL in your team's runbook

What Pyroscope failures look like

Here are the most common failure modes and how Vigilmon surfaces them:

Object storage credentials expire — Pyroscope's /ready starts returning 503, but /healthz stays 200. Your HTTP monitor on /ready fires; the TCP monitor stays green. You know the process is alive but storage is broken.

Kubernetes pod OOMKilled — all monitors drop simultaneously (HTTP and TCP). Multi-region consensus confirms it's not a transient network issue. The alert fires within 1-2 minutes.

Ingester falls out of ring — distributor /ready stays green, but profiling data stops being written because no ingesters are available. Your querier /ready may also degrade. The component-level monitors fire for the ingester.

Query frontend port conflict — query API probes fail while ingest probes succeed. The split lets you diagnose the problem before opening Grafana.

Summary

Pyroscope is silent when it fails — your SDKs don't error, your apps don't crash, you just quietly lose profiling data. External monitoring with Vigilmon catches these failures at the network boundary, so you know before your next incident response when you discover there are no flamegraphs to look at.

Start monitoring Pyroscope for free at vigilmon.online — setup takes under two minutes.