tutorial

Monitoring Rekor with Vigilmon: Sigstore Transparency Log Health, API Uptime, and Supply Chain Integrity Alerts

How to monitor Rekor sigstore transparency log with Vigilmon — log API health, tree head consistency, checkpoint endpoints, and supply chain security alert coverage.

Rekor is the Sigstore transparency log — a tamper-evident, append-only ledger that records cryptographic signatures, attestations, and software artifacts. When you run cosign sign or use any Sigstore-compatible tool, it writes an immutable entry to Rekor. When your code, CI pipeline, or admission controller calls cosign verify, it reads from Rekor to confirm the artifact was signed and the entry is consistent with the public log. If Rekor is unavailable, Sigstore-based signing workflows stall and verification calls fail. If your self-hosted Rekor instance goes down, every policy enforcement point that depends on it starts rejecting good artifacts or, worse, falls back to permissive mode. Vigilmon gives you external uptime monitoring of Rekor's API, health endpoints, and the services that depend on a healthy transparency log.

What You'll Build

  • A monitor on Rekor's /api/v1/log endpoint for transparency log health
  • A monitor on the /api/v1/log/proof endpoint for consistency proof availability
  • An SSL certificate monitor for your Rekor domain
  • Monitors for Rekor's supporting services (Trillian, Redis, MySQL/PostgreSQL)
  • An alerting setup that fires before Rekor unavailability breaks signing or verification workflows

Prerequisites

  • A running Rekor instance (self-hosted) or access to the public Rekor at rekor.sigstore.dev
  • A free account at vigilmon.online

Step 1: Verify Rekor's API Health

Rekor exposes a REST API that returns the current state of the transparency log. The primary health signal is the /api/v1/log endpoint, which returns the current tree head:

curl https://rekor.example.com/api/v1/log

A healthy Rekor instance returns the current signed tree head:

{
  "rootHash": "sha256:...",
  "signedTreeHead": "...",
  "treeID": "1193050959916422184",
  "treeSize": 87654321,
  "inactiveShards": []
}

The treeSize field is the total number of log entries — a growing number confirms the log is accepting new entries. The signedTreeHead is the Trillian-signed root that clients use for consistency verification.

# Also check the ping endpoint for a quick liveness check
curl https://rekor.example.com/api/v1/log/publicKey

Step 2: Create a Vigilmon Monitor for the Rekor Log API

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://rekor.example.com/api/v1/log.
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: treeSize (confirms the log API is returning a valid tree head).
  7. Label: Rekor log API.
  8. Click Save.

This monitor catches:

  • Rekor process crashes or container restarts
  • Trillian backend failures that prevent the log from serving the tree head
  • Database connectivity failures (Rekor can't serve a signed tree head without MySQL/PostgreSQL)
  • Deployment failures after Rekor upgrades

Alert sensitivity: Set alerts after 1 consecutive failure. Any Rekor API failure means cosign sign, cosign verify, and any Sigstore-based supply chain tools fail within seconds.

Step 3: Monitor the Rekor Public Key Endpoint

Rekor's public key endpoint serves the log's signing key — used by clients to verify the signed tree head. If this endpoint is unavailable, client verification calls fail even if the log itself is healthy:

curl https://rekor.example.com/api/v1/log/publicKey

A healthy Rekor returns a PEM-encoded public key:

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE...
-----END PUBLIC KEY-----
  1. Add Monitor → HTTP.
  2. URL: https://rekor.example.com/api/v1/log/publicKey.
  3. Check interval: 2 minutes.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: BEGIN PUBLIC KEY.
  7. Label: Rekor public key endpoint.
  8. Click Save.

Step 4: Monitor Log Consistency with the Proof Endpoint

Rekor provides inclusion and consistency proofs that clients use to verify entries haven't been tampered with. The proof API is a critical path for verification-heavy workflows:

# Request a consistency proof between two tree sizes
curl "https://rekor.example.com/api/v1/log/proof?firstSize=1&lastSize=100"

A healthy response returns the Merkle proof hashes confirming the log is consistent.

  1. Add Monitor → HTTP.
  2. URL: https://rekor.example.com/api/v1/log/proof?firstSize=1&lastSize=2.
  3. Check interval: 5 minutes.
  4. Response timeout: 20 seconds.
  5. Expected status: 200.
  6. Keyword: hashes.
  7. Label: Rekor consistency proof API.
  8. Click Save.

Why monitor the proof endpoint separately? The log API can return the current tree head even when Trillian's proof generation is degraded. Monitoring the proof endpoint catches partial failures where entries can be read but not verified — a dangerous state for enforcement workflows.

Step 5: Monitor Rekor's Supporting Services

A self-hosted Rekor stack depends on several backend services. Each has its own health endpoint worth monitoring:

Trillian Log Server

Trillian is the underlying Merkle tree engine. Without it, Rekor can't append or serve entries:

  1. Add Monitor → HTTP.
  2. URL: http://trillian-log-server.example.com:8090/healthz (adjust for your deployment).
  3. Check interval: 60 seconds.
  4. Expected status: 200.
  5. Keyword: OK.
  6. Label: Trillian log server (Rekor backend).
  7. Click Save.

Trillian Signer

The Trillian signer is the component that creates signed tree heads. Without it, the tree head goes stale and clients start detecting consistency failures:

  1. Add Monitor → HTTP.
  2. URL: http://trillian-signer.example.com:8091/healthz.
  3. Check interval: 60 seconds.
  4. Expected status: 200.
  5. Keyword: OK.
  6. Label: Trillian signer (Rekor tree head).
  7. Click Save.

Step 6: Monitor SSL Certificates

Rekor clients and CI tools connect over TLS. An expired or invalid certificate causes all Sigstore tooling to reject connections with cryptic errors:

# Check your Rekor instance's certificate expiry
openssl s_client -connect rekor.example.com:443 -servername rekor.example.com 2>/dev/null \
  | openssl x509 -noout -dates
  1. Add Monitor → SSL Certificate.
  2. Domain: rekor.example.com.
  3. Alert when expiry is within: 30 days.
  4. Alert again: 14 days, 7 days, 3 days.
  5. Click Save.

Public Rekor: If you use rekor.sigstore.dev from the public Sigstore infrastructure, you don't control the certificate — monitor your own Rekor proxy or your CI/CD pipeline's ability to reach Rekor instead.

Step 7: Monitor Rekor Entry Ingestion (via Webhook)

To verify that Rekor is actually accepting new entries (not just serving reads), you can monitor a custom health endpoint that attempts an entry lookup using a known log index:

# Retrieve a specific log entry to confirm reads work
curl https://rekor.example.com/api/v1/log/entries?logIndex=0
  1. Add Monitor → HTTP.
  2. URL: https://rekor.example.com/api/v1/log/entries?logIndex=0.
  3. Check interval: 5 minutes.
  4. Expected status: 200.
  5. Keyword: body (the log entry body is always in the response).
  6. Label: Rekor entry read (logIndex=0).
  7. Click Save.

Step 8: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | Rekor log API | Non-200 or treeSize missing | Rekor process down; all cosign sign/verify operations fail | | Public key endpoint | Non-200 or BEGIN PUBLIC KEY missing | Client verification fails; check Rekor key configuration | | Consistency proof API | Non-200 or hashes missing | Trillian proof generation degraded; verification may fail | | Trillian log server | Non-200 | Rekor backend down; new entries cannot be appended | | Trillian signer | Non-200 | Tree head goes stale; clients detect staleness and fail | | SSL certificate | < 30 days to expiry | Renew cert; all Sigstore tooling rejects TLS connections | | Entry read check | Non-200 | Read path broken; verification workflows fail |

Alert channels: Configure PagerDuty or a high-priority Slack channel for the Rekor log API and Trillian signer monitors — these are on the critical path for any supply chain enforcement policy.

Common Rekor Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Rekor process crash | Log API monitor fires within 60 s; all signing/verification stalls | | Trillian backend down | Log API returns error; Trillian health monitor fires | | Database connection pool exhausted | Rekor API starts returning 500; log API monitor fires | | Tree head goes stale (signer crash) | Proof API fails consistency checks; clients detect divergence | | SSL certificate expires | SSL monitor fires at 30 days; all Sigstore clients reject TLS | | Rekor deployment upgrade fails | Log API monitor catches startup failures | | Network partition to transparency log | Log API unreachable; cosign verify calls time out in CI pipelines | | Public key rotation without client update | Client verification fails; public key monitor may catch 404 | | Disk full on Trillian storage | Log API returns 500 on new entry attempts; read-path still works |

Rekor is the audit trail that makes Sigstore's trust model work — it's the log that lets you prove a signature existed before a key was compromised and catch supply chain tampering by detecting inconsistency. When Rekor is unavailable, either your CI pipelines queue up unsigned artifacts or your admission controllers start rejecting legitimate images. Vigilmon gives you external, continuous monitoring of Rekor's API, proof endpoints, supporting services, and SSL certificates so you can respond to infrastructure problems before they become supply chain security gaps.

Start monitoring your Rekor instance in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles