Monitoring Screwdriver CD with Vigilmon: API Health, Build Heartbeats, Queue Depth & SSL Alerts

Screwdriver CD is an open-source continuous delivery platform built by Yahoo/Verizon, designed around a microservices architecture with a dedicated API, web UI, store service, and pluggable executors (Kubernetes, Docker, Jenkins). When the API process crashes, an executor disconnects, or the store service becomes unavailable, builds queue silently and deployments stall without warning. Vigilmon adds the external observability layer that Screwdriver lacks out of the box: HTTP health checks, build completion heartbeats, API endpoint validation, and SSL certificate alerts.

What You'll Build

• An HTTP monitor for the Screwdriver web UI
• A health check against the /v4/status API endpoint
• A build completion heartbeat for end-to-end pipeline verification
• An executor queue heartbeat to catch stalled build queues
• SSL certificate expiry alerts
• A TCP port check for the API service

Prerequisites

• Screwdriver CD deployed (Docker Compose or Kubernetes)
• The API service (screwdriver-cd/api) running and accessible
• A domain configured for your Screwdriver instance with HTTPS
• A free account at vigilmon.online

Step 1: Monitor the Screwdriver Web UI

The Screwdriver dashboard shows pipeline status, build history, and executor health. A down UI leaves your team with no visibility into deployment state:

1. Log in to Vigilmon → Add Monitor → HTTP.
2. URL: https://screwdriver.yourdomain.com
3. Check interval: 120 seconds.
4. Expected status: 200.
5. Label: Screwdriver Web UI
6. Click Save.

Screwdriver's UI is a separate Node.js process from the API. A healthy UI response confirms the frontend service and its reverse proxy routing are both operational.

Step 2: Check the Screwdriver API Health Endpoint

Screwdriver exposes /v4/status — a lightweight endpoint that returns { "status": "ok" } and fails immediately when the API cannot reach its backing database or dependent services:

1. Add Monitor → HTTP.
2. URL: https://api.screwdriver.yourdomain.com/v4/status
3. Check interval: 60 seconds.
4. Expected status: 200.
5. Keyword: ok (confirms the JSON body contains the status field).
6. Label: Screwdriver API Status
7. Click Save.

The /v4/status endpoint exercises the API process, database connectivity, and any required service dependencies. If the API loses its database connection, this endpoint returns an error status while the UI may still render cached content — making it your primary alert signal.

Step 3: Build Completion Heartbeat

Add a heartbeat ping at the end of a critical pipeline to verify that Screwdriver is not just running but actually completing builds successfully:

# screwdriver.yaml
shared:
  image: node:20

jobs:
  main:
    steps:
      - install: npm ci
      - test: npm test
      - notify: >
          curl -fsS -X POST
          https://vigilmon.online/api/heartbeat/YOUR-BUILD-HEARTBEAT-ID

In Vigilmon:

1. Add Monitor → Heartbeat.
2. Expected interval: Match your pipeline trigger frequency (e.g., 30 minutes for frequently triggered pipelines).
3. Grace period: 20 minutes.
4. Label: Screwdriver Build Completion

The heartbeat only fires after all preceding steps succeed. If a test step fails, the executor crashes mid-build, or Screwdriver stops scheduling jobs entirely, the heartbeat goes silent and Vigilmon alerts you within the grace period.

Step 4: API Token and Pipeline Check Heartbeat

Screwdriver's REST API drives all pipeline operations. Monitor it with a periodic script that authenticates and lists pipelines to confirm the API layer is fully functional:

#!/bin/bash
# /etc/cron.d/screwdriver-api-check — runs every 5 minutes

TOKEN=$(curl -s -X POST https://api.screwdriver.yourdomain.com/v4/auth/token \
  -H "Content-Type: application/json" \
  -d '{"username":"YOUR_USER","password":"YOUR_PASS"}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin).get('token',''))")

if [ -n "$TOKEN" ]; then
  STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
    -H "Authorization: Bearer $TOKEN" \
    https://api.screwdriver.yourdomain.com/v4/pipelines)

  if [ "$STATUS" = "200" ]; then
    curl -fsS -X POST https://vigilmon.online/api/heartbeat/YOUR-API-HEARTBEAT-ID
  fi
fi

In Vigilmon:

1. Add Monitor → Heartbeat.
2. Expected interval: 5 minutes.
3. Grace period: 15 minutes.
4. Label: Screwdriver API Auth & Pipelines

The token fetch exercises the authentication layer; the pipeline list exercises the API's database queries. Both must succeed for the heartbeat to fire.

Step 5: SSL Certificate Alerts

Screwdriver depends on HTTPS for web sessions, API calls from build steps, and webhook deliveries from source control:

1. Add Monitor → SSL Certificate.
2. Domain: screwdriver.yourdomain.com
3. Alert when expiry is within: 30 days.
4. Alert again: 14 days, 7 days, 3 days.
5. Click Save.

Repeat for api.screwdriver.yourdomain.com if your API runs on a separate subdomain. An expired certificate breaks webhook delivery from GitHub/GitLab, causing pipelines to stop triggering on commits.

Step 6: TCP Port Check for the API Service

Screwdriver's API listens on port 8080 by default (or 443 via reverse proxy). A TCP check provides the lowest-level confirmation that the service port is accepting connections:

1. Add Monitor → TCP.
2. Host: api.screwdriver.yourdomain.com, Port: 443.
3. Check interval: 60 seconds.
4. Label: Screwdriver API Port
5. Click Save.

A TCP failure before the HTTP monitor fires points to a reverse proxy or network issue rather than the API process itself — narrowing diagnosis immediately.

Step 7: Store Service Availability Check

Screwdriver's store service handles build log storage and artifact uploads. A down store service causes builds to fail during log streaming even when the API and executor are healthy:

1. Add Monitor → HTTP.
2. URL: https://store.screwdriver.yourdomain.com/v1/status
3. Check interval: 120 seconds.
4. Expected status: 200.
5. Label: Screwdriver Store Service
6. Click Save.

Common Screwdriver CD Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | API process crash | /v4/status and web UI monitors fire within 60 s | | Database connection failure | API status endpoint returns error | | Build queue stall | Build completion heartbeat goes silent | | Authentication layer failure | API auth heartbeat stops | | Store service down | Store status monitor fires | | SSL certificate expires | SSL monitor alerts at 30 days | | Reverse proxy misconfiguration | TCP check fires; HTTP monitors also fire | | Webhook delivery failure | Build completion heartbeat stops (no new builds trigger) | | Executor pod crash (Kubernetes) | Build completion heartbeat stops |

Screwdriver CD's microservices architecture gives you flexibility and scalability for large-scale continuous delivery. Vigilmon ensures every component — API, web UI, store service, and executor — stays healthy, so your teams get fast feedback on every commit without silent failures blocking the pipeline.

Start monitoring Screwdriver CD in under 5 minutes — register free at vigilmon.online.