How to Monitor Semaphore CI with Vigilmon

Semaphore CI delivers some of the fastest CI/CD build times in the industry, with per-second billing that rewards lean pipelines. But speed only matters when your pipelines are actually running — stalled queues, misconfigured secrets, or a failing deployment stage can silently block your team. Vigilmon adds external oversight to Semaphore: uptime checks on deployed services, cron heartbeats from pipeline jobs, and instant alerts when something stops working.

What You'll Set Up

• HTTP uptime monitors for services deployed through Semaphore pipelines
• Cron heartbeat monitors triggered at the end of scheduled Semaphore jobs
• Webhook alerts when pipeline health degrades
• Status-page check for Semaphore's own infrastructure

Prerequisites

• A Semaphore CI organization with at least one active project
• One or more services deployed via Semaphore pipelines
• A free Vigilmon account

Step 1: Monitor Services Deployed by Semaphore

Every pipeline that ships a deployment changes the live state of a service. Add a Vigilmon HTTP monitor for each production endpoint that Semaphore deploys to:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your production URL: https://api.yourapp.com/health.
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Click Save.

Use a dedicated /health endpoint rather than the root URL. A health route that checks database connectivity and downstream dependencies catches partial outages that a homepage check misses.

Step 2: Add a Health Endpoint to Your Application

If your app doesn't expose a health route yet, add one before wiring up Vigilmon. Here are minimal examples for common runtimes:

Node.js / Express

app.get('/health', (req, res) => {
  res.json({ status: 'ok', uptime: process.uptime(), timestamp: Date.now() });
});

Python / FastAPI

@app.get('/health')
def health():
    return {'status': 'ok'}

Go

http.HandleFunc('/health', func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.Write([]byte(`{"status":"ok"}`))
})

Deploy the updated app through your Semaphore pipeline so the health endpoint reaches production.

Step 3: Send a Heartbeat from Scheduled Semaphore Jobs

Semaphore lets you run pipelines on a cron schedule for tasks like database backups, report generation, or dependency audits. A Vigilmon cron heartbeat monitor detects when a scheduled job silently stops running.

Create a heartbeat monitor in Vigilmon:

1. Click Add Monitor → choose Cron Job / Heartbeat.
2. Enter a name like Semaphore nightly backup.
3. Set the expected interval to match your Semaphore schedule (e.g., 24 hours).
4. Copy the generated Heartbeat URL.

Ping the heartbeat at the end of your Semaphore job block:

# .semaphore/semaphore.yml
version: v1.0
name: Nightly backup
agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu2004

blocks:
  - name: Backup
    task:
      jobs:
        - name: Run backup
          commands:
            - ./scripts/backup.sh
            - curl -fsS --retry 3 "https://vigilmon.online/api/heartbeat/YOUR_HEARTBEAT_TOKEN" > /dev/null

Replace YOUR_HEARTBEAT_TOKEN with the token from the Vigilmon heartbeat monitor. The --retry 3 flag ensures transient network errors don't generate false alerts.

If the curl call is reached, the backup completed without fatal errors. If Semaphore fails before that point — agent crash, timeout, secret missing — Vigilmon never receives the ping and fires an alert after the grace period expires.

Step 4: Alert on Deployment Failures via Webhooks

Semaphore supports pipeline notifications. Combined with Vigilmon's webhook endpoint, you can route deployment failure events to your existing alert channel.

In Semaphore, create a notification rule under Organization → Notifications:

# .semaphore/notifications.yml
apiVersion: v1alpha
kind: Notification
metadata:
  name: vigilmon-alerts
spec:
  rules:
    - name: Deployment failures
      filter:
        projects:
          - your-project-name
        branches:
          - main
        pipelines:
          - .semaphore/deploy.yml
        results:
          - failed
          - stopped
      notify:
        webhook:
          endpoint: "https://vigilmon.online/api/webhook/YOUR_WEBHOOK_TOKEN"
          secret: "YOUR_WEBHOOK_SECRET"

Push this file to .semaphore/notifications.yml in your repo. Semaphore picks it up automatically and posts a payload whenever a deploy pipeline matches the filter.

Step 5: Monitor Semaphore's Own Status Page

If Semaphore's infrastructure is degraded, your pipelines slow down even when your code is fine. Add a Vigilmon monitor for Semaphore's status endpoint:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: https://semaphoreci.statuspage.io/api/v2/status.json
3. Set Expected response contains to "indicator":"none".
4. Interval: 5 minutes.

When Semaphore's own status degrades, you'll know before you start hunting for bugs in your .semaphore/ config.

Step 6: Set Up Alerting

Configure alerts so the right people are notified immediately:

1. Go to Monitors → [your monitor] → Alerts in Vigilmon.
2. Add channels — email, Slack, or webhook.
3. Set alert after to 1 failed check for production endpoints, 2 failed checks for staging.

For the heartbeat monitor, set the grace period to at least 15 minutes longer than your longest typical job runtime to avoid false alerts during slow runs.

Conclusion

Semaphore CI's speed advantage disappears when your deployed services are down or scheduled jobs silently fail. Vigilmon closes that visibility gap: HTTP monitors confirm each deployment lands successfully, heartbeat monitors prove scheduled jobs still run, and webhook notifications surface pipeline failures before they reach your users. The combination gives you CI/CD confidence that extends beyond the build log and into the live environment.

Start monitoring your Semaphore pipelines at vigilmon.online.