tutorial

Monitoring Puppet with Vigilmon

Puppet drives your infrastructure configuration at scale — but a silent Puppet Server failure can leave agents unable to fetch catalogs. Here's how to monitor Puppet Server health, PuppetDB, SSL certificates, and agent run heartbeats with Vigilmon.

Puppet is the backbone of configuration management in thousands of enterprise environments — when it goes down, agents can't fetch catalogs, drifted infrastructure goes uncorrected, and the blast radius grows silently until someone notices. Vigilmon gives you continuous uptime checks on every Puppet component: the server API, PuppetDB, SSL certificates, and the heartbeat from your scheduled agent runs.

What You'll Set Up

  • HTTP uptime monitor for the Puppet Server web UI (port 8140)
  • JSON status check on the /status/v1/services health endpoint
  • PuppetDB health endpoint monitor
  • SSL certificate expiry alerts for Puppet's CA-signed certificates
  • Cron heartbeat for scheduled Puppet agent runs

Prerequisites

  • Puppet Server 7+ running and reachable over the network
  • PuppetDB installed and connected to Puppet Server
  • A free Vigilmon account

Step 1: Monitor the Puppet Server API Endpoint

Puppet Server exposes its API on port 8140 using mutual TLS by default. The /status/v1/services endpoint returns a JSON object describing the health of all server services and is accessible without a client certificate when client-auth is set to want or none in puppetserver.conf.

Add an HTTP monitor in Vigilmon:

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: https://puppet.example.com:8140/status/v1/services
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Under Response body must contain, enter "state":"running" to verify services are up.
  7. Click Save.

A healthy response looks like:

{
  "puppet-server": {
    "service_version": "7.12.0",
    "service_status_version": 1,
    "state": "running",
    "status": { "jruby-pool": { "allocated-instances": 4 } }
  },
  "puppetdb-status": {
    "state": "running"
  }
}

If state is error or the endpoint returns a non-200, Vigilmon fires an alert.

Step 2: Monitor the PuppetDB Health Endpoint

PuppetDB stores exported resources, facts, and catalog data. Its /pdb/meta/v1/version endpoint is a lightweight health check that returns the running version and confirms the service is up. PuppetDB listens on port 8081 by default.

  1. Click Add Monitor in Vigilmon.
  2. Set Type to HTTP / HTTPS.
  3. Enter the URL: https://puppet.example.com:8081/pdb/meta/v1/version
  4. Set Check interval to 2 minutes.
  5. Set Expected HTTP status to 200.
  6. Under Response body must contain, enter "version" to confirm the JSON payload is returned.
  7. Click Save.

For a deeper health check, also monitor the /pdb/query/v4 query API endpoint to ensure PuppetDB is processing queries:

https://puppet.example.com:8081/pdb/query/v4?query=["=","certname","puppet.example.com"]

Step 3: TCP Port Monitor for Puppet Server Availability

Before HTTP-layer checks can succeed, TCP connectivity to port 8140 must be working. Add a TCP monitor as a lower-level canary:

  1. Click Add Monitor → TCP Port.
  2. Set Host to puppet.example.com.
  3. Set Port to 8140.
  4. Set Check interval to 1 minute.
  5. Click Save.

Add a second TCP monitor for PuppetDB on port 8081:

  1. Click Add Monitor → TCP Port.
  2. Set Host to puppet.example.com.
  3. Set Port to 8081.
  4. Set Check interval to 1 minute.
  5. Click Save.

TCP monitors fire faster than HTTP monitors because they don't wait for an HTTP response — they're your first signal that the service is down.

Step 4: SSL Certificate Alerts for Puppet's CA Certificates

Puppet uses its own internal CA to sign all agent and server certificates. These certificates have a default TTL of 5 years, but intermediate CA certificates, external load balancer certificates, or manually issued certificates can expire unexpectedly.

Enable SSL monitoring on your existing Puppet Server HTTP monitor:

  1. Open the HTTP monitor for https://puppet.example.com:8140/status/v1/services.
  2. Scroll to the SSL Certificate section.
  3. Enable Monitor SSL certificate.
  4. Set Alert when certificate expires in less than 30 days.
  5. Click Save.

Add a second SSL check for PuppetDB's HTTPS endpoint:

  1. Open the PuppetDB HTTP monitor.
  2. Enable Monitor SSL certificate with a 30 days alert threshold.
  3. Click Save.

To check current certificate expiry dates on the Puppet Server:

# Check Puppet Server certificate
puppetserver ca list --all

# Check the CA certificate expiry directly
openssl x509 -in /etc/puppetlabs/puppet/ssl/certs/ca.pem -noout -dates

# Check the server's own certificate
openssl x509 -in /etc/puppetlabs/puppet/ssl/certs/puppet.example.com.pem -noout -dates

A 30-day alert window gives you time to rotate certificates before agents start refusing to connect.

Step 5: Heartbeat Monitoring for Scheduled Puppet Agent Runs

Puppet agents run on a 30-minute schedule by default (runinterval = 30m in puppet.conf). If an agent silently stops running — due to a process crash, systemd failure, or resource exhaustion — configuration drift accumulates undetected.

Set up a Vigilmon cron heartbeat for each critical agent:

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set Expected ping interval to 35 minutes (slightly longer than the 30-minute run interval to tolerate small delays).
  3. Name it after the node: puppet-agent-webserver01.
  4. Copy the heartbeat URL (e.g. https://vigilmon.online/heartbeat/abc123).

Add the heartbeat ping to your Puppet agent run. Create a Puppet exec resource or a post-run hook:

# /etc/puppetlabs/puppet/postrun_command.sh
#!/bin/bash
# This runs after every successful Puppet agent run
curl -fsS --retry 3 https://vigilmon.online/heartbeat/abc123

Register it in puppet.conf:

[agent]
runinterval = 30m
postrun_command = /etc/puppetlabs/puppet/postrun_command.sh

The postrun_command executes after a successful catalog apply. If the agent crashes before completing or fails to apply the catalog, the heartbeat is never sent — and Vigilmon alerts after 35 minutes of silence.

For systemd-based agent runs, you can also use a systemd service override:

# /etc/systemd/system/puppet.service.d/heartbeat.conf
[Service]
ExecStartPost=/usr/bin/curl -fsS https://vigilmon.online/heartbeat/abc123

Step 6: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add Slack, PagerDuty, or email.
  2. Set Consecutive failures before alert to 2 on HTTP monitors — Puppet Server may briefly return non-200 during a JRuby pool restart.
  3. Group Puppet Server + PuppetDB monitors under a single Puppet Infrastructure alert policy so on-call receives a single notification when the entire stack degrades.

For the heartbeat monitors, keep Consecutive failures before alert at 1 — a missed agent run is always worth investigating immediately.

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP health check | :8140/status/v1/services | Puppet Server JRuby crash, service error | | HTTP health check | :8081/pdb/meta/v1/version | PuppetDB down, query engine failure | | TCP port | :8140 | Network-level Puppet Server unavailability | | TCP port | :8081 | Network-level PuppetDB unavailability | | SSL certificate | Puppet Server cert | CA-signed cert expiry, renewal failure | | Cron heartbeat | Per-agent ping | Silent agent crash, missed 30-minute run |

Puppet manages your entire infrastructure footprint — it deserves the same rigorous monitoring you'd apply to any production service. With Vigilmon watching the Puppet Server API, PuppetDB, certificates, and agent heartbeats, you'll know within minutes when configuration management has silently stopped working.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles