Hubzilla Monitoring with Vigilmon: Web Server, PHP-FPM, MySQL, Zot6 Federation & Poller Health

Your Hubzilla hub had been running for two months without incident. Channels were active, posts were flowing to Mastodon and other Hubzilla hubs via Zot6, and nomadic identity clones were syncing. Nobody noticed when the background poller — the cron-driven federation crawler that delivers content between hubs — stopped running after a PHP-FPM restart misconfiguration. Federation to remote Hubzilla hubs froze while the web interface continued to work perfectly. Nomadic clones went out of sync. Posts from followed channels on remote hubs stopped arriving.

Hubzilla is an open source distributed social network and content publishing platform with powerful federation via its native Zot6 protocol and ActivityPub compatibility. Built with PHP and Apache (or Nginx), it supports nomadic identity — a unique feature allowing a user's channel to be cloned and operated simultaneously across multiple hubs. Keeping Hubzilla healthy means keeping Apache, PHP-FPM, MySQL/MariaDB, the background poller, file storage, and federation endpoints all running. Vigilmon gives you external uptime monitoring and alerting to catch failures before your users do.

What You'll Set Up

• HTTP uptime monitor for the Hubzilla web interface
• PHP-FPM process health check
• MySQL/MariaDB database connectivity check
• Zot6 federation channel connectivity monitor
• ActivityPub inbox and outbox endpoint health checks
• Heartbeat monitor for the background poller job (cron-driven federation crawl)
• File and media storage service availability check
• Nomadic identity endpoint health monitor
• OAuth application endpoint health check
• Scheduled cleanup job execution heartbeat
• SSL certificate expiry alerts

Prerequisites

• A running Hubzilla hub on port 80/443 with Apache or Nginx
• PHP-FPM running with its status and ping endpoints enabled
• MySQL or MariaDB accessible
• A free Vigilmon account

Step 1: Monitor the Hubzilla Web Interface

The Hubzilla homepage confirms that Apache is serving requests, PHP-FPM is processing PHP, and MySQL is returning channel directory data.

Test it:

curl -I https://hubzilla.yourdomain.com/

You should see 200 OK with Content-Type: text/html.

Set up the Vigilmon monitor:

1. Log in to Vigilmon and click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/.
3. Set Check interval to 60 seconds.
4. Set Expected status code to 200.
5. Under Advanced → Keyword check, add keyword present: Hubzilla.
6. Save the monitor.

Step 2: Monitor PHP-FPM Process Pool Health

PHP-FPM manages the pool of worker processes that handle all Hubzilla PHP requests. Under federation load — incoming Zot6 deliveries, ActivityPub inbox processing, or a channel crawl burst — the process pool can exhaust itself. When max children are reached, Apache returns 502 errors while the site appears technically alive.

Enable the PHP-FPM ping endpoint in your pool configuration:

; /etc/php/8.x/fpm/pool.d/www.conf
ping.path = /php-fpm-ping
ping.response = pong
pm.status_path = /php-fpm-status

Expose the ping path through Apache:

<Location "/php-fpm-ping">
    ProxyPass "unix:/run/php/php8.2-fpm.sock|fcgi://localhost/php-fpm-ping"
    Require ip 127.0.0.1
</Location>

Or create a lightweight PHP health endpoint at /health.php in your Hubzilla webroot:

<?php
// /var/www/hubzilla/health.php
// Restrict via Apache config to monitoring IPs only
http_response_code(200);
echo json_encode(['status' => 'ok', 'php' => PHP_VERSION]);

Then expose through Apache with IP restrictions:

<Location "/health.php">
    Require ip 127.0.0.1 YOUR_VIGILMON_CHECK_IP
</Location>

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/health.php.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: ok.
5. Save.

Step 3: Monitor MySQL/MariaDB Database Connectivity

Hubzilla stores channels, posts, federation state, Zot6 keys, contact records, OAuth tokens, and application data in MySQL or MariaDB. A database failure causes immediate application errors on every page — Hubzilla's core framework will return a database connection error page.

Add a database health check to your Hubzilla PHP health endpoint:

<?php
// /var/www/hubzilla/health-db.php
// Restrict to monitoring IPs in Apache config
require_once 'boot.php';
require_once 'include/dba.php';

try {
    $r = q("SELECT 1");
    if ($r) {
        http_response_code(200);
        echo json_encode(['status' => 'ok']);
    } else {
        http_response_code(503);
        echo json_encode(['status' => 'error']);
    }
} catch (Exception $e) {
    http_response_code(503);
    echo json_encode(['status' => 'error']);
}

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/health-db.php.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: ok.
5. Save.

When this monitor fails while the homepage passes, the issue is MySQL/MariaDB connectivity or a pool exhaustion.

Step 4: Monitor Zot6 Federation Channel Connectivity

Hubzilla's native Zot6 protocol enables nomadic identity and rich federation between Hubzilla hubs. The Zot6 channel discovery endpoint — /.well-known/zot-info — is how remote hubs locate and verify channel actors, exchange public keys, and establish federation trust.

Test the Zot6 channel info endpoint:

curl "https://hubzilla.yourdomain.com/.well-known/zot-info?address=yourchannel@hubzilla.yourdomain.com"

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/.well-known/zot-info?address=admin@hubzilla.yourdomain.com.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: id.
5. Save.

Also monitor the Hubzilla API endpoint for hub-level health:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/api/z/1.0/siteinfo.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: url.
5. Save.

Step 5: Monitor ActivityPub Inbox and Outbox Endpoints

In addition to Zot6, Hubzilla implements ActivityPub for compatibility with Mastodon, Misskey, Pleroma, and other Fediverse platforms. The ActivityPub inbox endpoint is where remote servers deliver posts, boosts, and interactions.

Check the WebFinger endpoint — Mastodon and other ActivityPub servers use this to look up your channels:

curl "https://hubzilla.yourdomain.com/.well-known/webfinger?resource=acct:admin@hubzilla.yourdomain.com"

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/.well-known/webfinger?resource=acct:admin@hubzilla.yourdomain.com.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: subject.
5. Save.

Check the NodeInfo endpoint for federation capability advertisement:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/.well-known/nodeinfo.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: links.
5. Save.

Step 6: Heartbeat Monitor for the Background Poller (Cron-Driven Federation Crawl)

The Hubzilla background poller (Zotlabs\Daemon\Master) runs via cron and performs all background federation work: delivering outgoing posts to followed channels, processing incoming deliveries from remote hubs, updating remote contact information, and maintaining federation state. When the cron job stops — due to a PHP error, cron misconfiguration, or a lock file issue — all federation halts while the web interface continues to work normally.

This is the most important monitor for a Hubzilla hub, and entirely invisible without explicit monitoring.

Step 6a: Create a Vigilmon heartbeat monitor.

1. Click New Monitor → Heartbeat in Vigilmon.
2. Set Name to Hubzilla Background Poller.
3. Set Expected heartbeat interval to 15 minutes.
4. Save. Copy the Heartbeat URL shown.

Step 6b: Add the Vigilmon ping to your Hubzilla cron job.

The standard Hubzilla cron entry calls the poller every 10 minutes:

# /etc/cron.d/hubzilla
*/10 * * * * www-data cd /var/www/hubzilla && php Zotlabs/Daemon/Master.php cron

Modify it to ping Vigilmon on success:

*/10 * * * * www-data cd /var/www/hubzilla && \
  php Zotlabs/Daemon/Master.php cron && \
  curl -fsS https://vigilmon.online/heartbeat/YOUR_TOKEN

If the poller exits with an error, the && short-circuit prevents the ping. Vigilmon alerts you after 15 minutes without a heartbeat — well before users notice that federation has stalled.

Step 7: Monitor File and Media Storage Service

Hubzilla stores uploaded files, photos, and attachments in its local storage directory (typically store/ in the Hubzilla webroot). Storage failures cause media uploads to fail, attached files to return 404 errors, and profile photos to stop loading.

Check a known static asset or the Hubzilla media serving endpoint:

curl -I https://hubzilla.yourdomain.com/images/hz-logo.png

For an availability signal, check whether the store directory is accessible via a known uploaded file, or create a sentinel file:

curl -I https://hubzilla.yourdomain.com/photo/KNOWN_PHOTO_HASH

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/images/hz-logo.png.
3. Set Expected status code to 200.
4. Save.

Monitor available disk space for the store directory with a sidecar cron health check:

# /etc/cron.d/hubzilla-disk-heartbeat
*/30 * * * * www-data df /var/www/hubzilla/store | awk 'NR==2{exit ($5+0 > 90)}' && \
  curl -fsS https://vigilmon.online/heartbeat/YOUR_DISK_TOKEN

Step 8: Monitor the Nomadic Identity Endpoint

Hubzilla's unique nomadic identity feature allows channels to clone themselves across multiple hubs. The nomadic identity endpoint — /.well-known/zot-info with a guid_hash parameter — is how hub-to-hub channel sync and clone verification works. If this endpoint is slow or returning errors, channel clones will fail to sync and nomadic identity operations will break.

Monitor the nomadic identity discovery endpoint:

curl "https://hubzilla.yourdomain.com/.well-known/zot-info"

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/.well-known/zot-info.
3. Set Expected status code to 200 or 400 (returns 400 without a valid address — still confirms the endpoint is live).
4. Under Advanced → Response time alert, set a threshold of 3000 ms.
5. Save.

A slow nomadic identity endpoint response (over 3 seconds) typically indicates database query performance issues on the channel lookup — worth investigating before it causes sync failures.

Step 9: Monitor OAuth Application Endpoint Health

Hubzilla supports OAuth2 for third-party application authentication and API access. The OAuth authorization endpoint being slow or unavailable affects users accessing Hubzilla through connected apps and clients.

Test the OAuth endpoint:

curl -I https://hubzilla.yourdomain.com/oauth/authorize

You should see a redirect or login page response (302 or 200), not a 500 error.

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://hubzilla.yourdomain.com/oauth/authorize.
3. Set Expected status code to 200 or 302.
4. Save.

Step 10: Heartbeat Monitor for Scheduled Cleanup Job Execution

In addition to the federation poller, Hubzilla runs periodic cleanup tasks — expiring old sessions, purging deleted content, cleaning up stale federation records, and rotating logs. These are typically handled by additional cron entries or by the poller's own cleanup cycles.

Create a separate heartbeat for weekly maintenance job confirmation:

1. Click New Monitor → Heartbeat in Vigilmon.
2. Set Name to Hubzilla Scheduled Cleanup Jobs.
3. Set Expected heartbeat interval to 2 hours.
4. Save. Copy the heartbeat URL.

Add a cron entry for the Hubzilla maintenance runner:

0 */2 * * * www-data cd /var/www/hubzilla && \
  php Zotlabs/Daemon/Master.php expire && \
  curl -fsS https://vigilmon.online/heartbeat/YOUR_CLEANUP_TOKEN

Step 11: SSL Certificate Expiry Alerts

Hubzilla requires HTTPS for Zot6 and ActivityPub federation. A lapsed certificate causes remote Hubzilla hubs to reject channel sync requests and Mastodon servers to reject ActivityPub deliveries.

1. Open the web UI monitor from Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Save.

Alerting Configuration

Set up notification channels so you're alerted immediately on failures:

1. In Vigilmon, go to Alerts → Notification Channels.
2. Add an Email channel with your admin address.
3. Optionally add a Slack webhook for your ops channel.
4. Assign all Hubzilla monitors to these notification channels.

Recommended alert thresholds:

| Monitor | Alert after | |---|---| | Web UI | 2 consecutive failures (2 min) | | PHP-FPM health | 2 consecutive failures | | MySQL/MariaDB health | 1 failure (immediate) | | Zot6 channel info | 2 consecutive failures | | ActivityPub/WebFinger | 3 consecutive failures | | Background poller heartbeat | 1 missed heartbeat (15 min) | | File storage | 3 consecutive failures | | Nomadic identity endpoint | 3 consecutive failures | | OAuth endpoint | 3 consecutive failures | | Cleanup job heartbeat | 1 missed heartbeat (2 hr) | | SSL certificate | 21 days before expiry |

MySQL/MariaDB and the background poller heartbeat are highest priority — a database failure causes immediate errors across all pages, and a stopped poller means all Zot6 and ActivityPub federation halts silently. The nomadic identity endpoint should be monitored with response time alerts since slowness directly affects channel sync reliability.

Conclusion

A healthy Hubzilla hub is far more than just an accessible homepage. Federation via Zot6 depends on the background poller running on schedule. Nomadic identity depends on the channel discovery endpoint responding reliably. ActivityPub interoperability with Mastodon depends on WebFinger and NodeInfo being reachable. Without external monitoring, any of these components can fail silently for hours — or days — while the web interface appears completely normal.

With Vigilmon watching your Hubzilla web interface, PHP-FPM health, MySQL/MariaDB connectivity, Zot6 federation endpoints, ActivityPub inbox health, background poller heartbeat, file storage availability, nomadic identity endpoint, OAuth health, and cleanup job execution, you'll catch failures within minutes rather than learning about them from frustrated channel owners.

Start monitoring your Hubzilla hub for free at vigilmon.online.