Monitoring Self-Hosted Hemmelig with Vigilmon

Hemmelig is a self-hosted secret sharing application that encrypts notes and credentials for time-limited sharing. When a team member shares a database password or API key via Hemmelig, they trust that the link will be accessible exactly once before it expires. If the Fastify server, database, or expiry job goes down, that trust breaks — and the person on the other end gets a cryptic error instead of the secret they need. Vigilmon monitors every layer of Hemmelig so you catch failures before they disrupt your team's secure credential workflows.

What You'll Set Up

• HTTP uptime monitor for the Hemmelig web UI (port 3000)
• Secret encryption and storage service health check
• Database connectivity monitor (SQLite or PostgreSQL)
• Redis cache monitor (if configured)
• Cron heartbeat for automatic secret expiry and deletion jobs
• File upload encryption service monitor
• Rate limiting middleware health check
• API endpoint response time monitoring

Prerequisites

• Hemmelig installed and running (Docker or bare metal)
• Hemmelig accessible at http://your-server:3000 or behind a reverse proxy
• A free Vigilmon account

Step 1: Monitor the Hemmelig Web Interface

Hemmelig serves both its React frontend and Fastify API from the same Node.js process on port 3000. If either crashes, secrets become inaccessible.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter your Hemmelig URL: https://hemmelig.yourdomain.com (or http://your-server:3000).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Enable Monitor SSL certificate with a 21-day expiry alert if you use HTTPS.
7. Click Save.

Hemmelig's root path returns the React app shell — a 200 response means the server process is alive and nginx (or Caddy) is routing correctly to it.

Step 2: Monitor the API Health Endpoint

Hemmelig exposes a health check route at /api/health that confirms the Fastify server and its plugins are initialized correctly.

1. Click Add Monitor → HTTP / HTTPS.
2. Set URL to https://hemmelig.yourdomain.com/api/health.
3. Set Expected HTTP status to 200.
4. Set Check interval to 1 minute.
5. Click Save.

# Verify manually
curl https://hemmelig.yourdomain.com/api/health
# {"status":"ok"}

This endpoint is lightweight and does not create or read any secrets — safe to probe as frequently as every 30 seconds on busy instances.

Step 3: Monitor the Secret Encryption and Storage Service

The core value of Hemmelig is that secrets are encrypted before storage. The encryption pipeline runs server-side using AES-256. You can verify this layer is functioning by probing the secret creation API with a test payload.

Create a lightweight test script:

#!/bin/bash
# hemmelig-health.sh
# Create a test secret and immediately verify it's retrievable

RESPONSE=$(curl -s -X POST \
  https://hemmelig.yourdomain.com/api/v1/secret \
  -H "Content-Type: application/json" \
  -d '{"text":"healthcheck","ttl":300,"password":""}')

SECRET_ID=$(echo "$RESPONSE" | grep -o '"id":"[^"]*"' | cut -d'"' -f4)

if [ -n "$SECRET_ID" ]; then
  # Verify retrieval works
  STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
    "https://hemmelig.yourdomain.com/api/v1/secret/$SECRET_ID")
  
  if [ "$STATUS" -eq 200 ]; then
    curl -s https://vigilmon.online/heartbeat/ENCRYPTION_HEARTBEAT_ID
  fi
fi

Schedule this every 5 minutes:

*/5 * * * * /opt/hemmelig/scripts/hemmelig-health.sh

Create a cron heartbeat in Vigilmon with a 5 minute expected interval. A missed ping means either secret creation or retrieval is broken.

Step 4: Monitor Database Connectivity

Hemmelig supports two database backends: SQLite (default for single-server installs) and PostgreSQL (recommended for production). All secrets, metadata, and expiry timestamps are stored here.

SQLite

For SQLite, monitor disk access and database integrity with a cron heartbeat:

#!/bin/bash
# Check SQLite database is readable and not corrupted
DB_PATH="/path/to/hemmelig/database.sqlite"

sqlite3 "$DB_PATH" "PRAGMA integrity_check;" 2>/dev/null | grep -q "ok" && \
  curl -s https://vigilmon.online/heartbeat/DB_HEARTBEAT_ID

Schedule every 5 minutes. Also add a disk usage monitor:

DISK_USAGE=$(df "$(dirname $DB_PATH)" | awk 'NR==2 {print $5}' | tr -d '%')
[ "$DISK_USAGE" -lt 85 ] && curl -s https://vigilmon.online/heartbeat/DISK_HEARTBEAT_ID

PostgreSQL

For PostgreSQL, add a TCP port monitor:

1. Click Add Monitor → TCP Port.
2. Set Host to your PostgreSQL server.
3. Set Port to 5432.
4. Set Check interval to 1 minute.
5. Click Save.

Pair with a connection-level heartbeat:

* * * * * pg_isready -h localhost -U hemmelig -d hemmelig && \
  curl -s https://vigilmon.online/heartbeat/DB_HEARTBEAT_ID

Step 5: Monitor Redis Cache (If Configured)

Hemmelig can use Redis to cache frequently accessed (non-secret) data and support rate limiting enforcement across multiple instances. If Redis is configured and goes down, Hemmelig may either fall back gracefully or start failing rate limiting checks — behavior depends on your configuration.

1. Click Add Monitor → TCP Port.
2. Set Host to your Redis server.
3. Set Port to 6379.
4. Set Check interval to 1 minute.
5. Click Save.

Verify with a cron heartbeat:

* * * * * redis-cli -h 127.0.0.1 ping 2>/dev/null | grep -q PONG && \
  curl -s https://vigilmon.online/heartbeat/REDIS_HEARTBEAT_ID

If Redis is not configured in your Hemmelig setup, skip this step.

Step 6: Heartbeat for Secret Expiry and Deletion Jobs

Hemmelig automatically deletes expired secrets based on the TTL set at creation time. This cleanup job is critical: if it stops running, your database grows indefinitely and expired secrets remain retrievable past their intended lifetime — a security concern.

Hemmelig runs the cleanup job internally on a schedule. Monitor it by checking that recently-expired secrets are actually being removed:

#!/bin/bash
# Check that expired secrets are being cleaned up
# Count secrets that should have expired in the last hour but haven't been deleted
EXPIRED_COUNT=$(sqlite3 /path/to/hemmelig/database.sqlite \
  "SELECT COUNT(*) FROM secrets WHERE expiresAt < datetime('now') AND deletedAt IS NULL;" \
  2>/dev/null)

# If no expired secrets are lingering, the cleanup job is running
if [ "${EXPIRED_COUNT:-0}" -lt 100 ]; then
  curl -s https://vigilmon.online/heartbeat/CLEANUP_HEARTBEAT_ID
fi

Schedule every 15 minutes and set the Vigilmon heartbeat interval to 15 minutes.

For PostgreSQL:

psql -U hemmelig -d hemmelig -c \
  "SELECT COUNT(*) FROM secrets WHERE expires_at < NOW() AND deleted_at IS NULL;" \
  -t 2>/dev/null | grep -q "^[[:space:]]*[0-9]" && \
  curl -s https://vigilmon.online/heartbeat/CLEANUP_HEARTBEAT_ID

Step 7: Monitor File Upload Encryption

Hemmelig supports encrypted file attachments alongside text secrets. The file upload pipeline encrypts files server-side before storing them on disk or in object storage. If the upload directory runs out of space or loses write permission, file attachments silently fail.

Add a disk and write-access check:

#!/bin/bash
# Check upload directory is writable and has sufficient space
UPLOAD_DIR="/path/to/hemmelig/uploads"

DISK_USAGE=$(df "$UPLOAD_DIR" 2>/dev/null | awk 'NR==2 {print $5}' | tr -d '%')

if [ "${DISK_USAGE:-100}" -lt 85 ] && \
   touch "$UPLOAD_DIR/.healthcheck" 2>/dev/null; then
  rm -f "$UPLOAD_DIR/.healthcheck"
  curl -s https://vigilmon.online/heartbeat/FILE_UPLOAD_HEARTBEAT_ID
fi

Schedule every 5 minutes and set the Vigilmon heartbeat interval accordingly.

Step 8: Monitor Rate Limiting Middleware

Hemmelig's rate limiting prevents brute-force attacks on secret retrieval. It is implemented via Fastify's rate limit plugin. If rate limiting is misconfigured or Redis (if used for distributed rate limiting) is unavailable, either all requests pass through unthrottled or all requests get blocked.

The best way to monitor rate limiting health is to verify the API returns 429 Too Many Requests when expected. Create a probe that intentionally exceeds the limit:

#!/bin/bash
# Send 10 rapid requests and verify the 11th gets rate-limited
for i in $(seq 1 10); do
  curl -s -o /dev/null https://hemmelig.yourdomain.com/api/health
done

RATE_LIMIT_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  https://hemmelig.yourdomain.com/api/health)

# Either 429 (rate limited) or 200 (not yet hit) means the middleware is working
if [ "$RATE_LIMIT_STATUS" -eq 429 ] || [ "$RATE_LIMIT_STATUS" -eq 200 ]; then
  curl -s https://vigilmon.online/heartbeat/RATELIMIT_HEARTBEAT_ID
fi

Step 9: Monitor API Response Time

Secret retrieval is time-sensitive by design — users expect Hemmelig links to open instantly. A degraded database or encrypted storage layer can cause response times to spike even when the service is technically "up."

Vigilmon tracks response times automatically for every HTTP monitor. Enable response time alerting:

1. Open the Hemmelig web UI monitor (from Step 1).
2. Under Performance, enable Alert on slow response.
3. Set Alert threshold to 2000ms (2 seconds).
4. Click Save.

For the API health endpoint monitor (Step 2), set the threshold to 500ms — the health endpoint should respond in under half a second. A spike above 500ms indicates the Fastify process is under heavy load or the database is slow.

You can also add a dedicated response time test for the secret creation endpoint:

1. Click Add Monitor → HTTP / HTTPS.
2. Set URL to https://hemmelig.yourdomain.com/api/v1/secret.
3. Set Method to POST with a minimal JSON body.
4. Enable Alert on slow response at 1000ms.
5. Click Save.

Step 10: Configure Alert Channels

1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
2. Set Consecutive failures before alert to 1 for the database monitor — any database failure is immediately critical.
3. Set it to 2 for the web UI monitor to absorb brief restart events.
4. For the secret expiry heartbeat, set Grace period to 30 minutes — cleanup jobs run on a schedule and a missed 15-minute window isn't always a failure.

Summary

| Monitor | Target | What It Catches | |---|---|---| | Web UI | https://hemmelig.domain.com | Fastify crash, nginx routing failure | | API health | /api/health | Server initialization failure | | Encryption heartbeat | Create + retrieve cron | Secret creation or decryption broken | | SQLite integrity | Cron PRAGMA integrity_check | Database corruption | | PostgreSQL TCP | :5432 | Database server unreachable | | Redis TCP | :6379 | Cache and rate limiting down | | Expiry cleanup heartbeat | Cron DB count check | Expired secrets not being deleted | | File upload heartbeat | Disk write check | File attachments silently failing | | Rate limit heartbeat | Cron probe for 429 | Rate limiting middleware broken | | Response time alert | Web UI and API | Encrypted storage performance degradation |

Hemmelig handles your team's most sensitive credentials — database passwords, API keys, two-factor recovery codes. With Vigilmon watching every layer, from the Fastify process to the SQLite integrity check, you'll know immediately when secret sharing is compromised and can restore service before anyone shares a credential through a less secure channel.