tutorial

Soft Serve Monitoring with Vigilmon

"Learn how to monitor Soft Serve, the self-hosted Git server with TUI from Charmbracelet, with Vigilmon — covering SSH server availability, HTTP web UI uptime, Git protocol endpoint health, repository access authentication, webhook delivery service, and LFS storage backend monitoring."

Soft Serve Monitoring with Vigilmon

Soft Serve is a self-hosted Git server built by Charmbracelet, written in Go. It provides SSH access with a terminal user interface (TUI), an HTTP web UI for browsing repositories, Git smart HTTP for clone/push/pull, and a webhook delivery system. Teams self-host Soft Serve to run a lightweight, self-contained Git server without the operational overhead of GitLab or Gitea, particularly in air-gapped or homelab environments.

Soft Serve binds to three distinct ports: SSH on 23231, HTTP on 23232, and Git protocol on 9418. Each serves a different access path — developers may rely on any combination of them. A failure on one port can silently block an entire workflow while the others appear healthy. Without monitoring, a stalled SSH daemon or a broken Git protocol endpoint can go undetected until a developer fails to push or pull.

This guide covers how to monitor Soft Serve with Vigilmon.

Why Soft Serve Needs Monitoring

Soft Serve failures can block developer workflows across multiple access paths:

  • SSH server failure — the SSH daemon on port 23231 stops accepting connections; developers relying on SSH clone URLs cannot push or pull, and the TUI is inaccessible
  • HTTP web UI failure — the HTTP server on port 23232 goes down; repository browsing, README rendering, and HTTP-based clone/pull operations are blocked
  • Git protocol endpoint failure — the Git daemon on port 9418 stops serving; unauthenticated read-only clones using the git:// protocol fail silently
  • Repository access or auth failure — public key authentication or access control rules become inconsistent; authenticated pushes are rejected while the SSH daemon appears healthy
  • Webhook delivery failure — outbound webhook calls stop being sent; CI/CD pipelines, Slack notifications, and deployment hooks that depend on push events receive no triggers
  • LFS storage backend failure — the Git LFS proxy or storage backend becomes unavailable; repositories with large file assets cannot push or pull LFS objects

Vigilmon monitors all of these failure modes so broken access paths are caught before developers hit them.

Soft Serve Architecture Overview

| Component | Default Port | Role | Monitoring Priority | |-----------|-------------|------|---------------------| | SSH server (Git + TUI) | 23231 | SSH access, TUI, authenticated push/pull | Critical | | HTTP web UI + Git smart HTTP | 23232 | Web browsing, HTTP clone/pull | Critical | | Git daemon (read-only) | 9418 | Unauthenticated git:// clone | High | | Webhook delivery | Internal | CI/CD and notification triggers | High | | LFS storage backend | External or local | Large file object storage | Medium |

Monitor 1: Soft Serve HTTP Web UI Availability

Monitor the HTTP web UI to confirm the Go HTTP server is responding:

  1. Log in to Vigilmon → Monitors → New Monitor
  2. Type: HTTP
  3. Name: Soft Serve - HTTP Web UI
  4. URL: http://your-soft-serve-host:23232/
  5. Method: GET
  6. Expected status: 200
  7. Keyword check: Soft Serve
  8. Interval: 1 minute

Test your endpoint:

curl -s http://your-soft-serve-host:23232/ | grep -i "soft serve"
# Expected: HTML containing "Soft Serve"

Monitor 2: Soft Serve Git Smart HTTP Info/Refs

Git smart HTTP exposes an info/refs endpoint for each repository. Monitoring this endpoint confirms both the HTTP server and repository index are healthy:

  1. Type: HTTP
  2. Name: Soft Serve - Git Smart HTTP
  3. URL: http://your-soft-serve-host:23232/your-repo.git/info/refs?service=git-upload-pack
  4. Method: GET
  5. Expected status: 200
  6. Keyword check: git-upload-pack
  7. Interval: 2 minutes

Replace your-repo with a public or accessible repository name.

curl -s "http://your-soft-serve-host:23232/your-repo.git/info/refs?service=git-upload-pack" | head -5
# Expected: response starting with the git-upload-pack pkt-line header

Monitor 3: Soft Serve SSH Port Availability (TCP)

Monitor the SSH port to confirm the SSH daemon is accepting TCP connections. This does not require authentication — a successful TCP handshake on port 23231 confirms the daemon is up:

  1. Type: TCP
  2. Name: Soft Serve - SSH Port
  3. Host: your-soft-serve-host
  4. Port: 23231
  5. Interval: 1 minute
nc -zv your-soft-serve-host 23231
# Expected: Connection succeeded

Monitor 4: Git Protocol Port Availability (TCP)

Monitor the Git daemon port to confirm unauthenticated read access is available:

  1. Type: TCP
  2. Name: Soft Serve - Git Protocol Port
  3. Host: your-soft-serve-host
  4. Port: 9418
  5. Interval: 2 minutes
nc -zv your-soft-serve-host 9418
# Expected: Connection succeeded
# Error: Connection refused (Git daemon stopped)

Monitor 5: SSL Certificate Alert

If Soft Serve's HTTP UI is exposed via a reverse proxy with HTTPS:

  1. Type: HTTP
  2. Name: Soft Serve - SSL Certificate
  3. URL: https://git.your-domain.com/
  4. SSL certificate expiry alert: 21 days before expiry
  5. Interval: 1 hour
curl -vI https://git.your-domain.com/ 2>&1 | grep "expire date"
# Expected: a future expiry date with at least 21 days remaining

Monitor 6: Soft Serve End-to-End Heartbeat

An hourly heartbeat clones a known small repository over HTTP to confirm that the full Git smart HTTP path — HTTP server, repository index, and pack file generation — is functional:

Heartbeat Script

#!/bin/bash
# soft-serve-heartbeat.sh — run hourly via cron
set -euo pipefail

SOFT_SERVE_HOST="your-soft-serve-host"
SOFT_SERVE_PORT="23232"
TEST_REPO="your-test-repo"  # a small public repo on your Soft Serve instance
VIGILMON_HEARTBEAT="YOUR_HEARTBEAT_ID"
TMPDIR_HB=$(mktemp -d)

cleanup() {
  rm -rf "$TMPDIR_HB"
}
trap cleanup EXIT

# Step 1: Confirm HTTP UI is up
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  "http://${SOFT_SERVE_HOST}:${SOFT_SERVE_PORT}/")

if [ "$HTTP_STATUS" -ne 200 ]; then
  echo "[soft-serve-hb] HTTP UI returned $HTTP_STATUS"
  exit 1
fi

# Step 2: Clone test repo over smart HTTP (verifies full Git path)
git clone --depth 1 \
  "http://${SOFT_SERVE_HOST}:${SOFT_SERVE_PORT}/${TEST_REPO}.git" \
  "${TMPDIR_HB}/repo" > /dev/null 2>&1

if [ ! -d "${TMPDIR_HB}/repo/.git" ]; then
  echo "[soft-serve-hb] Git clone failed"
  exit 1
fi

echo "[soft-serve-hb] HTTP UI and Git smart HTTP clone both healthy"

# Step 3: Signal Vigilmon
curl -s "https://heartbeat.vigilmon.online/$VIGILMON_HEARTBEAT"

Add to cron:

# /etc/cron.d/soft-serve-heartbeat
0 * * * * root /opt/soft-serve/soft-serve-heartbeat.sh >> /var/log/soft-serve-heartbeat.log 2>&1

In Vigilmon:

  • Type: Heartbeat
  • Name: Soft Serve - Git Clone Pipeline
  • Expected interval: 1 hour
  • Grace period: 10 minutes

Alert Configuration

| Monitor | Type | Interval | Alert Channel | |---------|------|----------|---------------| | HTTP web UI | HTTP | 1 min | Slack + Email | | Git smart HTTP | HTTP | 2 min | Slack | | SSH port | TCP | 1 min | PagerDuty + Slack | | Git protocol port | TCP | 2 min | Slack | | SSL certificate | HTTP | 1 hour | Email (21-day lead) | | Git clone pipeline | Heartbeat | 1 hour | PagerDuty + Email |

Use PagerDuty for the SSH port monitor. SSH is the primary developer access path — a failed push during a deployment will surface immediately as a blocked CI pipeline.

Status Page

  1. Status Pages → New Page → name it "Soft Serve Git Server"
  2. Add all monitors above
  3. Share with your engineering team

When a developer reports that git push is hanging or the web UI is returning errors, they can check this page before opening a ticket.

Summary

Soft Serve exposes three distinct access paths (SSH, HTTP, Git protocol), each of which can fail independently. Vigilmon keeps all of them healthy:

  • HTTP monitor for web UI availability and keyword confirmation
  • Git smart HTTP monitor for end-to-end repository index health
  • TCP monitors for SSH and Git daemon port availability
  • SSL certificate monitoring with a 21-day lead time for HTTPS deployments
  • Hourly heartbeat that clones a test repository over smart HTTP to verify the full Git serving pipeline

Get started at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles