Monitoring fn Project (Fn) with Vigilmon

fn Project (commonly called "Fn") takes a bring-your-own-container approach to serverless: every function is a Docker image, the fn server routes invocations to Docker-managed containers, and you control the entire runtime stack without vendor lock-in. That control comes with an operational responsibility: there's no hosted control plane watching your fn server for you. If the fn API goes down, all function invocations fail silently to callers. If Docker daemon crashes, running containers are orphaned and new invocations never start. Vigilmon fills that gap, giving you continuous health checks for the fn Server API, invocation success rates, Docker daemon health, and runner queue depth from a single dashboard.

What You'll Set Up

• fn Server API health monitoring (port 8080)
• Function invocation success rate tracking
• Cold start vs. warm invocation latency monitoring
• Docker daemon health heartbeat
• Function runner queue depth alerts
• Database health monitoring (SQLite/MySQL/PostgreSQL)
• Memory and CPU utilization heartbeats
• fn LB health monitoring (multi-server deployments)

Prerequisites

• fn server running (single node or multi-server with fn LB)
• fn CLI installed and configured
• Docker daemon accessible on the fn server host
• A free Vigilmon account

Step 1: Monitor the fn Server API

The fn server API (port 8080) handles function creation, deployment, and every invocation. The /version endpoint is the canonical health probe — a non-200 response means the server is down and all invocations are failing.

Add an HTTP monitor:

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter: http://your-fn-server:8080/version
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Enable Keyword match and enter "version" to verify the response body.
7. Click Save.

Verify manually:

curl -s http://your-fn-server:8080/version
# Expected: {"version":"0.3.749"}

If you expose fn behind a reverse proxy (Nginx, Caddy, Traefik) with TLS, point Vigilmon at the HTTPS URL and enable Monitor SSL certificate with a 21 days alert threshold.

Step 2: Monitor Function Invocation Success Rate

fn tracks invocation outcomes through its API. An error rate above 1% for any function indicates a regression in the function code or a resource constraint on the runner.

Add a Vigilmon cron heartbeat for invocation success rate:

1. Add Monitor → Cron Heartbeat.
2. Expected ping interval: 5 minutes.
3. Copy the heartbeat URL.
4. Deploy a monitoring script that checks invocation stats:

#!/bin/bash
# check_fn_invocations.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_INVOCATION_HEARTBEAT_ID"
FN_API="http://your-fn-server:8080"
ERROR_THRESHOLD=1  # percent

# List all apps
APPS=$(curl -s "$FN_API/v2/apps" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for app in data.get('items', []):
    print(app['name'])
")

TOTAL=0
ERRORS=0

for APP in $APPS; do
  # Get functions in this app
  FNS=$(curl -s "$FN_API/v2/fns?app_id=$(curl -s "$FN_API/v2/apps" | python3 -c "import sys,json; [print(a['id']) for a in json.load(sys.stdin).get('items',[]) if a['name']=='$APP']")" | \
    python3 -c "import sys,json; [print(f['id']+':'+f['name']) for f in json.load(sys.stdin).get('items',[])]")

  for FN_ENTRY in $FNS; do
    FN_ID="${FN_ENTRY%%:*}"
    # Check recent calls via fn calls list (if fn stats endpoint available)
    STATS=$(curl -s "$FN_API/v2/fns/$FN_ID/stats" 2>/dev/null)
    if [ -n "$STATS" ]; then
      FN_TOTAL=$(echo "$STATS" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('calls',0))" 2>/dev/null || echo 0)
      FN_ERR=$(echo "$STATS" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('failed',0))" 2>/dev/null || echo 0)
      TOTAL=$((TOTAL + FN_TOTAL))
      ERRORS=$((ERRORS + FN_ERR))
    fi
  done
done

if [ "$TOTAL" -gt 0 ]; then
  ERROR_PCT=$(( ERRORS * 100 / TOTAL ))
  if [ "$ERROR_PCT" -gt "$ERROR_THRESHOLD" ]; then
    echo "ALERT: Invocation error rate ${ERROR_PCT}% (${ERRORS}/${TOTAL})"
    exit 1
  fi
fi

curl -s "$HEARTBEAT_URL" > /dev/null

# /etc/cron.d/fn-invocation-check
*/5 * * * * root /usr/local/bin/check_fn_invocations.sh

Step 3: Monitor Invocation Latency (Cold Start vs Warm)

fn's concurrency model creates a runner queue for concurrent invocations. Latency at P99 exceeding the function timeout means callers are experiencing timeouts. Tracking cold start vs. warm invocation separately identifies whether the issue is container startup overhead or function execution time.

Test invocation latency directly:

#!/bin/bash
# measure_fn_latency.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_LATENCY_HEARTBEAT_ID"
APP="myapp"
FUNC="myfunc"
P99_THRESHOLD_MS=5000  # 5 seconds

# Invoke the function and measure wall-clock time
START=$(date +%s%N)
RESULT=$(fn invoke "$APP" "$FUNC" 2>&1)
END=$(date +%s%N)

ELAPSED_MS=$(( (END - START) / 1000000 ))

if [ "$ELAPSED_MS" -gt "$P99_THRESHOLD_MS" ]; then
  echo "ALERT: fn invoke ${APP}/${FUNC} took ${ELAPSED_MS}ms (threshold: ${P99_THRESHOLD_MS}ms)"
  exit 1
fi

echo "OK: ${APP}/${FUNC} latency ${ELAPSED_MS}ms"
curl -s "$HEARTBEAT_URL" > /dev/null

For production, run this check from a separate monitoring host to measure end-to-end latency including network. A heartbeat interval of 5 minutes balances coverage against extra invocation load.

Step 4: Monitor Docker Image Pull Health

fn downloads function container images on cold starts. Registry unavailability or Docker Hub rate limiting causes cold starts to fail with image pull errors — your functions silently stop working for new container instances.

Add a Docker registry health check:

#!/bin/bash
# check_fn_docker_pull.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_DOCKER_PULL_HEARTBEAT_ID"

# Test pulling a small known-good image to verify Docker Hub / registry connectivity
TEST_IMAGE="hello-world:latest"

if ! docker pull "$TEST_IMAGE" > /dev/null 2>&1; then
  echo "ALERT: Docker image pull failed for $TEST_IMAGE"
  exit 1
fi

# Clean up
docker rmi "$TEST_IMAGE" > /dev/null 2>&1

curl -s "$HEARTBEAT_URL" > /dev/null

If you use a private registry, replace hello-world:latest with a small image from your registry. Set the heartbeat interval to 15 minutes — you don't need to test pulls every minute, but you want to know before the next cold start.

If you use Docker Hub, also add a direct HTTP monitor for the Docker Hub status page:

1. Add Monitor → HTTP / HTTPS.
2. URL: https://hub.docker.com (or your private registry health endpoint)
3. Check interval: 5 minutes.
4. Expected status: 200.

Step 5: Monitor the Function Runner Queue

fn uses an internal runner queue to manage concurrent invocations. When the queue is saturated, new invocations are rejected or delayed beyond their timeout.

Add a cron heartbeat for queue health:

#!/bin/bash
# check_fn_queue.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_QUEUE_HEARTBEAT_ID"
FN_API="http://your-fn-server:8080"

# Check runner stats if available
RUNNER_STATS=$(curl -s "$FN_API/stats" 2>/dev/null)
if [ -n "$RUNNER_STATS" ]; then
  QUEUE_DEPTH=$(echo "$RUNNER_STATS" | python3 -c "
import sys, json
try:
    d = json.load(sys.stdin)
    print(d.get('runner', {}).get('queue_depth', 0))
except:
    print(0)
" 2>/dev/null || echo "0")

  QUEUE_THRESHOLD=100
  if [ "$QUEUE_DEPTH" -gt "$QUEUE_THRESHOLD" ]; then
    echo "ALERT: fn runner queue depth = $QUEUE_DEPTH (threshold: $QUEUE_THRESHOLD)"
    exit 1
  fi
fi

# Alternatively, detect queue saturation via slow response time:
START=$(date +%s%N)
curl -s -o /dev/null "$FN_API/version"
END=$(date +%s%N)
API_MS=$(( (END - START) / 1000000 ))

if [ "$API_MS" -gt 2000 ]; then
  echo "ALERT: fn API response took ${API_MS}ms — possible queue saturation"
  exit 1
fi

curl -s "$HEARTBEAT_URL" > /dev/null

Step 6: Monitor Docker Daemon Health

fn depends entirely on Docker for container lifecycle. If the Docker daemon crashes or becomes unresponsive, no new function containers can start and running containers are orphaned.

Expose Docker daemon health as an HTTP endpoint:

#!/bin/bash
# /usr/local/bin/docker_health_server.sh
while true; do
  if docker info > /dev/null 2>&1; then
    CONTAINERS=$(docker ps -q | wc -l)
    echo -e "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nOK: Docker daemon running, $CONTAINERS containers" | nc -l -p 9997 -q 1
  else
    echo -e "HTTP/1.1 503 Service Unavailable\r\nContent-Type: text/plain\r\n\r\nCRITICAL: Docker daemon not responding" | nc -l -p 9997 -q 1
  fi
done

Run as a systemd service:

# /etc/systemd/system/docker-health-server.service
[Unit]
Description=Docker Health HTTP Server
After=docker.service

[Service]
ExecStart=/usr/local/bin/docker_health_server.sh
Restart=always

[Install]
WantedBy=multi-user.target

systemctl daemon-reload && systemctl enable --now docker-health-server

Add a Vigilmon HTTP monitor:

1. Add Monitor → HTTP / HTTPS.
2. URL: http://your-fn-server:9997
3. Check interval: 1 minute.
4. Expected status: 200.

Step 7: Monitor Database Health

fn stores function metadata in a database — SQLite by default, or MySQL/PostgreSQL for production deployments. Database unavailability prevents fn from creating, updating, or routing to functions.

SQLite (default)

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_DB_HEARTBEAT_ID"
FN_DB="/var/fn/data/fn.db"

if ! sqlite3 "$FN_DB" "SELECT count(*) FROM apps;" > /dev/null 2>&1; then
  echo "ALERT: fn SQLite database query failed"
  exit 1
fi

curl -s "$HEARTBEAT_URL" > /dev/null

MySQL/PostgreSQL

For MySQL:

RESULT=$(mysql -h db-host -u fn_user -p"$FN_DB_PASS" fn -e "SELECT 1;" 2>&1)
if ! echo "$RESULT" | grep -q "1"; then
  echo "ALERT: fn MySQL connection failed"
  exit 1
fi

For PostgreSQL:

RESULT=$(psql -h db-host -U fn_user -d fn -c "SELECT 1;" 2>&1)
if ! echo "$RESULT" | grep -q "1 row"; then
  echo "ALERT: fn PostgreSQL connection failed"
  exit 1
fi

Add a cron heartbeat with interval 2 minutes.

Step 8: Monitor Memory and CPU for OOM Prevention

fn runner processes and function containers share host memory. If memory usage exceeds 90%, the Linux OOM killer terminates running function containers — causing silent invocation failures with no error logs from fn itself.

Add a cron heartbeat for resource utilization:

#!/bin/bash
# check_fn_resources.sh
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_RESOURCE_HEARTBEAT_ID"
MEM_THRESHOLD=90
CPU_THRESHOLD=95

# Memory usage
MEM_USED=$(free | awk '/^Mem:/ {printf "%.0f", $3/$2*100}')
if [ "$MEM_USED" -gt "$MEM_THRESHOLD" ]; then
  echo "ALERT: Host memory at ${MEM_USED}% — OOM kill risk"
  exit 1
fi

# CPU usage (1-minute average)
CPU_IDLE=$(top -bn1 | grep "Cpu(s)" | awk '{print $8}' | tr -d '%id,')
CPU_USED=$(python3 -c "print(int(100 - $CPU_IDLE))")
if [ "$CPU_USED" -gt "$CPU_THRESHOLD" ]; then
  echo "ALERT: CPU at ${CPU_USED}%"
  exit 1
fi

curl -s "$HEARTBEAT_URL" > /dev/null

# /etc/cron.d/fn-resource-check
* * * * * root /usr/local/bin/check_fn_resources.sh

Step 9: Monitor Function Timeouts

fn enforces per-function timeout limits. A spike in timeout events for a specific function indicates that function is taking longer than expected — often due to a downstream dependency (database, external API) becoming slow.

Check timeout events from fn call logs:

#!/bin/bash
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_TIMEOUT_HEARTBEAT_ID"
TIMEOUT_THRESHOLD=5  # per function per 5 minutes
FN_API="http://your-fn-server:8080"

# Query recent calls for timeout status
TIMEOUTS=$(fn calls list "$APP" --limit 100 2>/dev/null | grep -c "timeout" || echo "0")

if [ "$TIMEOUTS" -gt "$TIMEOUT_THRESHOLD" ]; then
  echo "ALERT: $TIMEOUTS timeout events in last 100 calls"
  exit 1
fi

curl -s "$HEARTBEAT_URL" > /dev/null

Set the cron heartbeat interval to 5 minutes.

Step 10: Monitor fn LB (Multi-Server Deployments)

In multi-server deployments, fn LB load-balances across multiple fn Server instances. If fn LB goes down, all invocations fail even if the fn server nodes are healthy.

Add HTTP monitors for fn LB:

1. Add Monitor → HTTP / HTTPS.
2. URL: http://fn-lb-host:8081/version (fn LB typically proxies on a different port)
3. Check interval: 1 minute.
4. Expected status: 200.

Monitor backend pool health via fn LB's admin endpoint (if available):

curl -s http://fn-lb-host:8081/1/lb/backends

Add a Vigilmon HTTP monitor for each fn server node directly (port 8080) in addition to the fn LB monitor. If all node monitors are up but fn LB is down, you have an LB-specific failure.

Step 11: Configure Alerts in Vigilmon

1. Go to Alert Channels and add Slack, PagerDuty, or email.
2. Set thresholds per monitor:
   • fn Server /version: alert after 1 failed check — all invocations down
   • Docker daemon HTTP: alert after 1 failed check — function containers cannot start
   • fn LB: alert after 1 failed check
   • Invocation success rate heartbeat: alert when ping is 1× interval late
   • Latency heartbeat: alert when ping is 2× interval late
   • Docker pull heartbeat: alert when ping is 2× interval late
   • Resource utilization heartbeat: alert when ping is 1× interval late
   • Timeout heartbeat: alert when ping is 2× interval late
   • Database heartbeat: alert when ping is 1× interval late

Step 12: Status Page for Your Team

1. Status Pages → New Page.
2. Add monitor groups:
   • API Layer: fn Server + fn LB monitors
   • Container Runtime: Docker daemon health monitor
   • Invocation Health: success rate + latency + timeout heartbeats
   • Infrastructure: memory/CPU + database + Docker pull heartbeats

Conclusion

fn Project's simplicity — a Go binary, Docker, and a database — means its failure modes are straightforward but still silent without active monitoring. The fn server going down, Docker daemon crashing, or memory reaching OOM territory all manifest as invocation failures that callers receive as errors, with no built-in alerting to notify you. Vigilmon covers every layer: the fn Server /version endpoint for direct availability, Docker daemon health for the container runtime, and cron heartbeats for the metrics that fn doesn't expose over HTTP — success rates, latency, queue depth, and resource utilization.

Start with the fn Server /version monitor and Docker daemon health — two monitors that cover the two most impactful failure modes. Add the invocation success rate and resource utilization heartbeats once your alerting channels are configured.

Sign up for Vigilmon free and add your first fn Project monitor in under two minutes.