tutorial

How to Monitor Daytona with Vigilmon (Workspace Latency, Runner Health + Alerts)

Daytona is a development environment manager that provisions standardised, reproducible workspaces from any Git repository on your own infrastructure — runne...

Daytona is a development environment manager that provisions standardised, reproducible workspaces from any Git repository on your own infrastructure — runner VMs, cloud machines, or Kubernetes. Teams use Daytona to eliminate configuration drift between developer machines and to spin up environments for feature branches, code reviews, and CI workflows.

Because Daytona is infrastructure you own and operate, its health is entirely your responsibility. When a runner goes offline, a Git provider connection breaks, or a dev container image can't be pulled, Daytona fails silently — the workspace creation just hangs or errors, and developers are left without context. In this tutorial you'll set up comprehensive monitoring for Daytona's critical infrastructure using Vigilmon — free tier, no credit card required.

Why Daytona needs external monitoring

Daytona's own logs and daytona server status tell you what the local server process knows. External monitoring tells you what happens when a developer actually tries to use it.

The failure modes that catch teams off-guard:

  • Daytona server unreachable — the server process crashes or the reverse proxy in front of it goes down; daytona list from developer machines hangs or returns connection errors
  • Runner goes offline — a runner VM reboots, runs out of disk space, or loses network connectivity; workspace creation requests queue up and eventually timeout with no clear error
  • Git provider connectivity broken — Daytona clones repositories at workspace creation time; if connectivity to GitHub, GitLab, or Gitea breaks, every workspace creation fails with a confusing error
  • Dev container image pull failure — a runner can't reach the container registry (authentication expired, rate limit hit, registry unreachable); workspace builds stall at the image pull step
  • API proxy latency spike — the Daytona API server becomes slow to respond when runners are overloaded; workspace operations from developer machines feel sluggish or timeout before completing

External monitoring catches these before the support tickets start.

What you'll need

  • A running Daytona server installation
  • One or more configured runners
  • A free Vigilmon account — sign up takes 30 seconds

Step 1: Monitor the Daytona server API

Daytona's server exposes an HTTP API for workspace management. Monitoring this endpoint validates that the core service is running and reachable.

First, confirm your Daytona server's API address:

# Check Daytona server info
daytona server info
# Server URL: https://daytona.example.com
# Status: running
# Version: 0.28.0

The Daytona API has a health endpoint you can probe:

# Check server health
curl -s https://daytona.example.com/health
# {"status":"ok","version":"0.28.0"}

# Or check the server info endpoint (may require auth token)
curl -s -H "Authorization: Bearer $DAYTONA_API_KEY" \
  https://daytona.example.com/api/server/config

In Vigilmon:

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. URL: https://daytona.example.com/health
  4. Interval: 1 minute
  5. Expected status code: 200
  6. (Optional) Response body contains: "status":"ok"
  7. Save the monitor

If your Daytona server is behind a reverse proxy like nginx or Caddy, also add an HTTP monitor for the root URL to check that the proxy is correctly forwarding requests:

  1. Add another HTTP monitor for https://daytona.example.com
  2. Expected status: 200 or 302 (redirect to login)
  3. Save

Step 2: Monitor workspace creation latency

Workspace creation latency is the end-to-end time from "workspace create requested" to "workspace SSH-accessible." This is the most developer-visible performance metric for Daytona.

Create a timed probe that exercises the full workspace creation flow:

#!/usr/bin/env bash
# daytona-create-probe.sh — run every 15 minutes via cron

DAYTONA_API="https://daytona.example.com"
DAYTONA_TOKEN="${DAYTONA_API_KEY}"
PROBE_REPO="https://github.com/yourorg/daytona-probe-repo"
WORKSPACE_NAME="monitor-probe-$(date +%s)"
MAX_WAIT=300  # 5-minute SLA
HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_CREATION_HEARTBEAT_ID"

start_time=$(date +%s)

# Create workspace via Daytona API
response=$(curl -s -X POST \
  -H "Authorization: Bearer $DAYTONA_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"name\": \"$WORKSPACE_NAME\", \"repositories\": [{\"url\": \"$PROBE_REPO\"}]}" \
  "${DAYTONA_API}/api/workspace")

workspace_id=$(echo "$response" | jq -r '.id')

if [ -z "$workspace_id" ] || [ "$workspace_id" = "null" ]; then
  echo "Failed to create workspace: $response"
  exit 1
fi

echo "Created workspace: $workspace_id"

# Poll until running or timeout
while true; do
  elapsed=$(( $(date +%s) - start_time ))

  if [ $elapsed -gt $MAX_WAIT ]; then
    echo "Workspace creation timed out after ${elapsed}s"
    curl -s -X DELETE \
      -H "Authorization: Bearer $DAYTONA_TOKEN" \
      "${DAYTONA_API}/api/workspace/$workspace_id"
    exit 1
  fi

  status=$(curl -s \
    -H "Authorization: Bearer $DAYTONA_TOKEN" \
    "${DAYTONA_API}/api/workspace/$workspace_id" | \
    jq -r '.state')

  echo "Status after ${elapsed}s: $status"

  if [ "$status" = "running" ]; then
    echo "Workspace ready in ${elapsed}s"
    break
  elif [ "$status" = "error" ]; then
    echo "Workspace entered error state"
    curl -s -X DELETE \
      -H "Authorization: Bearer $DAYTONA_TOKEN" \
      "${DAYTONA_API}/api/workspace/$workspace_id"
    exit 1
  fi

  sleep 10
done

# Send heartbeat to Vigilmon
curl -s "$HEARTBEAT_URL" -d "creation_seconds=$(( $(date +%s) - start_time ))"

# Clean up probe workspace
curl -s -X DELETE \
  -H "Authorization: Bearer $DAYTONA_TOKEN" \
  "${DAYTONA_API}/api/workspace/$workspace_id"

Schedule this probe via cron on a host outside the Daytona server:

*/15 * * * * /opt/probes/daytona-create-probe.sh >> /var/log/daytona-probe.log 2>&1

In Vigilmon, create a Heartbeat monitor with a 20-minute grace window. A missed heartbeat means workspace creation is broken or SLA-busting.

Step 3: Monitor runner health

Runners are the compute layer where workspaces actually execute. If a runner goes offline, workspace creation requests destined for that runner hang or fail with cryptic errors.

Runner API health

Daytona runners expose health information through the server API:

# List all runners and their status
curl -s -H "Authorization: Bearer $DAYTONA_TOKEN" \
  https://daytona.example.com/api/runner | \
  jq '[.[] | {name: .name, state: .state, workspaces: .workspaceCount}]'

Create a runner health probe:

#!/usr/bin/env bash
# daytona-runner-probe.sh

DAYTONA_API="https://daytona.example.com"
DAYTONA_TOKEN="${DAYTONA_API_KEY}"
RUNNER_HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_RUNNER_HEARTBEAT_ID"

# Get all runners
runners=$(curl -s \
  -H "Authorization: Bearer $DAYTONA_TOKEN" \
  "${DAYTONA_API}/api/runner")

total=$(echo "$runners" | jq '. | length')
offline=$(echo "$runners" | jq '[.[] | select(.state != "running")] | length')

if [ "$total" -eq 0 ]; then
  echo "No runners registered"
  exit 1
fi

if [ "$offline" -gt 0 ]; then
  echo "$offline of $total runners offline"
  # Alert if more than 50% of runners are down
  threshold=$(( total / 2 ))
  if [ "$offline" -gt "$threshold" ]; then
    exit 1
  fi
fi

curl -s "$RUNNER_HEARTBEAT_URL" -d "total=$total&offline=$offline"

Create a Heartbeat monitor in Vigilmon with a 5-minute grace window for runner health.

TCP connectivity to runner VMs

Also monitor TCP connectivity to each runner VM directly — this validates that the runner is reachable at the network level even if the Daytona API doesn't surface runner-level detail:

  1. In Vigilmon, go to Monitors → New Monitor
  2. Choose TCP Port
  3. Host: runner-1.example.com, Port: 22 (SSH)
  4. Interval: 1 minute
  5. Save

Repeat for each runner VM. SSH port reachability is a prerequisite for Daytona to provision workspaces on that runner.

Step 4: Monitor Git provider connectivity

Every workspace creation in Daytona involves cloning a Git repository. If connectivity to your Git providers breaks, 100% of workspace creations fail.

Set up connectivity probes for each Git provider you use:

GitHub connectivity

#!/usr/bin/env bash
# git-connectivity-probe.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_GIT_HEARTBEAT_ID"

# Test HTTPS connectivity to GitHub
if ! curl -sf --max-time 10 https://github.com/status > /dev/null 2>&1; then
  echo "GitHub HTTPS connectivity failed"
  exit 1
fi

# Test actual clone operation (use a small public repo)
TMPDIR=$(mktemp -d)
if ! git clone --depth=1 --quiet https://github.com/octocat/Hello-World.git "$TMPDIR" 2>&1; then
  echo "GitHub clone failed"
  rm -rf "$TMPDIR"
  exit 1
fi

rm -rf "$TMPDIR"
curl -s "$HEARTBEAT_URL"

For GitLab or self-hosted Gitea, monitor those HTTP endpoints directly in Vigilmon:

  1. Go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. URL: https://gitlab.example.com (or your self-hosted Gitea URL)
  4. Expected status: 200
  5. Save

Git provider HTTP monitors

| Provider | Monitor URL | Expected response | |----------|-------------|-------------------| | GitHub | https://github.com | 200 | | GitLab (cloud) | https://gitlab.com | 200 | | GitLab (self-hosted) | https://gitlab.example.com/-/health | 200, body GitLab OK | | Gitea | https://gitea.example.com/api/v1/version | 200 |

Step 5: Monitor dev container image pull status

Dev container image pull failures are the most opaque failure mode in Daytona — the workspace just stalls during creation with no clear progress indicator. Monitor your container registry endpoints and image pull success rates.

Container registry health

# Check Docker Hub (if using public images)
curl -sf https://hub.docker.com/v2/ > /dev/null && echo "Docker Hub reachable"

# Check your private registry
curl -sf -H "Authorization: Bearer $REGISTRY_TOKEN" \
  https://registry.example.com/v2/ > /dev/null && echo "Private registry reachable"

Add HTTP monitors in Vigilmon for your registries:

  1. Docker Hub: Monitor https://hub.docker.com/v2/ expecting 200
  2. Private registry: Monitor https://registry.example.com/v2/ expecting 200 or 401 (auth required, but registry is responding)
  3. GitHub Container Registry: Monitor https://ghcr.io/v2/ expecting 200 or 401

Image pull probe

Set up a probe that exercises an actual image pull:

#!/usr/bin/env bash
# image-pull-probe.sh

HEARTBEAT_URL="https://vigilmon.online/heartbeat/YOUR_IMAGE_HEARTBEAT_ID"
IMAGE="ubuntu:22.04"

# Remove local cache to force a real pull
docker rmi "$IMAGE" > /dev/null 2>&1 || true

# Time the pull
start=$(date +%s)
if ! docker pull "$IMAGE" > /dev/null 2>&1; then
  echo "Image pull failed for $IMAGE"
  exit 1
fi

elapsed=$(( $(date +%s) - start ))
echo "Image pull took ${elapsed}s"

# Alert if pull takes longer than 60 seconds (sign of registry issues)
if [ $elapsed -gt 60 ]; then
  echo "Image pull too slow: ${elapsed}s"
  exit 1
fi

curl -s "$HEARTBEAT_URL" -d "pull_seconds=$elapsed"

Run this on one of your runner VMs every 15 minutes. A missed heartbeat indicates registry connectivity or performance issues that will stall workspace creation.

Configure alert channels

In Vigilmon, go to Alert Channels → Add Channel:

  • Email — platform engineering on-call distribution list
  • Webhook — Slack #platform-alerts channel for team visibility

Example webhook payload Vigilmon sends:

{
  "monitor_name": "daytona-runner-1-ssh",
  "status": "down",
  "url": "tcp://runner-1.example.com:22",
  "started_at": "2024-01-15T10:23:00Z",
  "duration_seconds": 180
}

Use this to trigger automated runbook steps — like rotating to a backup runner or restarting the Daytona agent on the affected VM.

Putting it all together

| Monitor | Type | What it catches | |---------|------|-----------------| | https://daytona.example.com/health | HTTP | Daytona server crashes, proxy failures | | https://daytona.example.com | HTTP | Reverse proxy and load balancer health | | Workspace creation latency heartbeat | Heartbeat | Creation SLA breaches, runner failures | | Runner health heartbeat | Heartbeat | Runner offline / reduced capacity | | runner-1.example.com:22 | TCP | Runner VM SSH reachability | | runner-2.example.com:22 | TCP | Runner VM SSH reachability | | Git provider heartbeat | Heartbeat | GitHub/GitLab/Gitea clone connectivity | | https://registry.example.com/v2/ | HTTP | Container registry availability | | Image pull latency heartbeat | Heartbeat | Registry rate limits, slow pull latency |

Set HTTP and TCP monitors to 1-minute intervals. Use 5-minute grace windows for runner health heartbeats and 20-minute windows for workspace creation and image pull heartbeats.

What's next

  • SSL certificate monitoring — Daytona's API and any workspace-proxied services use TLS; Vigilmon monitors certificate expiry and alerts you 30 days in advance so a lapsed certificate doesn't surprise your team
  • Status page for developers — create a Vigilmon status page with your Daytona monitors grouped by component (Server, Runners, Git Providers, Registry); give developers the URL so they can self-diagnose connectivity issues before filing platform tickets
  • Runner auto-healing — combine Vigilmon webhooks with a small automation script: when a runner's TCP monitor triggers a down alert, automatically restart the Daytona agent on that runner or mark it as unavailable for scheduling

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles