tutorial

How to Monitor Yggdrasil Mesh Networks with Vigilmon

Yggdrasil is an end-to-end encrypted, self-healing IPv6 overlay mesh network. Unlike traditional VPNs with a central server, Yggdrasil uses a distributed tre...

Yggdrasil is an end-to-end encrypted, self-healing IPv6 overlay mesh network. Unlike traditional VPNs with a central server, Yggdrasil uses a distributed tree routing algorithm where every node is equal — there's no single point of failure, no central coordinator, and no predefined topology. Nodes discover each other through public or private peers and form an adaptive mesh that routes around failures automatically.

This architecture makes Yggdrasil compelling for distributed systems, edge deployments, and privacy-conscious infrastructure. But it also means monitoring looks fundamentally different: there's no server to ping, no control plane to check, and "the network is up" depends on the local node's view of the mesh. This tutorial shows you how to build meaningful observability for Yggdrasil deployments with Vigilmon.

Why Yggdrasil needs dedicated monitoring

Yggdrasil's decentralized design creates unique observability challenges:

  • Node isolation — a node that loses all its configured peers becomes silently isolated. Traffic to and from that node fails, but the node itself stays "running" with a healthy process and interface.
  • Peer connection churn — Yggdrasil maintains persistent TCP connections to peers. Network instability, ISP routing changes, or peer host reboots can drop all connections simultaneously without immediate self-healing.
  • IPv6 address stability — Yggdrasil derives each node's IPv6 address from its public key. As long as the key is stable, the address is permanent. But a key regeneration (intentional or from data loss) silently changes the node's address, breaking all references to it.
  • No built-in health API — Yggdrasil provides an admin socket (/var/run/yggdrasil.sock) and optional HTTP API, but neither sends alerts. You have to poll.
  • Services over Yggdrasil — applications running on Yggdrasil IPv6 addresses can fail independently of network health. A service crash is invisible to the mesh layer.

Complete monitoring requires checking node peer connectivity, the Yggdrasil daemon health, and application services running over the overlay.

What you'll need

  • A running Yggdrasil node (version 0.4+)
  • At least one configured peer
  • Services accessible over the Yggdrasil IPv6 address
  • A free Vigilmon account

Step 1: Expose health endpoints on Yggdrasil-hosted services

Services bound to Yggdrasil addresses use the node's 200::/7 IPv6 address range. Bind health endpoints to this address:

Go:

package main

import (
    "encoding/json"
    "net/http"
)

func main() {
    http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        json.NewEncoder(w).Encode(map[string]string{
            "status": "ok",
            "network": "yggdrasil",
        })
    })
    // Bind to Yggdrasil IPv6 address (replace with your node's actual address)
    http.ListenAndServe("[200:1234:abcd::1]:8080", nil)
}

Python / Flask:

from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/health')
def health():
    return jsonify(status='ok', network='yggdrasil')

if __name__ == '__main__':
    # Replace with your node's Yggdrasil address
    app.run(host='200:1234:abcd::1', port=8080)

Nginx (IPv6 listener):

server {
    listen [200:1234:abcd::1]:80;

    location /health {
        access_log off;
        return 200 '{"status":"ok"}';
        add_header Content-Type application/json;
    }

    location / {
        proxy_pass http://127.0.0.1:3000;
    }
}

Step 2: Monitor the Yggdrasil daemon and peer connectivity

Use the yggdrasilctl command-line tool (or the admin socket API) to check the node's current peer count. A node with zero peers is isolated:

#!/bin/bash
# yggdrasil-peer-probe.sh
# Monitors peer connectivity and pings Vigilmon

HEARTBEAT_URL="https://vigilmon.online/api/heartbeat/YOUR_PEER_HEARTBEAT"
MIN_PEERS=1
LOG_FILE="/var/log/yggdrasil-probe.log"

# Check the Yggdrasil daemon is running
if ! pgrep -x yggdrasil > /dev/null; then
  echo "$(date): Yggdrasil daemon not running" >> "$LOG_FILE"
  exit 1
fi

# Get peer count from the admin socket
peer_count=$(yggdrasilctl getPeers 2>/dev/null | python3 -c "
import json, sys
try:
    d = json.load(sys.stdin)
    peers = d.get('peers', {})
    connected = sum(1 for p in peers.values() if p.get('up', False))
    print(connected)
except:
    print(0)
" 2>/dev/null)

if [ -z "$peer_count" ] || [ "$peer_count" -lt "$MIN_PEERS" ]; then
  echo "$(date): Yggdrasil peer count is ${peer_count:-0} (minimum: $MIN_PEERS)" >> "$LOG_FILE"
  exit 1
fi

echo "$(date): $peer_count peers connected" >> "$LOG_FILE"
curl -s "$HEARTBEAT_URL" --max-time 10 --retry 2

Schedule every 2 minutes:

*/2 * * * * /opt/scripts/yggdrasil-peer-probe.sh

In Vigilmon: Monitors → New Monitor → Heartbeat, interval 5 minutes, name Yggdrasil Peer Connectivity. A node that loses all peers stops pinging and Vigilmon alerts.

Step 3: Monitor the Yggdrasil self-address reachability

Each Yggdrasil node should be able to ping its own overlay address (self-reachability confirms the interface is up and the crypto layer is functional):

#!/bin/bash
# yggdrasil-self-check.sh

HEARTBEAT_URL="https://vigilmon.online/api/heartbeat/YOUR_SELF_CHECK_HEARTBEAT"

# Get self address from yggdrasilctl
self_addr=$(yggdrasilctl getSelf 2>/dev/null | python3 -c "
import json, sys
try:
    d = json.load(sys.stdin)
    print(d.get('self', {}).get('address', ''))
except:
    print('')
" 2>/dev/null)

if [ -z "$self_addr" ]; then
  echo "$(date): Could not retrieve Yggdrasil self address" >> /var/log/yggdrasil-self.log
  exit 1
fi

# Ping self address to confirm interface + routing stack is healthy
if ! ping6 -c 2 -W 3 "$self_addr" > /dev/null 2>&1; then
  echo "$(date): Cannot ping self address $self_addr" >> /var/log/yggdrasil-self.log
  exit 1
fi

curl -s "$HEARTBEAT_URL" --max-time 10

Step 4: Probe services over Yggdrasil from another node

Since Vigilmon's global probes run on the public internet, they can't reach Yggdrasil addresses directly. Run a probe from another Yggdrasil node to verify end-to-end service reachability:

#!/bin/bash
# yggdrasil-service-probe.sh
# Run on a Yggdrasil node that should reach critical services

HEARTBEAT_URL="https://vigilmon.online/api/heartbeat/YOUR_SERVICE_HEARTBEAT"

# Services to check — use Yggdrasil IPv6 addresses
SERVICES=(
  "http://[200:1234:abcd::10]:8080/health"
  "http://[200:1234:abcd::20]:3000/health"
  "http://[200:1234:abcd::30]:5000/health"
)

all_ok=true

for svc in "${SERVICES[@]}"; do
  code=$(curl -s -o /dev/null -w "%{http_code}" --max-time 8 -6 "$svc")
  if [ "$code" != "200" ]; then
    echo "$(date): Service $svc returned $code" >> /var/log/yggdrasil-services.log
    all_ok=false
  fi
done

$all_ok && curl -s "$HEARTBEAT_URL" --max-time 10

Pair this with a Vigilmon Heartbeat monitor at 5-minute interval, name Yggdrasil Service Health.

Step 5: Monitor the Yggdrasil configuration and key integrity

Key regeneration silently changes a node's IPv6 address, breaking all existing references. Monitor for accidental key changes:

#!/bin/bash
# yggdrasil-key-check.sh
# Verifies the node's public key hasn't changed

HEARTBEAT_URL="https://vigilmon.online/api/heartbeat/YOUR_KEY_HEARTBEAT"
KNOWN_KEY_FILE="/etc/yggdrasil/known_pubkey.txt"

# Get current public key
current_key=$(yggdrasilctl getSelf 2>/dev/null | python3 -c "
import json, sys
try:
    d = json.load(sys.stdin)
    print(d.get('self', {}).get('key', ''))
except:
    print('')
" 2>/dev/null)

if [ -z "$current_key" ]; then
  echo "$(date): Could not retrieve public key" >> /var/log/yggdrasil-key.log
  exit 1
fi

# On first run, save the key as the baseline
if [ ! -f "$KNOWN_KEY_FILE" ]; then
  echo "$current_key" > "$KNOWN_KEY_FILE"
  echo "$(date): Baseline key saved: $current_key"
fi

known_key=$(cat "$KNOWN_KEY_FILE")

if [ "$current_key" != "$known_key" ]; then
  echo "$(date): PUBLIC KEY CHANGED — was $known_key, now $current_key" >> /var/log/yggdrasil-key.log
  exit 1
fi

curl -s "$HEARTBEAT_URL" --max-time 10

Run daily. A Heartbeat monitor with 25-hour interval will alert if the check stops (key changed or script failed).

Step 6: Monitor external peer reachability (for public Yggdrasil nodes)

If your node accepts incoming peer connections on a public port (typically TCP 443 or a custom port), verify that port is reachable:

In Vigilmon: Monitors → New Monitor → TCP Port

  • Host: your node's public hostname or IP
  • Port: your configured Yggdrasil listen port (e.g., 443)
  • Name: Yggdrasil Public Peer Port

This catches firewall rule drift, NAT configuration changes, and host unreachability before your peers notice the disconnect.

Step 7: Configure alerting

Structure escalation by failure severity:

  1. Peer Connectivity — immediate alert; zero peers = isolated node, all services unreachable
  2. Service Health — immediate for production services
  3. Key Integrity — immediate; key change breaks all existing references
  4. Self-Address Reachability — 2-minute delay; transient interface issues can self-resolve
  5. Public Peer Port — 5-minute delay; brief unreachability is common during reboots

In Vigilmon: Alert Channels → Add Channel → Slack (#yggdrasil-ops) and Email for the key change monitor specifically — that one warrants immediate human attention.

Step 8: Create an overlay network status page

Aggregate Yggdrasil health metrics into a shared status page:

  1. Status Pages → New Status Page
  2. Name: Yggdrasil Mesh Network
  3. Add monitors: Peer Connectivity, Self Reachability, Service Health, Public Peer Port
  4. Publish URL to #infra

Complete monitoring reference

| Monitor | Type | What it catches | |---------|------|-----------------| | Peer Connectivity | Heartbeat | Node isolated (zero peers) | | Self-Address Reachability | Heartbeat | Interface down, routing stack broken | | Service Health | Heartbeat | Application-layer failures | | Key Integrity | Heartbeat | Accidental key regeneration | | Public Peer Port | TCP | Node unreachable from internet |

Troubleshooting common Yggdrasil failures

  • Zero peers (node isolated) — check yggdrasilctl getPeers manually; if all peers show "up": false, verify TCP connectivity to the configured peer addresses. Common causes: peer host down, firewall blocking the Yggdrasil port, or ISP blocking outbound TCP 443.
  • Self-address ping fails — run ip -6 addr show to verify the Yggdrasil interface has an address in 200::/7; if the interface is missing, restart the daemon with systemctl restart yggdrasil.
  • Service unreachable over Yggdrasil — confirm the service is bound to the Yggdrasil IPv6 address (not [::] or 0.0.0.0); check ss -tlnp6 on the service host.
  • Key change detected — if accidental, restore from backup config and restart Yggdrasil; the old key's derived IPv6 address will return. Update the baseline key file only after confirming the new key is intentional.

Next steps

  • Multi-node dashboards — deploy the probe script on each Yggdrasil node and create separate heartbeat monitors per node for granular isolation detection
  • Bandwidth monitoring — parse yggdrasilctl getSelf traffic counters in the probe script and log throughput trends to detect relay saturation

Start monitoring your Yggdrasil mesh network today at vigilmon.online — free, no credit card required.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles