Takahe Monitoring with Vigilmon: Web Server, Stator Workers, ActivityPub & Federation Health

Takahe was designed to be simple to run. No Sidekiq, no Celery, no separate queue service — just the Django web process and the Stator background worker. An admin set it up for a small friend group and assumed that simplicity meant nothing could go silently wrong. Three days later, a friend on Mastodon reported they couldn't see any of the admin's posts. Stator had stopped after hitting an unhandled exception in a federation delivery task. No error pages, no user-visible symptoms — just silence on the Fediverse for 72 hours.

Takahe is a lightweight ActivityPub server designed for small communities and personal instances. Built on Python and Django, it replaces complex async infrastructure with a single background worker called Stator. That simplicity is a feature, not a limitation — but it also means Stator is the single point of failure for all federation, notifications, and scheduled maintenance. Vigilmon gives you external monitoring and alerting so you know the moment Stator stops or federation breaks.

What You'll Set Up

• HTTP uptime monitor for the Takahe web interface
• PostgreSQL database connectivity health check
• Stator background worker heartbeat monitor
• ActivityPub inbox delivery health check
• Outbox federation relay endpoint monitor
• Media storage (S3 or local) availability check
• WebFinger discovery endpoint monitor
• Push notification service health check
• SSL certificate expiry alerts

Prerequisites

• A running Takahe instance (v0.11+ recommended) on port 8000 or behind a reverse proxy
• A free Vigilmon account

Step 1: Monitor the Takahe Web Interface

Takahe's web interface serves the login page, profiles, and timeline. A healthy response confirms that Gunicorn and Django are running and the reverse proxy is forwarding traffic correctly.

Test it:

curl -I https://takahe.yourdomain.com/

You should see 200 OK with Content-Type: text/html.

Set up the Vigilmon monitor:

1. Log in to Vigilmon and click New Monitor → HTTP.
2. Set URL to https://takahe.yourdomain.com/.
3. Set Check interval to 60 seconds.
4. Set Expected status code to 200.
5. Save.

Takahe's login page is a reliable proxy for overall health — it queries the database for domain configuration on every render, so a 200 here also confirms basic database connectivity.

Step 2: Monitor PostgreSQL Database Connectivity

PostgreSQL is Takahe's only persistent store — it holds users, posts, follows, ActivityPub objects, domain configuration, and Stator task state. Takahe has no Redis or secondary cache layer, so a database outage causes a complete application failure immediately.

Takahe exposes a health endpoint that checks database connectivity. Check it directly:

curl https://takahe.yourdomain.com/api/v1/instance/

A healthy Takahe instance returns the instance metadata JSON from its database. This doubles as both an API health check and a database connectivity test.

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://takahe.yourdomain.com/api/v1/instance/.
3. Set Expected status code to 200.
4. Under Advanced → Keyword check, add keyword present: "domain".
5. Save.

For a more explicit DB check, Takahe supports a /healthcheck/ endpoint in recent versions. If yours exposes it:

1. Click New Monitor → HTTP.
2. Set URL to https://takahe.yourdomain.com/healthcheck/.
3. Set Expected status code to 200.
4. Save.

Step 3: Stator Background Worker Heartbeat Monitor

Stator is Takahe's built-in background processing system. It handles all ActivityPub federation deliveries, incoming activity processing, notification delivery, scheduled post sending, and maintenance tasks like cleaning up old objects. Stator runs as a separate process (manage.py stator) and is the single most critical non-web component.

When Stator stops, federation delivery stops immediately. Incoming activities queue up and are eventually dropped. Notifications are never delivered. The web interface remains perfectly functional, giving no indication anything is wrong.

Step 3a: Create the Vigilmon heartbeat monitor.

1. Click New Monitor → Heartbeat in Vigilmon.
2. Set Name to Takahe Stator Worker.
3. Set Expected heartbeat interval to 5 minutes.
4. Save. Copy the Heartbeat URL shown (e.g., https://vigilmon.online/heartbeat/abc123).

Step 3b: Configure a cron check for Stator.

Rather than modifying Takahe's source, use a cron job that checks whether the Stator process is alive and pings the heartbeat if so:

# /etc/cron.d/takahe-stator-heartbeat
*/5 * * * * takahe pgrep -f "stator" > /dev/null && curl -fsS https://vigilmon.online/heartbeat/YOUR_TOKEN

If you run Takahe with Docker Compose, check the container health instead:

*/5 * * * * root docker inspect --format='{{.State.Health.Status}}' takahe-stator | grep -q "healthy" && curl -fsS https://vigilmon.online/heartbeat/YOUR_TOKEN

Alternatively, add a Django management command that sends the heartbeat and schedule it through your process supervisor:

# takahe/management/commands/send_heartbeat.py
from django.core.management.base import BaseCommand
import urllib.request

class Command(BaseCommand):
    def handle(self, *args, **kwargs):
        urllib.request.urlopen("https://vigilmon.online/heartbeat/YOUR_TOKEN")

Then run it every 5 minutes via a second cron entry alongside Stator.

Step 4: Monitor ActivityPub Inbox Delivery Processing

Takahe's ActivityPub inbox accepts incoming activities from the Fediverse — follows, likes, boosts, replies, and DMs from remote users. The inbox endpoint at /inbox/ (per-user) and the shared inbox at /shared-inbox are critical for federation to work.

The shared inbox is the most important: it's the entry point for all bulk federation deliveries from remote servers.

Test it:

curl -I https://takahe.yourdomain.com/shared-inbox

An idle shared inbox returns 405 Method Not Allowed for GET requests (it only accepts POST from remote servers), which is the expected and correct response for monitoring purposes.

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://takahe.yourdomain.com/shared-inbox.
3. Set Expected status code to 405.
4. Set Check interval to 60 seconds.
5. Save.

A 404 or 500 here means Takahe's ActivityPub routing is broken and your instance can't receive any federated activities.

Step 5: Monitor Outbox Federation Relay Health

Takahe sends activities to remote servers by posting to their inboxes via Stator. While you can't monitor the remote delivery directly, you can verify that your outbox endpoint is healthy — remote servers query it to backfill your posts when a user first follows you.

Check a user's ActivityPub actor document, which links to the outbox:

curl -H "Accept: application/activity+json" https://takahe.yourdomain.com/@username@takahe.yourdomain.com/

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://takahe.yourdomain.com/@admin@takahe.yourdomain.com/.
3. Set Request header Accept to application/activity+json.
4. Set Expected status code to 200.
5. Under Keyword check, add keyword present: "outbox".
6. Save.

Step 6: Monitor WebFinger Discovery

WebFinger is how other Fediverse servers discover Takahe users. When someone on Mastodon searches for @user@takahe.yourdomain.com, Mastodon calls your WebFinger endpoint to look up the ActivityPub actor URL. A broken WebFinger means no new follows can be established from any remote server.

Test it:

curl "https://takahe.yourdomain.com/.well-known/webfinger?resource=acct:admin@takahe.yourdomain.com"

Healthy response:

{
  "subject": "acct:admin@takahe.yourdomain.com",
  "links": [
    { "rel": "self", "type": "application/activity+json", "href": "..." }
  ]
}

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://takahe.yourdomain.com/.well-known/webfinger?resource=acct:admin@takahe.yourdomain.com.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: subject.
5. Save.

Step 7: Monitor Media Storage Backend

Takahe stores post attachments and avatar images either in local filesystem storage or in an S3-compatible object store (configured via MEDIA_BACKEND). When media storage fails, post attachments return 404 errors and newly uploaded images aren't saved.

For S3-backed storage, verify the bucket is reachable by checking a known-good URL in your media bucket:

1. Upload a small test file to your S3 bucket (e.g., health-check.txt with content ok).
2. Make it publicly readable.
3. In Vigilmon, click New Monitor → HTTP.
4. Set URL to your test file's public URL (https://your-bucket.s3.amazonaws.com/health-check.txt).
5. Set Expected status code to 200.
6. Under Keyword check, add keyword present: ok.
7. Save.

For local storage, verify Nginx is serving the media directory:

1. Place a test file at MEDIA_ROOT/health-check.txt.
2. In Vigilmon, monitor https://takahe.yourdomain.com/media/health-check.txt.
3. Set Expected status code to 200.
4. Save.

Step 8: Monitor Push Notification Delivery

Takahe supports Web Push notifications for users with browsers that support the Push API. Stator handles push notification delivery. While push notification failures are less critical than federation delivery failures, they represent a user-experience gap.

Push notification delivery issues are usually caught by the Stator heartbeat monitor (Step 3) rather than a separate endpoint — if Stator is alive, push is likely working. However, if you use a dedicated push relay or VAPID endpoint, you can add a separate availability check:

1. Click New Monitor → HTTP.
2. Set URL to your VAPID or push relay endpoint.
3. Set Expected status code to 200 or 405 (depending on whether the endpoint accepts GET).
4. Save.

Step 9: Monitor Email Notification Service

Takahe sends email notifications for new follows, mentions, and DMs via Django's email backend (typically SMTP or a service like SendGrid). Email delivery failures are invisible — the user simply doesn't receive a notification.

Add an email health check route to Takahe. The simplest approach is a management command you can run periodically to send a test message to yourself:

# takahe/management/commands/check_email.py
from django.core.management.base import BaseCommand
from django.core.mail import send_mail

class Command(BaseCommand):
    def handle(self, *args, **kwargs):
        send_mail(
            subject="Takahe email health check",
            message="ok",
            from_email="noreply@takahe.yourdomain.com",
            recipient_list=["your-monitoring-inbox@example.com"],
            fail_silently=False,
        )

Run this daily from cron and track delivery to verify email is working. For more automated monitoring, use an email-to-heartbeat service that triggers a Vigilmon ping when the test message arrives.

Step 10: SSL Certificate Expiry Alerts

ActivityPub federation requires HTTPS. An expired certificate breaks federation immediately — remote servers won't deliver or accept activities over a broken TLS connection.

1. Open the web interface monitor from Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Save.

Alerting Configuration

Configure your notification channels in Vigilmon:

1. Go to Alerts → Notification Channels.
2. Add an Email channel with your admin address.
3. Add a Slack or Discord webhook if your community uses them.
4. Assign all Takahe monitors to these channels.

Recommended alert thresholds:

| Monitor | Priority | Alert after | |---|---|---| | Web interface | Critical | 2 consecutive failures | | PostgreSQL / instance API | Critical | 1 failure (immediate) | | Stator worker heartbeat | Critical | 1 missed heartbeat (5 min) | | Shared inbox (405 check) | High | 2 consecutive failures | | WebFinger discovery | High | 3 consecutive failures | | Actor outbox endpoint | High | 3 consecutive failures | | Media storage | Medium | 3 consecutive failures | | SSL certificate | High | 21 days before expiry |

Stator is the highest-priority alert after the web interface itself — it's responsible for all federation, notifications, and scheduled maintenance, and its failure is entirely invisible from the outside.

Conclusion

Takahe's minimalist design makes it one of the easiest ActivityPub servers to self-host. But simplicity doesn't eliminate failure modes — it just concentrates them. When Stator stops, everything that keeps your Fediverse presence alive stops with it, and nothing visible warns you.

With Vigilmon watching your web interface, PostgreSQL connectivity, Stator worker heartbeat, ActivityPub inbox and outbox endpoints, WebFinger discovery, media storage, and SSL certificate, you have a complete external view of your Takahe instance's health — so you know within minutes instead of days.

Start monitoring your Takahe instance for free at vigilmon.online.