Bookwyrm Monitoring with Vigilmon: Web Server, Celery Workers, Federation & Book Data APIs

A Bookwyrm community manager noticed members complaining that book covers weren't loading. After digging in, they found the Celery worker had run out of file descriptors four hours earlier and stopped processing all async tasks — not just cover fetches, but also outgoing ActivityPub deliveries, email notifications, and federation sync jobs. The web interface was fully functional. There were no error pages. Without a worker heartbeat monitor, the failure was completely invisible.

Bookwyrm is a federated social network for book readers and reviewers, offering an ActivityPub-compatible alternative to Goodreads. Built on Python and Django, it relies on a Celery task queue backed by Redis to handle federation, notifications, media processing, and external book data fetches from OpenLibrary and Inventaire. Keeping all of these working together is the real challenge of self-hosting Bookwyrm. Vigilmon gives you external uptime monitoring and alerting across every layer of the stack.

What You'll Set Up

• HTTP uptime monitor for the Bookwyrm web interface
• API health check on the ActivityPub instance metadata endpoint
• Celery worker heartbeat monitor for async task processing
• Redis cache and queue broker connectivity check
• PostgreSQL database health check
• OpenLibrary external API integration health monitor
• Media file storage availability check
• ActivityPub federation endpoint monitors
• SSL certificate expiry alerts

Prerequisites

• A running Bookwyrm instance on port 8000 (or behind Nginx/Caddy)
• Redis running on its default port (6379)
• A free Vigilmon account

Step 1: Monitor the Bookwyrm Web Interface

Bookwyrm's homepage is the first thing users and external federation crawlers see. A healthy response here confirms that Gunicorn is serving Django, the database is reachable, and the reverse proxy is functioning.

Test it:

curl -I https://bookwyrm.yourdomain.com/

You should see 200 OK with Content-Type: text/html.

Set up the Vigilmon monitor:

1. Log in to Vigilmon and click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/.
3. Set Check interval to 60 seconds.
4. Set Expected status code to 200.
5. Under Advanced → Keyword check, add keyword present: BookWyrm.
6. Save.

Step 2: Monitor the ActivityPub Instance Metadata Endpoint

Bookwyrm exposes ActivityPub-compatible instance metadata at /api/v1/instance. This endpoint returns the instance name, description, version, and admin contact. It's queried by federation peers when they first discover your server, and a failure here means remote servers can't establish a relationship with your instance.

Test it:

curl https://bookwyrm.yourdomain.com/api/v1/instance

Healthy response:

{
  "uri": "bookwyrm.yourdomain.com",
  "title": "My Bookwyrm",
  "version": "0.7.x",
  "stats": { "user_count": 25 }
}

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/api/v1/instance.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: "title", keyword absent: "error".
5. Save.

Step 3: Celery Worker Heartbeat Monitor

Celery workers handle everything asynchronous in Bookwyrm: sending ActivityPub activities to remote servers, fetching book covers from OpenLibrary, delivering email notifications, processing incoming federation messages, and running scheduled sync jobs. When Celery stops — due to Redis connectivity loss, a memory limit being hit, or an uncaught exception in a long-running task — the web interface continues working normally but all async operations silently stop.

Step 3a: Create the Vigilmon heartbeat monitor.

1. Click New Monitor → Heartbeat in Vigilmon.
2. Set Name to Bookwyrm Celery Worker.
3. Set Expected heartbeat interval to 5 minutes.
4. Save. Copy the Heartbeat URL shown (e.g., https://vigilmon.online/heartbeat/abc123).

Step 3b: Configure Celery to send the heartbeat.

Add a periodic Celery beat task that pings the Vigilmon heartbeat URL:

# bookwyrm/tasks.py  (add alongside existing task definitions)
from celery import shared_task
import requests

@shared_task
def vigilmon_heartbeat():
    try:
        requests.get("https://vigilmon.online/heartbeat/YOUR_HEARTBEAT_TOKEN", timeout=5)
    except Exception:
        pass

Register it in your Celery beat schedule in settings.py:

CELERY_BEAT_SCHEDULE = {
    # ... existing schedule entries ...
    "vigilmon-heartbeat": {
        "task": "bookwyrm.tasks.vigilmon_heartbeat",
        "schedule": 300,  # every 5 minutes
    },
}

Restart Celery beat after making this change:

docker compose restart celery_beat
# or: systemctl restart bookwyrm-celery-beat

If any component in the Celery chain — the worker process, beat scheduler, or Redis connection — fails, the heartbeat ping stops. Vigilmon alerts you after one missed interval.

Step 4: Monitor Redis Cache and Queue Broker

Redis serves two roles in Bookwyrm: as a cache backend for Django views and as the message broker for Celery tasks. An unhealthy Redis instance causes Celery to stop accepting and dispatching tasks immediately, and degrades web performance as cache misses hit the database directly.

Add a lightweight health route to Bookwyrm's Django app, or use a Redis PING check via a sidecar endpoint:

# bookwyrm/views/health.py
from django.http import JsonResponse
from django.core.cache import cache

def redis_health(request):
    try:
        cache.set("health_check", "ok", timeout=10)
        value = cache.get("health_check")
        return JsonResponse({"status": "ok" if value == "ok" else "error"})
    except Exception as e:
        return JsonResponse({"status": "error", "detail": str(e)}, status=500)

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/health/redis.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: ok.
5. Save.

Step 5: Monitor PostgreSQL Database Connectivity

Bookwyrm's PostgreSQL database stores all books, reviews, shelves, users, follows, ActivityPub objects, and federation records. A database outage makes every page return a 500 error.

Add a database health check view:

# bookwyrm/views/health.py
from django.db import connection
from django.http import JsonResponse

def db_health(request):
    try:
        with connection.cursor() as cursor:
            cursor.execute("SELECT 1")
        return JsonResponse({"status": "ok"})
    except Exception as e:
        return JsonResponse({"status": "error", "detail": str(e)}, status=500)

Wire it up in bookwyrm/urls.py:

path("health/db", views.health.db_health),

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/health/db.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: ok.
5. Save.

Step 6: Monitor OpenLibrary External API Integration

Bookwyrm fetches book metadata (titles, ISBNs, author names, cover images) from external APIs: primarily OpenLibrary and optionally Inventaire. When the OpenLibrary API is slow or returning errors, Bookwyrm's book search degrades and cover image fetches fail. Users can still browse and review existing books, but adding new books becomes broken.

Monitor the OpenLibrary API health directly by checking its status endpoint:

1. Click New Monitor → HTTP in Vigilmon.
2. Set URL to https://openlibrary.org/api/books?bibkeys=ISBN:9780140449136&format=json.
3. Set Check interval to 5 minutes.
4. Set Expected status code to 200.
5. Under Keyword check, add keyword present: ISBN.
6. Save.

This confirms that the OpenLibrary API is reachable and returning valid data. If this monitor goes down, book search and metadata import in Bookwyrm will be degraded until OpenLibrary recovers. This is useful for distinguishing between your own outage and an upstream provider outage.

Step 7: Monitor ActivityPub Federation Endpoints

Bookwyrm's ActivityPub implementation lets users follow and be followed by accounts on Mastodon, Pixelfed, and other Fediverse servers. The WebFinger endpoint is the discovery mechanism; a failure here means no new federation relationships can be established.

Check the WebFinger endpoint:

curl "https://bookwyrm.yourdomain.com/.well-known/webfinger?resource=acct:user@bookwyrm.yourdomain.com"

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/.well-known/webfinger?resource=acct:admin@bookwyrm.yourdomain.com.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: subject.
5. Save.

Also check the NodeInfo endpoint, which federation peers use to discover your server's software and capabilities:

1. Click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/.well-known/nodeinfo.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: links.
5. Save.

Step 8: Monitor Media File Storage

Bookwyrm stores cover images and user avatar uploads either locally (in a media/ directory) or in an S3-compatible object store. A storage failure causes all cover images to return 404 errors — a highly visible degradation for a book-focused community.

For local storage, add a health check that reads a known test file:

def storage_health(request):
    from django.core.files.storage import default_storage
    try:
        # Write and read back a small test object
        default_storage.save("health_check.txt", ContentFile(b"ok"))
        content = default_storage.open("health_check.txt").read()
        default_storage.delete("health_check.txt")
        return JsonResponse({"status": "ok" if content == b"ok" else "error"})
    except Exception as e:
        return JsonResponse({"status": "error"}, status=500)

In Vigilmon:

1. Click New Monitor → HTTP.
2. Set URL to https://bookwyrm.yourdomain.com/health/storage.
3. Set Expected status code to 200.
4. Under Keyword check, add keyword present: ok.
5. Save.

Step 9: SSL Certificate Expiry Alerts

HTTPS is required for ActivityPub federation. An expired certificate breaks federation with all remote servers immediately.

1. Open the web interface monitor from Step 1.
2. Enable Monitor SSL certificate.
3. Set Alert when certificate expires in less than 21 days.
4. Save.

Alerting Configuration

Configure notification channels in Vigilmon for immediate alerting:

1. Go to Alerts → Notification Channels.
2. Add an Email channel with your admin address.
3. Optionally add a Slack or Discord webhook for your community.
4. Assign all Bookwyrm monitors to your notification channels.

Recommended alert thresholds:

| Monitor | Priority | Alert after | |---|---|---| | Web interface | Critical | 2 consecutive failures | | ActivityPub instance endpoint | High | 2 consecutive failures | | PostgreSQL health | Critical | 1 failure (immediate) | | Redis health | Critical | 1 failure | | Celery worker heartbeat | Critical | 1 missed heartbeat (5 min) | | WebFinger / NodeInfo | High | 3 consecutive failures | | OpenLibrary API | Medium | 5 consecutive failures | | Media storage | High | 3 consecutive failures | | SSL certificate | High | 21 days before expiry |

PostgreSQL, Redis, and the Celery worker heartbeat are your highest-priority alerts — any of these failures causes immediate or near-immediate user-visible breakage. OpenLibrary is a lower priority because it's an external dependency you can't control and its failures are usually brief.

Conclusion

Bookwyrm's charm is its community-focused reading experience, but the async infrastructure underneath — Celery workers fetching covers, processing federation, delivering notifications — is where invisible failures live. The web interface looks healthy right up until a user tries to add a book or follow someone on Mastodon.

With Vigilmon watching your web server, Celery workers, Redis, PostgreSQL, ActivityPub endpoints, OpenLibrary integration, and media storage, you'll catch failures within minutes and keep your reading community running smoothly.

Start monitoring your Bookwyrm instance for free at vigilmon.online.