tutorial

How to Monitor Giskard with Vigilmon

Giskard scans your LLMs and ML models for vulnerabilities — but its scanning infrastructure needs its own monitoring. Here's how to monitor Giskard's API, CI/CD scan jobs, the Giskard Hub, and backing services with Vigilmon.

Giskard is the open-source ML model testing and scanning platform that automatically detects vulnerabilities in LLMs (hallucination, bias, toxicity, prompt injection) and traditional ML models (performance drift, data leakage, spurious correlations). It generates automated test suites and integrates into CI/CD pipelines as a quality gate — the last line of defense before a vulnerable model ships to production. When Giskard's scanning infrastructure goes down or a scan job fails silently, that quality gate stops blocking bad deployments. Vigilmon monitors Giskard's own uptime and scan execution cadence so your model quality gates stay active.

What You'll Set Up

  • HTTP uptime monitor for the Giskard Hub API
  • Cron heartbeat to verify model scan jobs are running in CI/CD
  • TCP port monitoring for Giskard's backing ML Worker service
  • SSL certificate monitoring for the Giskard Hub UI
  • Alert channel configuration for ML team on-call

Prerequisites

  • Giskard installed (self-hosted Hub or giskard Python client)
  • At least one ML model or LLM registered and scanned in Giskard
  • Giskard Hub accessible on a known URL (default port: 9000)
  • A free Vigilmon account

Step 1: Monitor the Giskard Hub API

The Giskard Hub is the central service your Python client and CI/CD pipeline communicate with. If it goes down, model uploads fail, scan results can't be stored, and the test suite UI is inaccessible to your team.

  1. Log in to vigilmon.online and click Add Monitor.
  2. Set Type to HTTP / HTTPS.
  3. Enter your Giskard Hub URL: https://giskard.yourdomain.com/api/v2/settings (the settings endpoint returns a 200 and is safe to probe repeatedly).
  4. Set Check interval to 1 minute.
  5. Set Expected HTTP status to 200.
  6. Click Save.

For local Giskard Hub instances not behind a reverse proxy, use http://your-server:9000/api/v2/settings.

Step 2: Add a Heartbeat to Verify CI/CD Scan Jobs

Giskard's most valuable capability is blocking bad model deployments in CI/CD. A scan job that crashes silently — or is skipped because a required service is down — passes the pipeline with no scan output. This is the most dangerous failure mode: the gate appears to pass while doing nothing.

  1. In Vigilmon, click Add Monitor → Cron Heartbeat.
  2. Set the expected ping interval to match your CI/CD pipeline frequency (e.g. 1440 minutes for daily model training runs, or 60 for hourly evaluation sweeps).
  3. Copy the heartbeat URL.
  4. Add the heartbeat ping at the end of your Giskard scan step:
# ci_model_scan.py
import giskard
import requests
import os

model = giskard.Model(
    model=my_classifier,
    model_type="classification",
    name="fraud-detector-v3",
    feature_names=FEATURE_NAMES,
)
dataset = giskard.Dataset(df=test_df, target="label")

scan_results = giskard.scan(model, dataset)

if scan_results.has_vulnerabilities:
    scan_results.to_html("scan_report.html")
    raise SystemExit("Giskard scan detected vulnerabilities — blocking deployment")

# Confirm successful scan to Vigilmon
requests.get(os.getenv("VIGILMON_HEARTBEAT_URL"), timeout=5)

Add VIGILMON_HEARTBEAT_URL as a CI/CD secret. If the scan crashes before the heartbeat line — or if the entire CI job is never triggered — Vigilmon alerts after the expected interval.

Step 3: Monitor the Giskard ML Worker

The Giskard ML Worker is a separate process that handles model inference during scans. If the ML Worker goes down, the Hub remains accessible but all scan executions will fail or hang. This failure mode passes an HTTP uptime check while breaking the core functionality.

  1. In Vigilmon, click Add Monitor → TCP Port.
  2. Enter your Giskard server's hostname and port 50051 (the ML Worker's gRPC port).
  3. Set Check interval to 1 minute.
  4. Click Save.

For Docker Compose deployments, the ML Worker runs in its own container. A TCP monitor on port 50051 directly confirms the gRPC server is accepting connections.

To also verify the ML Worker from within Python:

# worker_health.py
import grpc

def check_ml_worker(host: str = "localhost", port: int = 50051) -> bool:
    try:
        channel = grpc.insecure_channel(f"{host}:{port}")
        grpc.channel_ready_future(channel).result(timeout=3)
        return True
    except grpc.FutureTimeoutError:
        return False

Expose this as an HTTP health endpoint (similar to the TruLens pattern) and add a second Vigilmon HTTP monitor for belt-and-suspenders coverage.

Step 4: SSL Certificate Alerts for the Giskard Hub

Your team accesses the Giskard Hub UI and your CI/CD pipelines upload models over HTTPS. A lapsed certificate breaks both.

  1. Open the HTTP monitor from Step 1.
  2. Enable Monitor SSL certificate under the SSL section.
  3. Set Alert when certificate expires in less than 21 days.
  4. Click Save.

Step 5: Monitor the Giskard Database

Giskard Hub uses PostgreSQL to store scan results, model metadata, and test suite definitions. If the database goes down, model uploads fail and scan history is inaccessible.

  1. In Vigilmon, click Add Monitor → TCP Port.
  2. Enter your PostgreSQL host and port 5432.
  3. Set Check interval to 1 minute.

For Docker Compose deployments, the Giskard docker-compose.yml starts a postgres service. Monitor the host machine's port that maps to the container's 5432.

Step 6: Configure Alert Channels

  1. Go to Alert Channels in Vigilmon and add Slack, email, or a webhook.
  2. Heartbeat alerts should go to your ML/MLOps on-call channel with high urgency — a missed scan heartbeat means a deployment is shipping without a vulnerability check.
  3. Hub and ML Worker alerts should page the DevOps team immediately — the scan infrastructure is down.
  4. Database and SSL alerts can go to a lower-priority ops channel.
  5. Use Maintenance windows during Giskard Hub upgrades:
curl -X POST https://vigilmon.online/api/maintenance \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{"monitor_id": "abc123", "duration_minutes": 30}'

Set a longer window for Hub upgrades — Giskard's database migrations can take several minutes on large scan history tables.

Summary

| Monitor | Target | What It Catches | |---|---|---| | HTTP uptime (Hub API) | /api/v2/settings | Hub unreachable | | Cron heartbeat | Heartbeat URL | CI/CD scan job skipped or crashed | | TCP port (ML Worker) | :50051 gRPC | Model inference worker down | | TCP port (PostgreSQL) | :5432 | Scan result storage failure | | SSL certificate | Hub domain | TLS cert expiry |

Giskard is your last line of defense against vulnerable models shipping to production — but that defense is only as good as its own availability. With Vigilmon watching the Hub, the ML Worker, scan execution cadence, and backing database, you ensure your model quality gates are always active when a new model is about to deploy.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles