tutorial

Monitoring Neptune.ai with Vigilmon: ML Experiment Tracking Endpoint Health

How to monitor Neptune.ai with Vigilmon — HTTP health checks on the Neptune API, dashboard availability, SSL certificate monitoring, and alerting for your ML experiment tracking infrastructure.

Neptune.ai is the cloud and self-hosted ML experiment tracking platform that logs metrics, parameters, artifacts, and model versions from PyTorch, TensorFlow, scikit-learn, and any Python ML framework. When Neptune's API endpoint goes down, your training runs silently lose experiment metadata — metrics stop logging, model checkpoints go untracked, and your team loses the comparison history needed to reproduce results. When the Neptune dashboard becomes unreachable, engineers cannot review ongoing runs or compare hyperparameter sweeps mid-training. Vigilmon gives you external uptime monitoring for Neptune.ai — API availability, dashboard health, SSL certificate validity, and alerting before your experiment tracking layer goes dark during critical training runs.

What You'll Build

  • An HTTP monitor on Neptune's API endpoint to detect ingestion failures during training runs
  • A monitor on the Neptune dashboard to catch web application outages
  • An SSL certificate monitor for your Neptune instance domain
  • Alerting rules that notify your team when experiment logging stops working

Prerequisites

  • A running Neptune.ai instance — self-hosted (https://neptune.example.com) or Neptune Cloud (https://app.neptune.ai)
  • The Neptune API endpoint network-reachable from Vigilmon probes
  • A free account at vigilmon.online

Step 1: Understand Neptune.ai's Architecture

Neptune.ai separates the API server that receives experiment data from the web dashboard that engineers use to review runs. The critical external-facing components are:

| Component | Default path | Role | |---|---|---| | API server | /api/ | Receives metrics, parameters, artifacts from ML training code | | Authentication endpoint | /api/auth/ | Validates API tokens for SDK connections | | Dashboard / frontend | / | Browser-accessible experiment comparison UI | | Artifact storage | /api/leaderboard/ | Stores and retrieves model artifacts, checkpoints |

For self-hosted Neptune, all components run under a single domain. For Neptune Cloud, the API runs at app.neptune.ai and data is stored in Neptune's managed infrastructure.

Step 2: Monitor the Neptune API Endpoint

The API server is the most critical component — it is the intake point for all experiment metrics, parameters, and artifacts logged by your ML training code. A liveness check on the API path confirms the service is accepting connections:

curl -I https://neptune.example.com/api/
# A 200 or 401 (unauthorized for unauthenticated GET) confirms the endpoint is alive
  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://neptune.example.com/api/ (or https://app.neptune.ai/api/ for Neptune Cloud).
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds.
  5. Expected status: 200 or 401 (unauthenticated requests return 401 — set the expected status to match your instance behavior).
  6. Label: Neptune API server.
  7. Click Save.

This monitor catches:

  • Neptune API container crashes
  • Kubernetes pod failures on the API deployment
  • Database connection failures preventing experiment ingestion
  • Out-of-memory kills on self-hosted Neptune deployments

Alert sensitivity: Set to trigger after 1 consecutive failure. A single API outage means every active training run stops logging data immediately, causing silent data loss.

Step 3: Monitor the Neptune Dashboard

The Neptune web frontend serves the experiment comparison UI that engineers use to review runs, compare metrics across experiments, and manage model versions. An HTTP check on the dashboard confirms the web layer is serving requests:

curl -I https://neptune.example.com
# Returns 200 with text/html content
  1. Add Monitor → HTTP.
  2. URL: https://neptune.example.com (or https://app.neptune.ai for Cloud).
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: Neptune (appears in the page title).
  7. Label: Neptune dashboard.
  8. Click Save.

Self-hosted note: On self-hosted deployments, dashboard availability depends on the same application server as the API. If the dashboard 404s or times out, investigate the full Neptune application stack — not just the frontend layer.

Step 4: Monitor the Authentication Endpoint

Neptune validates API tokens through an authentication endpoint before accepting experiment data. If the auth endpoint is degraded, new SDK connections fail even if the API server itself is running:

curl -I https://neptune.example.com/api/auth/
# Returns 200, 401, or 405 depending on the authentication flow
  1. Add Monitor → HTTP.
  2. URL: https://neptune.example.com/api/auth/.
  3. Check interval: 2 minutes.
  4. Response timeout: 10 seconds.
  5. Expected status: 200 or 401.
  6. Label: Neptune authentication endpoint.
  7. Click Save.

Why auth monitoring matters: Neptune SDKs authenticate once at connection time. If auth is down when a training run starts, the SDK raises a connection error and experiment tracking is silently disabled for that run — you get no metrics, parameters, or artifacts logged.

Step 5: Monitor SSL Certificates

Neptune's TLS certificate protects both the dashboard and the API endpoint used by training scripts. An expired certificate causes Python SDKs to raise SSL errors, stopping experiment logging across all your training pipelines:

openssl s_client -connect neptune.example.com:443 2>/dev/null | openssl x509 -noout -dates
  1. Add Monitor → SSL Certificate.
  2. Domain: neptune.example.com.
  3. Alert when expiry is within: 30 days.
  4. Alert again: 14 days, 7 days, 3 days, 1 day.
  5. Click Save.

Why SSL matters for Neptune specifically: Unlike web apps where users see a browser warning, an expired Neptune certificate breaks automated training scripts silently. GPU clusters running unattended overnight will fail to log any metrics, and you may not discover the loss until post-training analysis.

Step 6: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | Neptune API server | Non-200/401 response or timeout | Check Neptune backend pod; inspect experiment queue; review container logs | | Dashboard | Non-200 or keyword missing | Neptune frontend service degraded; check web container health | | Authentication endpoint | Unexpected status | SDK connections failing; new training runs cannot log data | | SSL certificate | < 30 days to expiry | Renew certificate; verify Python SDKs can still connect after renewal |

Alert after: 1 consecutive failure for the API server. 2 consecutive failures for dashboard and auth monitors.

Common Neptune.ai Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | API container OOM killed | API monitor fires; all active training runs stop logging silently | | Database connection pool exhausted | API returns 500; monitor fires within 60 s | | Disk full (artifact storage) | API returns 500 on artifact writes; API monitor fires | | SSL certificate expired | SSL monitor alerts at 30-day threshold; Python SDKs raise SSL errors | | Auth service misconfiguration | Auth monitor fires; new SDK connections rejected | | Kubernetes pod restart during training | API monitor fires; in-progress run metrics may be lost | | DNS cutover failure after migration | All monitors fire; training scripts cannot reach Neptune API | | High-throughput training event causes overload | API response time increases; Vigilmon timeout alert fires | | Frontend build failure after upgrade | Dashboard monitor fires; API may still accept data | | Network policy change blocks probe access | Monitors fire; teams alerted before training pipelines begin |

Monitoring Experiment Continuity vs. Service Availability

Vigilmon monitors whether Neptune's endpoints are reachable — it does not verify that experiments are being logged end-to-end within your training code. For deeper data quality monitoring:

  • Run heartbeat gaps: If a Neptune run shows no new metrics for an expected active experiment, check whether the API monitor is healthy and inspect SDK-level logs in your training script.
  • Artifact upload completeness: Neptune's internal storage shows upload progress — a degraded API combined with a healthy monitor status suggests a storage backend issue below the HTTP layer.
  • Project data consistency: After API recovery, verify that metrics logged during the outage window were either buffered by the SDK or are documented as lost — Neptune SDKs may not retroactively flush all buffered data after a reconnection.

Vigilmon catches infrastructure-level failures — endpoint unreachability, certificate expiry, service crashes. End-to-end experiment logging completeness requires Neptune's internal health metrics alongside external uptime monitoring.

Neptune.ai tracks every metric, parameter, and artifact your models produce, but your experiment tracking platform itself needs monitoring. A Neptune API outage silently stops logging for every active training run — you lose the comparison history you need to understand why a model improved or degraded. Vigilmon checks Neptune's API, dashboard, and SSL certificate on a 60-second cycle, so you know within a minute when your experiment tracking infrastructure stops collecting data.

Start monitoring Neptune.ai in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles