tutorial

Monitoring Castor with Vigilmon: Automated Data Catalog Health, API Availability & Documentation Delivery

How to monitor Castor automated data catalog and documentation platform with Vigilmon — application uptime, API availability, connector health, and SSL certificate validity.

Castor is the automated data catalog and data documentation platform that discovers, documents, and governs your data assets — automatically crawling data warehouses, dbt projects, and BI tools to populate a searchable catalog of tables, columns, dashboards, and metrics, then surfacing ownership, usage patterns, and lineage context that data teams need to trust the data they use. When the Castor application becomes unavailable, analysts cannot search for trusted datasets, data engineers lose access to lineage and impact analysis tools, and data governance workflows that route documentation requests through Castor stop functioning. When the Castor API is degraded, integrations with dbt Cloud, Snowflake, BigQuery, and Looker stop syncing fresh metadata and the catalog grows stale. Vigilmon gives you external visibility into the Castor infrastructure: the application platform, the API endpoints, and your connector sync status, so you know the moment the data catalog itself needs attention.

What You'll Build

  • An HTTP monitor on the Castor application to detect platform availability failures
  • A health-check monitor on the Castor API to catch backend degradation
  • A monitor on your webhook receiver to verify Castor notification delivery is flowing
  • A monitor on your Castor connector sync endpoint to catch catalog staleness
  • An SSL certificate monitor for your Castor domain
  • Alerting that distinguishes platform outages from API and catalog sync failures

Prerequisites

  • A Castor account (cloud-hosted SaaS) with the application accessible via HTTPS
  • API credentials for Castor if using the REST API for integrations
  • A webhook receiver endpoint configured to accept Castor notifications (if applicable)
  • A free account at vigilmon.online

Step 1: Understand Castor's Service Architecture

Castor operates as a SaaS data catalog platform with several monitorable components:

| Component | Default port | Role | |---|---|---| | Castor web application | 443 (HTTPS) | Primary interface for searching, browsing, and documenting data assets | | Castor REST API | 443 (HTTPS) | Programmatic access to catalog metadata, documentation, and lineage data | | Connector sync engine | Outbound | Crawls connected data sources (Snowflake, BigQuery, dbt, Looker) to refresh catalog | | Notification dispatcher | Outbound | Delivers documentation request alerts and catalog change notifications |

External monitoring focuses on the web application and REST API — these surface the availability of the entire Castor platform and are the interfaces your data team uses to discover, document, and govern data assets.

Step 2: Monitor the Castor Web Application

The Castor web application is the primary interface for data teams — analysts search for trusted datasets, data engineers trace lineage to assess the impact of schema changes, data owners review and complete documentation requests, and governance teams audit data asset ownership and access patterns:

curl -I https://app.castordoc.com
# Returns HTTP 200 with HTML when the Castor application is healthy
  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://app.castordoc.com (or your Castor workspace URL).
  3. Check interval: 60 seconds.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: Castor (appears in the page title and application HTML).
  7. Click Save.

This monitor catches:

  • Castor platform deployment failures or infrastructure outages on the SaaS side
  • Application load balancer misconfigurations returning 5xx errors
  • DNS resolution failures for your Castor workspace domain
  • Castor service degradation that renders the catalog unsearchable

Alert sensitivity: Set to trigger after 1 consecutive failure. When the Castor application is unavailable, analysts cannot find trusted datasets for reports, data engineers cannot check lineage before making breaking schema changes, and documentation requests cannot be submitted or approved — stopping data governance workflows cold.

Step 3: Monitor the Castor API

The Castor REST API powers programmatic catalog access — fetching asset metadata, querying lineage graphs, updating documentation programmatically, and syncing catalog content into downstream tools like internal developer portals or BI dashboards. When the API degrades, automated catalog integrations fail, dbt documentation syncs stop updating the catalog with fresh model descriptions, and any custom tooling that reads Castor metadata returns stale or empty results:

curl https://api.castordoc.com/v1/health \
  -H "Authorization: Bearer YOUR_API_TOKEN"
# Returns: {"status": "ok"} when the API backend is operational
  1. Add Monitor → HTTP.
  2. URL: https://api.castordoc.com/v1/health.
  3. Check interval: 60 seconds.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: ok (present in healthy API responses).
  7. Label: Castor API health.
  8. Click Save.

Why monitor the API separately from the application? The Castor web application may continue to render catalog search results from a cached index even when the API cannot process new requests. A failing API means dbt Cloud cannot push fresh documentation into the catalog after each run, connector syncs cannot write newly discovered assets, and any CI/CD pipeline that reads Castor metadata to enforce documentation coverage will receive empty or stale responses.

Step 4: Monitor Your Webhook Receiver

Castor sends notifications for documentation requests, ownership assignments, catalog change alerts, and data quality flags to configured webhook receivers. If your receiver becomes unavailable or returns errors, notification delivery fails silently — data owners stop receiving documentation requests, engineers miss alerts about schema changes discovered by the catalog crawler, and governance workflows stall because the routing mechanism is broken:

curl https://your-webhook-receiver.example.com/health
# Returns: {"status": "ok"} when the receiver is up
  1. Add Monitor → HTTP.
  2. URL: https://your-webhook-receiver.example.com/health.
  3. Check interval: 2 minutes.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: ok.
  7. Label: Castor notification webhook.
  8. Click Save.

For Slack integrations where Castor posts catalog change notifications, monitor https://slack.com/api/api.test to confirm the Slack API is reachable. For teams that route Castor governance notifications through a custom service, ensure that service's health endpoint is monitored so routing failures are caught before a documentation backlog accumulates.

Step 5: Monitor the Connector Sync Endpoint

Castor's catalog value depends entirely on connector syncs staying current — without fresh crawls of your Snowflake schemas, BigQuery datasets, dbt models, and Looker dashboards, the catalog drifts out of sync with your actual data stack. While you cannot directly monitor Castor's internal crawlers, you can monitor the endpoint that receives sync triggers or the API endpoint that reports last-sync timestamps:

curl https://api.castordoc.com/v1/integrations/status \
  -H "Authorization: Bearer YOUR_API_TOKEN"
# Returns integration health and last sync timestamps
  1. Add Monitor → HTTP.
  2. URL: https://api.castordoc.com/v1/integrations/status.
  3. Check interval: 10 minutes.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: status (present in the integration response).
  7. Label: Castor connector sync.
  8. Click Save.

Catalog staleness is an invisible failure: When connector syncs stop running, the Castor catalog continues to show the last successfully crawled state. Analysts searching for datasets find correct results for assets that existed before the sync stopped, but newly added tables, columns, and dashboards are missing — causing teams to miss trusted data sources or continue using deprecated assets that have been replaced.

Step 6: Monitor SSL Certificates

The Castor web application and REST API are accessed over HTTPS by both data team members using the browser and automated integrations (dbt Cloud pipelines, custom tooling) that call the API on scheduled runs. An expired certificate breaks browser access to the catalog and causes all API calls from automated systems to fail with TLS errors:

openssl s_client -connect app.castordoc.com:443 2>/dev/null | openssl x509 -noout -dates
  1. Add Monitor → SSL Certificate.
  2. Domain: app.castordoc.com.
  3. Alert when expiry is within: 30 days.
  4. Alert again: 14 days, 7 days, 3 days, 1 day.
  5. Click Save.

Governance workflow impact: If the Castor application's TLS certificate expires, data governance workflows that route through the Castor UI — documentation requests, ownership transfers, access reviews — become inaccessible to the stakeholders responsible for acting on them. Governance backlogs accumulate silently until the certificate is renewed, with no automated indication that the approval queue is blocked.

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | Web application | Non-200 or Castor missing | Check Castor status page; verify platform availability | | API health | Non-200 or ok missing | API down; programmatic integrations failing; check Castor status | | Notification webhook | Non-200 or ok missing | Governance alerts not delivering; verify receiver deployment | | Connector sync | Non-200 or status missing | Integration status unclear; verify sync schedules in Castor admin | | SSL certificate | < 30 days to expiry | Renew certificate; verify automated integrations reconnect |

Alert after: 1 consecutive failure for the web application and API health monitors. 2 consecutive failures for the notification webhook and connector sync monitors to reduce noise from transient network timeouts during crawl windows.

Common Castor Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Castor platform outage | Application monitor fires; catalog search unavailable | | REST API degradation | API monitor fires; dbt docs sync fails; custom integrations break | | Notification webhook receiver crash | Webhook monitor fires; documentation and governance alerts stop posting | | Connector sync failure | Sync monitor fires; catalog grows stale; new assets go undiscovered | | TLS certificate expired | SSL monitor alerts; browser and API client access fails | | DNS misconfiguration | All monitors fire simultaneously | | Authentication service degradation | Application may return 401 or login errors; API monitor fires | | Catalog search index failure | Application returns 200 but search results are empty or degraded |

Monitoring Data Documentation Quality (Beyond Infrastructure)

Vigilmon monitors Castor's infrastructure endpoints — the web application, REST API, notification webhook, and connector sync status. The quality of catalog documentation and coverage across your data assets is a separate concern managed within Castor itself:

  • Documentation coverage: Track the percentage of tables and columns with owner assignments, descriptions, and sensitivity tags in Castor to identify undocumented assets before they proliferate.
  • Connector health within Castor: Review the connector sync logs within the Castor admin panel to catch partial sync failures that may not surface as API errors — for example, a BigQuery connector that syncs successfully but only crawls a subset of the expected datasets.
  • Documentation request routing: Audit the documentation request queue regularly to ensure requests are being assigned to and resolved by the correct data owners, not accumulating unaddressed.

Vigilmon catches infrastructure-level failures — when Castor itself is down or its API is unreachable. Documentation completeness and catalog accuracy within a healthy Castor deployment require active curation and connector configuration by your data governance team.

Castor is the layer that makes your data assets findable, documented, and governed — when it goes down, analysts cannot discover trusted data, engineers lose lineage context for impact analysis, and data governance workflows stall without the platform that routes them. Vigilmon gives you external visibility into Castor's availability that doesn't depend on Castor self-reporting: application uptime, API health, connector sync status, and SSL certificate validity, so you know the moment the data catalog itself needs attention and can restore discoverability before the next analytics project hits a dead end searching for trustworthy data.

Start monitoring Castor in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles