tutorial

Monitoring Rivery with Vigilmon: ELT Pipeline Health, Orchestration API & Cloud Data Integration Availability

How to monitor Rivery cloud-native ELT and data orchestration pipelines with Vigilmon — API availability, River execution health, webhook receiver monitoring, and SSL certificate validity.

Rivery is a cloud-native ELT and data orchestration platform that ingests, transforms, and orchestrates data across your entire data stack — pulling from hundreds of SaaS sources, APIs, and databases, loading into cloud data warehouses like Snowflake, BigQuery, and Redshift, and orchestrating the full pipeline graph through a visual workflow engine. Rivery's architecture is fully managed and serverless: there are no cluster configurations to manage, no Spark jobs to tune, and no infrastructure to maintain — you define Rivers (individual data pipeline units) and Flows (orchestrated sequences of Rivers), and Rivery handles scheduling, retries, compute allocation, and incremental data loading. When the Rivery API becomes unavailable, scheduled Rivers cannot execute, Flow orchestrations cannot be triggered, and any automation that depends on completion webhooks loses its synchronization signal. When a River's source connection credentials expire or a destination endpoint becomes unreachable, that River fails silently until someone notices that the data warehouse table stopped updating. When the webhook receiver that captures River completion or failure events is down, your downstream dbt runs, dashboard refreshes, and data quality checks proceed without knowing whether the upstream data actually landed. Vigilmon gives your data engineering team external HTTP visibility into the Rivery surfaces that keep ELT pipelines healthy — the API control plane, the River execution endpoint, your source and destination connections, and your completion webhook receivers — so you catch failures before they silently stall the data flow your dashboards and ML models depend on.

What You'll Build

  • An HTTP monitor on the Rivery API to detect platform outages before they block River execution and Flow orchestration
  • A monitor on the River management endpoint to catch degradation in pipeline execution status reporting
  • A monitor on your completion webhook receiver to verify River success and failure notifications are being delivered
  • A monitor on your Rivery workspace domain to detect access failures during incident response
  • An SSL certificate monitor for your Rivery workspace
  • Alerting that distinguishes API outages from River execution failures and webhook delivery breakdowns

Prerequisites

  • A Rivery account with one or more Rivers and Flows configured
  • A Rivery API key for programmatic access to River status and execution history
  • An external health endpoint or webhook receiver that captures Rivery pipeline events
  • A free account at vigilmon.online

Step 1: Understand Rivery's Infrastructure Architecture

Rivery ELT pipelines run on fully managed cloud infrastructure, but several surfaces are externally monitorable via the Rivery REST API and your integration layer:

| Component | Surface | Role | |---|---|---| | Rivery API | HTTPS REST (api.rivery.io) | Central control plane; manages Rivers, Flows, connections | | River execution endpoint | /api/v1/rivers | Lists Rivers, triggers runs, reports execution state | | Flow orchestration endpoint | /api/v1/flows | Manages orchestrated pipeline sequences and schedules | | Connection health surface | /api/v1/connections | Reports on source and destination credential validity | | River completion webhook | Webhook / Slack / PagerDuty | Delivers run success, failure, and warning events | | Rivery web console | HTTPS | Engineer access to pipeline graphs, logs, and run history |

External monitoring focuses on the Rivery API availability, the River execution surface, and your webhook receivers — these are where ELT failures become externally observable before they propagate into stale warehouse tables and broken downstream dependencies. An API outage halts all River scheduling; a source credential expiry silently fails individual River runs; a downed webhook receiver leaves dbt, Looker, and your data quality platform scheduling runs against data that never updated.

Step 2: Monitor the Rivery API

The Rivery REST API is the control plane for all ELT operations. Every River trigger, status query, Flow orchestration request, and connection validation flows through api.rivery.io. When it becomes unavailable or returns errors, scheduled River executions cannot be triggered via API, external orchestration tools cannot read execution status, and any downstream system depending on Rivery completion signals stalls:

# Test Rivery API availability
curl -H "Authorization: Bearer $RIVERY_API_KEY" \
  https://api.rivery.io/api/v1/rivers
# Returns: {"rivers": [...]} when the API is healthy

# Verify environment context
curl -H "Authorization: Bearer $RIVERY_API_KEY" \
  https://api.rivery.io/api/v1/account
# Returns account details; a 200 confirms authentication and API availability
  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: https://api.rivery.io/api/v1/rivers.
  3. Check interval: 2 minutes.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: rivers (present in every successful River list response).
  7. Label: Rivery API.
  8. Click Save.

This monitor catches:

  • Full Rivery platform outages that halt all River and Flow scheduling
  • Infrastructure incidents affecting the Rivery API control plane
  • Authentication service failures that cause all API requests to return 401
  • Network path changes that have cut off your monitoring infrastructure from the Rivery API

Alert sensitivity: Set to trigger after 1 consecutive failure. A Rivery API outage means no River can be triggered via API, no execution status can be queried, and no Flow orchestration can be managed from any external system.

Step 3: Monitor the River Execution Endpoint

The River execution endpoint is the specific surface that tracks individual pipeline run state — running, completed, failed, or queued. External orchestration tools, dbt run triggers, dashboard refresh automations, and incident runbooks all depend on this endpoint to understand whether the upstream ELT data actually landed before scheduling downstream processing:

# List Rivers and their configuration
curl -H "Authorization: Bearer $RIVERY_API_KEY" \
  https://api.rivery.io/api/v1/rivers
# Returns: {"rivers": [{"river_id": "...", "name": "...", "type": "..."}]}

# Get execution history for a specific River
curl -H "Authorization: Bearer $RIVERY_API_KEY" \
  https://api.rivery.io/api/v1/rivers/YOUR_RIVER_ID/runs
# Returns run history with status, start time, end time, and error details
  1. Add Monitor → HTTP.
  2. URL: https://api.rivery.io/api/v1/rivers.
  3. Check interval: 3 minutes.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: rivers.
  7. Label: Rivery River execution endpoint.
  8. Click Save.

Why monitor the execution endpoint separately from the general API? Rivery may deploy partial degradation where the account endpoint responds normally while the River management surface encounters errors. The execution endpoint can be degraded while the API appears broadly healthy, leaving pipeline status invisible to external orchestration tools and dbt cloud run triggers.

Step 4: Monitor the Flow Orchestration Endpoint

Rivery Flows orchestrate sequences of Rivers with dependency graphs — a flow might extract from ten sources in parallel, then trigger a dbt transformation run once all extractions complete, then refresh a Looker dashboard. When the Flow orchestration endpoint is degraded, complex pipeline workflows cannot be triggered or monitored, and the entire dependency chain downstream of the flow stalls:

# List configured Flows
curl -H "Authorization: Bearer $RIVERY_API_KEY" \
  https://api.rivery.io/api/v1/flows
# Returns: {"flows": [{"flow_id": "...", "name": "...", "status": "..."}]}

# Get a specific Flow's execution history
curl -H "Authorization: Bearer $RIVERY_API_KEY" \
  https://api.rivery.io/api/v1/flows/YOUR_FLOW_ID/runs
# Returns run history with step-level detail
  1. Add Monitor → HTTP.
  2. URL: https://api.rivery.io/api/v1/flows.
  3. Check interval: 3 minutes.
  4. Response timeout: 15 seconds.
  5. Expected status: 200.
  6. Keyword: flows.
  7. Label: Rivery Flow orchestration endpoint.
  8. Click Save.

This monitor detects orchestration-layer failures that would prevent multi-step pipeline workflows from executing, without requiring you to wait for a downstream dbt run to fail and surface the missing upstream data.

Step 5: Monitor Your River Completion Webhook Receiver

Rivery emits events — River completions, failures, and warnings — to external webhooks that feed your incident management, downstream orchestration, or notification systems. If the webhook receiver is down when a River fails to extract from a source, your downstream dbt runs proceed on tables that haven't been refreshed, your dashboards serve stale data, and your data quality checks validate yesterday's numbers rather than today's:

curl https://your-rivery-webhook.example.com/health
# Returns: {"status": "ok"} when the webhook receiver is operational
  1. Add Monitor → HTTP.
  2. URL: https://your-rivery-webhook.example.com/health.
  3. Check interval: 2 minutes.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. Keyword: ok.
  7. Label: Rivery completion webhook.
  8. Click Save.

For Slack-based Rivery failure notifications, verify https://slack.com/api/api.test to confirm Slack API reachability from your notification infrastructure. For PagerDuty-routed River failures, monitor https://api.pagerduty.com/ability to verify the Events API is accessible.

Step 6: Monitor SSL Certificates

The Rivery API is accessed by data engineers during incident response and by automated tools over TLS. An expired certificate on api.rivery.io or any custom domain fronting your webhook receiver blocks both human access and automated integrations at the worst possible moment — when a River has failed and your team needs to query execution logs to understand which warehouse tables are stale:

openssl s_client -connect api.rivery.io:443 2>/dev/null | openssl x509 -noout -dates
  1. Add Monitor → SSL Certificate.
  2. Domain: api.rivery.io.
  3. Alert when expiry is within: 30 days.
  4. Alert again: 14 days, 7 days, 3 days, 1 day.
  5. Click Save.

Repeat for any custom domains that front your webhook receivers or internal tooling that integrates with the Rivery API. Rivery-managed certificates on api.rivery.io are renewed by Rivery, but custom webhook domains and internal automation proxies require your team's certificate management.

Step 7: Configure Alerting

In Vigilmon under Settings → Notifications, configure your alert channels:

| Monitor | Trigger | Action | |---|---|---| | Rivery API | Non-200 or rivers missing | All River and Flow scheduling halted; escalate to data engineering | | River execution endpoint | Non-200 or rivers missing | Pipeline status invisible; downstream orchestration blind | | Flow orchestration endpoint | Non-200 or flows missing | Multi-step workflows cannot execute; dependency chain stalled | | Completion webhook | Non-200 or ok missing | River events not being delivered; dbt runs on stale data | | SSL certificate | < 30 days to expiry | Renew certificate; API access blocked on expiry |

Alert after: 1 consecutive failure for the Rivery API and River execution endpoint. 2 consecutive failures for the Flow orchestration endpoint and webhook receiver to reduce noise from brief transient timeouts during Rivery platform maintenance windows.

Common Rivery Infrastructure Failure Modes and What Vigilmon Catches

| Scenario | Vigilmon monitor | |---|---| | Rivery platform outage | API monitor fires; all Rivers and Flows halt | | River execution endpoint degradation | River monitor fires; pipeline status invisible to orchestration | | Flow orchestration endpoint failure | Flow monitor fires; multi-step pipelines cannot run | | Source credential expiry | Rivers fail at extraction; API healthy but run history shows errors | | Webhook receiver crash | Webhook monitor fires; completion events not delivered | | TLS certificate expired on webhook domain | SSL monitor alerts at 30-day threshold | | Control plane maintenance window | API monitor fires during window | | Authentication service degradation | API returns 401; API monitor fires | | Network path to Rivery API blocked | All API monitors fire with connection timeout | | Destination warehouse downtime | Rivers fail at load; connections endpoint may show healthy | | Concurrent River quota exhaustion | Individual Rivers queue; API and endpoint monitors healthy |

Monitoring River Data Quality (Beyond Infrastructure)

Vigilmon monitors the infrastructure that Rivery depends on — the API control plane, the River execution surface, the Flow orchestration layer, and your webhook receivers. The actual completeness of the data extracted, the accuracy of incremental watermarks, and the freshness of what lands in your warehouse require additional observability practices:

  • Incremental watermark drift: Rivery's incremental Rivers track a high-water mark for each source. A River that succeeds at the API level may have silently advanced its watermark past records that were delayed in the source system, causing permanent gaps in your warehouse tables. Infrastructure health cannot detect watermark drift.
  • Row count anomaly detection: Monitor the actual row counts landing in your warehouse tables across River runs. A River that completes successfully but extracts zero records due to a source API filter regression will report COMPLETED to the Rivery execution API — the infrastructure monitor fires green while your tables stop growing.
  • End-to-end data freshness: Track the updated_at or event timestamp of the most recently loaded record in your critical warehouse tables. A healthy Rivery API does not guarantee that a River completed within its SLA window or that the source system had records to deliver during the extraction window.

Vigilmon catches infrastructure failures — when the Rivery API is unavailable, the River execution surface is degraded, or the Flow orchestration layer is rejecting requests. The completeness of extracted data, the correctness of incremental watermarks, and the freshness of what lands in your warehouse for downstream dbt and BI consumption require domain-specific data quality monitoring layered on top of infrastructure health checks.

Rivery brings fully managed cloud-native ELT to data teams who need reliable ingestion from hundreds of sources into modern cloud data warehouses, but the API infrastructure it runs on — the River control plane, the Flow orchestration surface, and your webhook receivers — can fail in ways that silently stall warehouse ingestion without surfacing visible errors to downstream dashboard and ML consumers. Vigilmon gives your data engineering team external visibility into that infrastructure: API availability, River execution reachability, Flow orchestration health, and SSL certificate validity, so you know the moment Rivery needs attention and can restore ELT pipelines before downstream dbt runs and dashboards operate on hours-old data.

Start monitoring your Rivery ELT infrastructure in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles