How to Monitor Apache Wayang with Vigilmon (Uptime, APIs, and Alerts)

Apache Wayang is a unified data processing abstraction layer that lets you run the same data pipeline across multiple execution engines — Spark, Flink, Java Streams, and others — without rewriting your code for each platform. Wayang sits between your application and the underlying execution engine, translating logical plans into engine-specific operations at runtime.

When your Wayang-powered data platform goes down or a REST API endpoint stops responding, pipelines stall, downstream jobs queue up, and data freshness SLAs are silently missed. This tutorial shows you how to set up uptime monitoring for Apache Wayang deployments using Vigilmon — free tier, no credit card.

Why Wayang deployments need external monitoring

Wayang introduces coordination points that vanilla engine monitoring won't catch:

• REST API outages — Wayang exposes a REST API for submitting and querying plan execution; if it goes down, no pipelines run and no errors surface until someone checks manually
• Engine connector failures — if the Spark or Flink backend becomes unreachable, Wayang silently falls back or throws at submission time rather than alerting ops
• Cross-engine job stalls — a pipeline spanning multiple engines can stall mid-execution; external latency checks are the fastest way to detect it
• Health endpoint drift — Wayang's health endpoint can return 200 even when the optimizer is deadlocked; body-content checks catch this where status-code checks miss it

External HTTP and TCP probes from Vigilmon catch all of these from outside the cluster.

What you'll need

• A running Apache Wayang deployment with its REST API accessible (we'll use port 8080 in the examples)
• A free Vigilmon account (sign up takes 30 seconds)

Step 1: Expose a health endpoint on your Wayang service

Wayang's REST API exposes built-in status endpoints. Make sure your deployment exposes at minimum:

GET /api/v1/health         → 200 OK when the optimizer is ready
GET /api/v1/plans          → list of submitted plans (tests DB connectivity)

If you've deployed Wayang behind a custom web layer, add a lightweight health route:

// Spring Boot controller wrapping Wayang
@RestController
public class HealthController {

    @Autowired
    private WayangContext wayangContext;

    @GetMapping("/health")
    public ResponseEntity<Map<String, String>> health() {
        Map<String, String> body = new HashMap<>();
        body.put("status", "ok");
        body.put("timestamp", Instant.now().toString());
        return ResponseEntity.ok(body);
    }
}

If you're running Wayang in Docker, expose the port:

services:
  wayang:
    image: apache/wayang:latest
    ports:
      - "8080:8080"
    environment:
      WAYANG_OPTIMIZER_TIMEOUT_MS: 30000
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/api/v1/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 30s
    restart: unless-stopped

Confirm the endpoint is reachable from outside the container before adding it to Vigilmon.

Step 2: Set up HTTP monitoring in Vigilmon

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS as the type
3. Set the URL to your health endpoint, e.g. https://wayang.example.com/api/v1/health
4. Set the check interval to 1 minute
5. Under Expected response, configure:
   • Status code: 200
   • Response body contains: "status":"ok"
6. Save the monitor

Vigilmon will probe your Wayang API from multiple regions every minute. If it returns anything other than 200 — or times out — an incident is triggered and your alert channels fire.

Step 3: Monitor the plan submission endpoint

The health endpoint tells you Wayang is alive. A deeper check against the plan-listing endpoint tells you the optimizer and its backing store are both functional:

1. Go to Monitors → New Monitor
2. Choose HTTP / HTTPS
3. URL: https://wayang.example.com/api/v1/plans
4. Expected status code: 200
5. Set a response time threshold of 5 seconds — slow plan queries indicate optimizer overload before it crashes

Save this as a second monitor. You now have a shallow liveness check and a deeper readiness check running in parallel.

Step 4: Add TCP port monitoring for the execution engine

Wayang communicates with Spark or Flink over their native ports. If those ports become unreachable, Wayang will accept plan submissions but fail silently during execution.

For a Spark master:

1. Go to Monitors → New Monitor
2. Choose TCP Port
3. Host: spark-master.example.com, Port: 7077
4. Save the monitor

For a Flink job manager:

1. Repeat for host flink-jobmanager.example.com, port 6123

These TCP checks ensure the full pipeline path — Wayang API → execution engine — is verified end to end.

Step 5: Configure webhook alerts

Data pipeline outages compound over time — every minute of downtime means more stale data. Get alerted in seconds, not hours.

1. Go to Alert Channels → Add Channel → Webhook
2. Enter your Slack, Discord, or PagerDuty webhook URL
3. Assign the channel to all your Wayang monitors

Example payload Vigilmon sends on an incident:

{
  "monitor_name": "Wayang REST API /health",
  "status": "down",
  "url": "https://wayang.example.com/api/v1/health",
  "started_at": "2026-03-10T14:05:00Z",
  "duration_seconds": 90
}

You can also configure an email alert channel for on-call escalations that need a paper trail.

Step 6: Create a status page for your data platform

If downstream teams depend on your Wayang pipelines, give them a self-service status page so they stop pinging you when pipelines are late:

1. Go to Status Pages → New Status Page
2. Name it (e.g. "Data Platform Status")
3. Add your monitors — Wayang API health, plan endpoint, Spark TCP, Flink TCP
4. Publish the page

You get a public URL at https://status.vigilmon.online/your-page showing real-time uptime for every component.

Monitor coverage summary

| Monitor | Type | What it catches | |---------|------|-----------------| | https://wayang.example.com/api/v1/health | HTTP | Wayang API crash, optimizer deadlock | | https://wayang.example.com/api/v1/plans | HTTP | DB connectivity, optimizer overload | | spark-master.example.com:7077 | TCP | Spark master unreachable | | flink-jobmanager.example.com:6123 | TCP | Flink job manager unreachable |

What's next

• Heartbeat monitors — if you run scheduled Wayang batch jobs, set up a heartbeat monitor so Vigilmon alerts you when a job stops checking in
• SSL expiry monitoring — Vigilmon checks your TLS certificate and alerts you before it lapses, keeping your Wayang API endpoints secure

Get started free at vigilmon.online — no credit card, monitors start running in under a minute.