Monitoring Shuffle SOAR with Vigilmon

Shuffle is the open source SOAR platform that automates your security response workflows — enriching alerts, creating tickets, isolating endpoints, and notifying teams without human intervention. But Shuffle is a complex multi-service platform: a Go API backend, a React frontend, an Orborus execution engine, Docker containers for each workflow step, and OpenSearch for execution logs. When any layer fails, workflows stop executing and your automation goes silent — often without any visible error. Vigilmon monitors every tier of the Shuffle stack so you know when your automation is broken before the security event that needs it.

What You'll Set Up

• Backend API health monitor (port 5001 — workflow triggers and management)
• Frontend web application monitor (port 3001 — workflow builder UI)
• Orborus execution engine heartbeat
• OpenSearch cluster health monitor (port 9200)
• Workflow execution heartbeat via a test workflow
• Disk usage alert for Docker images and execution logs (>80%)
• TLS certificate monitoring for HTTPS deployments

Prerequisites

• Shuffle deployed via Docker Compose (the standard installation method)
• Shuffle backend API (port 5001) and frontend (port 3001) accessible from your monitoring vantage point
• A free Vigilmon account

Step 1: Monitor the Backend API (Port 5001)

The Shuffle backend API is the core of the platform — it receives webhook triggers, manages workflow configurations, and orchestrates executions. If it goes down, all workflow triggers stop and the frontend becomes inoperable.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. Enter: http://your-shuffle-host:5001/api/v1/health
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Add keyword check: "success":true (the health endpoint returns a JSON body).
7. Click Save.

Note: If your Shuffle instance runs behind a reverse proxy on port 443, use the proxied URL instead. The /api/v1/health path is the same.

Step 2: Monitor the Frontend Web Application (Port 3001)

The Shuffle React frontend is where analysts build and debug workflows. An unreachable frontend prevents workflow creation, editing, and manual execution triggers.

1. Click Add Monitor and set Type to HTTP / HTTPS.
2. Enter: http://your-shuffle-host:3001
3. Set Check interval to 2 minutes.
4. Set Expected HTTP status to 200.
5. Add keyword check: Shuffle (the page title).
6. Click Save.

If Shuffle is behind a reverse proxy with a single entry point, this monitor may overlap with your backend API URL — in that case, focus on the backend API monitor and skip the separate frontend check.

Step 3: Monitor Orborus (Execution Engine Heartbeat)

Orborus is the Shuffle execution engine that spawns Docker containers for each workflow step. If Orborus stops, workflows trigger but never execute — you'll see executions stuck in "EXECUTING" state indefinitely. This is the most insidious failure mode in Shuffle.

Create a Cron Heartbeat monitor in Vigilmon:

1. Click Add Monitor and set Type to Cron Heartbeat.
2. Set Expected interval to 2 minutes.
3. Copy the heartbeat URL.

Add a health check to your Orborus container via Docker Compose override:

# docker-compose.override.yml
services:
  orborus:
    healthcheck:
      test: ["CMD", "pgrep", "-x", "orborus"]
      interval: 60s
      timeout: 10s
      retries: 3

Then add a cron job on the Shuffle host that fires the Vigilmon heartbeat only when Orborus is healthy:

# /etc/cron.d/shuffle-orborus-heartbeat
* * * * * root docker inspect --format='{{.State.Health.Status}}' shuffle-orborus 2>/dev/null | grep -q healthy && curl -fsS YOUR_HEARTBEAT_URL > /dev/null
* * * * * root sleep 60; docker inspect --format='{{.State.Health.Status}}' shuffle-orborus 2>/dev/null | grep -q healthy && curl -fsS YOUR_HEARTBEAT_URL > /dev/null

If Orborus crashes or becomes unhealthy, the heartbeat stops and Vigilmon alerts within 2 minutes.

Step 4: Monitor OpenSearch / Elasticsearch (Port 9200)

Shuffle stores all workflow execution logs, results, and history in OpenSearch. A red cluster means execution history is unavailable and new execution data may be lost.

1. Click Add Monitor and set Type to HTTP / HTTPS.
2. Enter: http://your-shuffle-host:9200/_cluster/health
3. Set Check interval to 2 minutes.
4. Set Expected HTTP status to 200.
5. Add keyword check: "status":"green" — or if a yellow status is acceptable in your deployment, use "number_of_nodes".
6. Click Save.

Warning: A "status":"red" cluster means one or more primary shards are unavailable. Shuffle will fail to write execution results and workflow history will appear empty. Alert on red immediately.

For OpenSearch running on a non-standard port or with TLS, adjust the URL accordingly:

https://your-shuffle-host:9200/_cluster/health?pretty

Step 5: Create a Workflow Execution Heartbeat

The most reliable way to confirm Shuffle is actually executing workflows (not just running as a process) is to build a minimal test workflow that pings Vigilmon on a schedule.

Create the Test Workflow in Shuffle

1. In Shuffle, click New Workflow and name it Vigilmon Health Heartbeat.
2. Add a Scheduler trigger — set it to fire every 1 minute (or your preferred interval).
3. Add an HTTP app node with:
   • URL: YOUR_VIGILMON_HEARTBEAT_URL
   • Method: GET
4. Connect the trigger to the HTTP node.
5. Save and enable the workflow.

Create a matching Cron Heartbeat in Vigilmon with the same expected interval. If the Shuffle workflow stops executing (Orborus failure, scheduler bug, OpenSearch write failure), the heartbeat stops and you get alerted.

This is the gold standard health check: it validates the entire execution path from trigger → Orborus → container → external HTTP → back.

Step 6: Monitor Docker Container Health

Shuffle runs each workflow app step in an isolated Docker container. Container pull failures (Docker Hub rate limits, network issues) or storage exhaustion cause individual workflow steps to fail silently.

Check Docker's disk usage on the Shuffle host:

# Run periodically to check Docker disk usage
docker system df --format '{{.Type}}: {{.Size}} ({{.Reclaimable}} reclaimable)'

Install the Vigilmon agent on your Shuffle host to monitor:

• Disk usage on the Docker data directory (/var/lib/docker) — alert at 80%
• Docker daemon health — systemctl is-active docker

Add a heartbeat that fires only when Docker is operational:

* * * * * root systemctl is-active --quiet docker && curl -fsS YOUR_DOCKER_HEARTBEAT_URL > /dev/null

Step 7: Monitor Disk Usage

Shuffle accumulates data in two places that can fill unexpectedly: workflow execution logs in OpenSearch and Docker image layers on disk.

Install the Vigilmon agent on your Shuffle host and configure disk monitors:

| Mount Point | Alert Threshold | Why | |---|---|---| | / or Shuffle data dir | 80% | Execution logs and workflow data | | /var/lib/docker | 80% | Docker images for app containers |

Set both to alert at 80% utilization. At full disk, Docker cannot pull new images and Shuffle workflow steps fail with no space left on device.

Step 8: Monitor TLS Certificate Expiry

If Shuffle is exposed behind an HTTPS reverse proxy (nginx, Caddy, Traefik), add a certificate expiry monitor:

1. Click Add Monitor and set Type to SSL Certificate.
2. Enter your Shuffle domain and port 443.
3. Set Alert when certificate expires in less than 30 days.
4. Click Save.

An expired certificate breaks access to the Shuffle UI and causes webhook triggers from external services to fail with TLS verification errors.

Step 9: Configure Alerting

Alert Channels

In Vigilmon, go to Alert Channels and connect:

• Email — immediate notification to the security operations team
• Slack / Teams — SOC channel visibility
• PagerDuty — on-call escalation for production SOAR platforms
• Webhook — ironically, you can alert into another SOAR or ticketing system (though not Shuffle itself if it's the one that's down)

Recommended Alert Rules

| Monitor | Condition | Severity | |---|---|---| | Backend API (5001) | Down > 1 min | Critical | | Frontend (3001) | Down > 2 min | High | | Orborus Heartbeat | Missing > 2 min | Critical | | OpenSearch Cluster | Status red | Critical | | Workflow Execution Heartbeat | Missing > 2 min | Critical | | Docker Disk Usage | > 80% | High | | TLS Certificate | < 30 days | Medium |

Status Page

Create a Vigilmon status page for your security operations team so they can check SOAR health at a glance before reporting that automation isn't working:

1. Go to Status Pages → Create Status Page.
2. Add all Shuffle monitors.
3. Share the URL with the SOC team and CISO.

Operational Considerations

Workflow failure rate — Shuffle tracks execution status per workflow. A workflow with a persistently high failure rate often indicates a broken app authentication credential or a downstream API that changed its schema. Export Shuffle execution metrics to your APM stack and alert on per-workflow failure rates exceeding 5%.

Trigger reliability — Webhook trigger failures (Shuffle not receiving inbound webhooks) are often caused by reverse proxy misconfigurations after updates. Monitor your reverse proxy access logs for 502/504 responses on the Shuffle webhook receiver path.

App authentication health — Shuffle stores API credentials for connected apps (SIEM, EDR, threat intel). Credential rotation or expiry causes silent workflow failures. Review the Shuffle app authentication dashboard weekly and set calendar reminders for credential renewal.

OpenSearch index size — Shuffle's execution logs accumulate indefinitely. Set up OpenSearch index lifecycle management (ILM) policies to roll over and delete old execution data, preventing the cluster from filling up.

Conclusion

Shuffle automates your security response — which means when Shuffle fails, your incident response is slower, less consistent, and more reliant on manual intervention. The hardest failure modes to detect are the silent ones: Orborus not spawning containers, workflows stuck in EXECUTING, OpenSearch going red.

With Vigilmon, you get:

• API and frontend uptime monitoring catching infrastructure failures immediately
• Orborus health heartbeat detecting the most common silent failure mode
• Workflow execution heartbeat validating the entire automation path end-to-end
• Disk and storage alerts preventing container pull failures and log storage exhaustion

Get started free at vigilmon.online.