tutorial

How to Monitor Services During Litmus Chaos Experiments with Vigilmon

Litmus Chaos is an open-source chaos engineering platform that helps SREs and DevOps teams validate the resilience of cloud-native applications on Kubernetes...

Litmus Chaos is an open-source chaos engineering platform that helps SREs and DevOps teams validate the resilience of cloud-native applications on Kubernetes. It provides a rich library of pre-built chaos experiments — pod delete, node CPU hog, network loss, disk I/O stress — that you can run against your workloads to find weaknesses before your users do.

But a chaos experiment only tells you something useful if you have an independent observer watching from outside your infrastructure. Vigilmon provides exactly that: continuous, external uptime monitoring that records the exact moment your service becomes unavailable and the moment it recovers, giving you an objective SLO measurement for every chaos run.


What Litmus Chaos and Vigilmon do differently

Litmus Chaos injects faults and measures the result using internal probes (HTTP, command, Prometheus). Vigilmon monitors your service from outside the cluster, from the perspective of a real end user.

| Capability | Litmus Chaos | Vigilmon | |-----------|-------------|---------| | Inject pod failures | Yes | No | | HTTP probe during experiment | Yes (internal) | Yes (external, continuous) | | 24/7 uptime monitoring | No | Yes | | Alert on service down | No | Yes | | Public status page | No | Yes | | SSL certificate monitoring | No | Yes | | Multi-region checks | No | Yes |

Together, they give you a complete picture: Litmus controls the fault injection, Vigilmon provides the external ground truth.


What you'll need

  • A Kubernetes cluster (EKS, GKE, AKS, or a local cluster like kind)
  • Litmus Chaos installed (ChaosCenter or the Kubernetes operator)
  • At least one HTTP service exposed with an Ingress or LoadBalancer
  • A free Vigilmon account — no credit card required

Step 1: Prepare your application health endpoint

Add a /health route to your application that reflects the health of its dependencies. This is what Vigilmon will probe every minute.

Python / FastAPI:

from fastapi import FastAPI
from contextlib import asynccontextmanager
import asyncpg
import redis.asyncio as aioredis

@app.get("/health")
async def health():
    checks = {}
    try:
        conn = await asyncpg.connect(DATABASE_URL)
        await conn.fetchval("SELECT 1")
        await conn.close()
        checks["database"] = "ok"
    except Exception as e:
        checks["database"] = f"error: {str(e)}"

    try:
        r = aioredis.from_url(REDIS_URL)
        await r.ping()
        checks["cache"] = "ok"
    except Exception:
        checks["cache"] = "error"

    healthy = all(v == "ok" for v in checks.values())
    status_code = 200 if healthy else 503
    return JSONResponse(
        content={"status": "ok" if healthy else "degraded", "checks": checks},
        status_code=status_code,
    )

Java / Spring Boot:

@RestController
@RequestMapping("/health")
public class HealthController {

    @GetMapping
    public ResponseEntity<HealthResponse> health() {
        boolean dbOk = checkDatabase();
        boolean cacheOk = checkRedis();
        boolean healthy = dbOk && cacheOk;

        return ResponseEntity
            .status(healthy ? HttpStatus.OK : HttpStatus.SERVICE_UNAVAILABLE)
            .body(new HealthResponse(
                healthy ? "ok" : "degraded",
                Map.of("database", dbOk ? "ok" : "error",
                       "cache", cacheOk ? "ok" : "error")
            ));
    }
}

Deploy with Kubernetes readiness and liveness probes:

containers:
  - name: api
    image: your-org/api:latest
    ports:
      - containerPort: 8080
    readinessProbe:
      httpGet:
        path: /health
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 5
      failureThreshold: 3
    livenessProbe:
      httpGet:
        path: /health
        port: 8080
      initialDelaySeconds: 30
      periodSeconds: 10
      failureThreshold: 3

Step 2: Install Litmus Chaos

Using Helm:

helm repo add litmuschaos https://litmuschaos.github.io/litmus-helm/
helm repo update

helm install chaos litmuschaos/litmus \
  --namespace litmus \
  --create-namespace \
  --set portal.frontend.service.type=NodePort

Install ChaosExperiments from the Hub:

kubectl apply -f https://hub.litmuschaos.io/api/chaos/master?file=charts/generic/pod-delete/experiment.yaml -n production
kubectl apply -f https://hub.litmuschaos.io/api/chaos/master?file=charts/generic/pod-cpu-hog/experiment.yaml -n production
kubectl apply -f https://hub.litmuschaos.io/api/chaos/master?file=charts/generic/network-loss/experiment.yaml -n production

Annotate target deployments:

kubectl annotate deployment api-server litmuschaos.io/chaos="true" -n production

Step 3: Set up Vigilmon monitors

Before running any experiments, configure your monitors and let them run for at least 30 minutes to establish a healthy baseline.

  1. Log in to vigilmon.online
  2. Monitors → New Monitor → HTTP / HTTPS
  3. URL: https://api.your-domain.com/health
  4. Check interval: 1 minute
  5. Expected status code: 200
  6. Response body contains: "status":"ok"
  7. Save

Add monitors for every service in your chaos scope:

| Service | Monitor URL | Expected Status | |---------|-------------|----------------| | API Server | https://api.your-domain.com/health | 200 | | Frontend App | https://app.your-domain.com | 200 | | Admin Panel | https://admin.your-domain.com/health | 200 |

Also add TCP monitors for any databases or queues that Litmus might target:

  1. Monitors → New Monitor → TCP Port
  2. Host: your-db.internal.example.com
  3. Port: 5432
  4. Save

Step 4: Configure Vigilmon alerts

Real-time alerting during chaos experiments is essential:

Slack:

  1. Alert Channels → Add Channel → Webhook
  2. Paste your Slack incoming webhook URL
  3. Assign to all monitors

PagerDuty:

  1. Alert Channels → Add Channel → Webhook
  2. Use your PagerDuty events API endpoint
  3. Assign to production-critical monitors only

When Vigilmon detects a failure, it sends:

{
  "monitor_name": "API Server Health",
  "status": "down",
  "url": "https://api.your-domain.com/health",
  "started_at": "2024-07-01T10:00:00Z",
  "duration_seconds": 0
}

Note the started_at timestamp — you'll use this to correlate with when Litmus injected the fault.


Step 5: Run a pod delete experiment

The pod delete experiment is the most common starting point for chaos engineering. It deletes one pod from a deployment and verifies the service remains available:

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: api-server-pod-delete
  namespace: production
spec:
  appinfo:
    appns: production
    applabel: 'app=api-server'
    appkind: deployment
  chaosServiceAccount: litmus-admin
  monitoring: false
  jobCleanUpPolicy: retain
  experiments:
    - name: pod-delete
      spec:
        components:
          env:
            - name: TOTAL_CHAOS_DURATION
              value: '120'     # 2 minutes
            - name: CHAOS_INTERVAL
              value: '30'      # delete a pod every 30 seconds
            - name: FORCE
              value: 'false'
        probe:
          - name: healthcheck-probe
            type: httpProbe
            mode: Continuous
            runProperties:
              probeTimeout: 5
              retry: 3
              interval: 10
            httpProbe/inputs:
              url: https://api.your-domain.com/health
              insecureSkipVerify: false
              method:
                get:
                  criteria: ==
                  responseCode: "200"

Apply it:

kubectl apply -f pod-delete-engine.yaml -n production

What to watch:

  • Litmus's internal HTTP probe runs every 10 seconds
  • Vigilmon's external probe runs every 60 seconds
  • If Vigilmon fires an incident during the experiment, you have a real resilience gap

Step 6: Network loss experiment

Simulate packet loss on your API service's network:

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: api-network-loss
  namespace: production
spec:
  appinfo:
    appns: production
    applabel: 'app=api-server'
    appkind: deployment
  chaosServiceAccount: litmus-admin
  experiments:
    - name: pod-network-loss
      spec:
        components:
          env:
            - name: TOTAL_CHAOS_DURATION
              value: '300'    # 5 minutes
            - name: NETWORK_INTERFACE
              value: 'eth0'
            - name: NETWORK_PACKET_LOSS_PERCENTAGE
              value: '100'   # 100% packet loss (partition)
            - name: CONTAINER_RUNTIME
              value: 'containerd'
            - name: SOCKET_PATH
              value: '/run/containerd/containerd.sock'

100% packet loss effectively partitions the pod from the network. Vigilmon will fire an incident if this causes the external health endpoint to become unreachable.


Step 7: CPU hog experiment

Stress CPU resources to verify your service stays responsive under compute pressure:

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: api-cpu-hog
  namespace: production
spec:
  appinfo:
    appns: production
    applabel: 'app=api-server'
    appkind: deployment
  chaosServiceAccount: litmus-admin
  experiments:
    - name: pod-cpu-hog
      spec:
        components:
          env:
            - name: TOTAL_CHAOS_DURATION
              value: '300'
            - name: CPU_CORES
              value: '2'
            - name: PODS_AFFECTED_PERC
              value: '50'    # affect 50% of pods

During this experiment, check Vigilmon's response time data. If your health endpoint starts timing out (Vigilmon marks it down), you've found the CPU threshold at which the service becomes functionally unavailable — even before Kubernetes marks it as unhealthy.


Step 8: Build a chaos experiment runbook with Vigilmon validation

Here's a repeatable workflow for every chaos experiment:

#!/bin/bash
# chaos-experiment.sh

EXPERIMENT_NAME=$1
DURATION=${2:-300}  # default 5 minutes

echo "[$(date -u)] Starting chaos experiment: $EXPERIMENT_NAME"
echo "[$(date -u)] Vigilmon baseline: check https://vigilmon.online for incident history"

# Apply chaos engine
kubectl apply -f "experiments/${EXPERIMENT_NAME}.yaml" -n production

# Wait for experiment duration
echo "[$(date -u)] Chaos running for ${DURATION}s. Watching Vigilmon..."
sleep $DURATION

# Check chaos result
VERDICT=$(kubectl get chaosresult "${EXPERIMENT_NAME}-result" -n production \
  -o jsonpath='{.status.experimentStatus.verdict}')
echo "[$(date -u)] Litmus verdict: $VERDICT"
echo "[$(date -u)] Experiment complete. Check Vigilmon for external incident history."

# Clean up
kubectl delete chaosengine "${EXPERIMENT_NAME}" -n production

After each experiment, check the Vigilmon incident log:

  • Zero incidents + Litmus Pass: service is resilient
  • Incident during experiment + Litmus Pass: internal probes passed but external users experienced downtime — your internal probes may be too lenient
  • Incident beyond experiment end: service didn't recover fully — investigate auto-restart or circuit breaker configuration

Step 9: Use Litmus probes alongside Vigilmon

Litmus supports HTTP probes that run during experiments. Use them together with Vigilmon for defense-in-depth verification:

  • Litmus HTTP probe: runs every 10 seconds, internal to the cluster, verifies the endpoint from within Kubernetes networking
  • Vigilmon probe: runs every 60 seconds, external to the cluster, verifies the endpoint from the public internet

When both agree (both see failure), you have high confidence the service is genuinely down. When only Vigilmon sees failure, the issue may be in your ingress controller, load balancer, or TLS termination — not in the pod itself.


Step 10: Status page during chaos experiments

Give your team a shared view of service health during experiments:

  1. Status Pages → New Status Page
  2. Name it "Chaos Test Status" or "Engineering Status"
  3. Add all monitored services
  4. Publish and share with the chaos testing team

Before each experiment, post a maintenance notice on the status page. During experiments, the page updates in real time. After each experiment, the page history gives you a timeline artifact.


Chaos experiment outcomes reference

| Litmus Result | Vigilmon Result | Interpretation | |--------------|----------------|----------------| | Pass | No incident | Service is resilient ✓ | | Pass | Incident (< 1 min) | Brief blip — investigate pod restart timing | | Pass | Incident (> 1 min) | External availability failed despite passing internal probes | | Fail | No incident | Internal degradation not externally visible — good graceful degradation | | Fail | Incident | Service fully unavailable — priority fix | | Fail | Incident longer than experiment | Recovery automation not working |


What's next

  • Litmus Chaos Workflows: chain experiments and gate each step on Vigilmon's monitor status using the Vigilmon API
  • SSL certificate monitoring: add expiry alerts for all domains in your chaos test scope
  • Heartbeat monitoring: ensure your chaos experiment scheduler (cron jobs or Litmus Schedules) continues to fire — Vigilmon alerts you when a scheduled chaos run stops reporting in

Get started free at vigilmon.online — monitors go live in under a minute, no credit card required.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →