Uptime monitoring for Apache SkyWalking

Apache SkyWalking is a distributed tracing and Application Performance Monitoring (APM) platform for microservices, cloud-native, and container-based architectures. It collects traces, metrics, logs, and service topology from your applications and provides deep operational visibility across your distributed system. When SkyWalking goes down, you lose observability across your entire microservices fleet — not just a single service, but the entire system's health picture.

This tutorial covers production-grade uptime monitoring for Apache SkyWalking using Vigilmon. We will walk through:

• Monitoring the SkyWalking OAP server health endpoint
• Monitoring the SkyWalking UI (Rocketbot)
• Monitoring the gRPC receiver port used by agents
• SSL certificate monitoring
• Webhook alerts for DOWN/UP events

Prerequisites

• Apache SkyWalking OAP (Observability Analysis Platform) server running standalone or in Kubernetes
• SkyWalking Rocketbot UI running (optional but recommended)
• A free account at vigilmon.online

Part 1: Enable the SkyWalking health check endpoint

The SkyWalking OAP server exposes a REST health check endpoint by default on port 12800. This is the primary liveness signal for your SkyWalking deployment.

Verify the health endpoint

curl http://localhost:12800/healthCheck

Expected response when healthy:

Health status is ok

If the OAP server is starting up or unhealthy, the endpoint returns HTTP 503 or a connection refused error.

SkyWalking configuration for the health endpoint

The REST receiver is enabled by default. Confirm it is enabled in your application.yml:

# config/application.yml
receiver-sharing-server:
  default:
    restHost: ${SW_RECEIVER_SHARING_REST_HOST:0.0.0.0}
    restPort: ${SW_RECEIVER_SHARING_REST_PORT:12800}
    restContextPath: ${SW_RECEIVER_SHARING_REST_CONTEXT_PATH:/}
    restMaxThreads: ${SW_RECEIVER_SHARING_REST_MAX_THREADS:200}
    restIdleTimeOut: ${SW_RECEIVER_SHARING_REST_IDLE_TIMEOUT:30000}
    restAcceptQueueSize: ${SW_RECEIVER_SHARING_REST_QUEUE_SIZE:0}

Docker Compose deployment

# docker-compose.yml
version: "3.8"

services:
  oap:
    image: apache/skywalking-oap-server:9.7.0
    environment:
      - SW_STORAGE=elasticsearch
      - SW_STORAGE_ES_CLUSTER_NODES=elasticsearch:9200
    ports:
      - "11800:11800" # gRPC receiver for agents
      - "12800:12800" # REST receiver and health check
    depends_on:
      - elasticsearch
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:12800/healthCheck"]
      interval: 30s
      timeout: 10s
      retries: 3

  ui:
    image: apache/skywalking-ui:9.7.0
    environment:
      - SW_OAP_ADDRESS=http://oap:12800
    ports:
      - "8080:8080"
    depends_on:
      - oap

  elasticsearch:
    image: elasticsearch:8.13.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
    ports:
      - "9200:9200"

Part 2: Set up HTTP monitoring in Vigilmon

Monitor the OAP server health endpoint

1. Log in to vigilmon.online and click Add Monitor.
2. Choose HTTP(S) monitor.
3. Enter: http://your-skywalking-host:12800/healthCheck
4. Set interval to 1 minute.
5. Add a keyword check: must contain Health status is ok.
6. Add your alert channel.
7. Click Save.

The keyword check is important: some load balancers return HTTP 200 from their own health page when the backend is unreachable. Checking for Health status is ok confirms the actual OAP server responded.

Monitor the SkyWalking UI (Rocketbot)

The SkyWalking UI is what your team uses to view traces, metrics, and service topologies. Monitor it separately so you know when the dashboard becomes unavailable even if the OAP server is healthy:

1. Add an HTTP(S) monitor for http://your-skywalking-host:8080/
2. Set interval to 2 minutes.
3. Add a keyword check: must contain SkyWalking or the page title text.
4. Add your alert channel.

Part 3: Monitor the gRPC receiver port

SkyWalking agents (Java, Go, Python, Node.js) send trace data to the OAP server's gRPC endpoint on port 11800. If this port goes down, all instrumented services stop reporting traces — silently. Add a TCP monitor:

1. In Vigilmon, click Add Monitor → TCP Port monitor.
2. Enter host: your-skywalking-host.
3. Enter port: 11800.
4. Set interval to 1 minute.
5. Add your alert channel.
6. Click Save.

This confirms the gRPC receiver port is accepting connections independently of the REST health check.

Kubernetes deployment

In Kubernetes, expose both the health check and gRPC ports via Services:

apiVersion: v1
kind: Service
metadata:
  name: skywalking-oap
  namespace: monitoring
spec:
  selector:
    app: skywalking-oap
  ports:
    - name: grpc
      port: 11800
      targetPort: 11800
    - name: rest
      port: 12800
      targetPort: 12800
  type: LoadBalancer

Once the LoadBalancer assigns external IPs:

kubectl get svc skywalking-oap -n monitoring
# NAME             TYPE           EXTERNAL-IP     PORT(S)
# skywalking-oap   LoadBalancer   203.0.113.10    11800:31800/TCP,12800:31200/TCP

Point Vigilmon at http://203.0.113.10:12800/healthCheck for HTTP monitoring and 203.0.113.10:11800 for TCP monitoring.

Part 4: SSL certificate monitoring

If your SkyWalking deployment uses TLS for the REST API or gRPC receiver (recommended when agents connect over the public internet):

# application.yml — TLS for gRPC receiver
receiver-grpc:
  default:
    gRPCHost: ${SW_RECEIVER_GRPC_HOST:0.0.0.0}
    gRPCPort: ${SW_RECEIVER_GRPC_PORT:11800}
    gRPCSslEnabled: ${SW_RECEIVER_GRPC_SSL_ENABLED:true}
    gRPCSslCertChainPath: ${SW_RECEIVER_GRPC_SSL_CERT_CHAIN_PATH:/etc/certs/server.crt}
    gRPCSslPrivateKeyPath: ${SW_RECEIVER_GRPC_SSL_PRIVATE_KEY_PATH:/etc/certs/server.key}

Add an SSL monitor in Vigilmon:

1. Click Add Monitor → SSL monitor.
2. Enter the hostname: skywalking.example.com.
3. Set alert threshold to 14 days before expiry.
4. Add your alert channel.

A certificate expiry on the gRPC receiver is a silent failure — agents start dropping spans with TLS handshake errors while the REST health check (if on a different port) may still return healthy.

Part 5: Webhook alerts

Configure Vigilmon to call a webhook when SkyWalking goes down:

// Example: Express.js webhook receiver
import express from 'express';

const app = express();
app.use(express.json());

app.post('/webhook/vigilmon', (req, res) => {
  const { monitor_name, status, url, response_code, checked_at } = req.body;

  if (status === 'down') {
    console.error('[VIGILMON] SkyWalking DOWN', {
      monitor: monitor_name,
      url,
      code: response_code,
      at: checked_at,
    });

    notifyOnCall({
      severity: 'critical',
      message: `SkyWalking OAP is DOWN. Distributed trace collection is interrupted.`,
    });
  } else if (status === 'up') {
    console.info('[VIGILMON] SkyWalking recovered', { monitor: monitor_name });
  }

  res.sendStatus(204);
});

Vigilmon sends this payload on DOWN and UP transitions:

{
  "monitor_id": "mon_abc123",
  "monitor_name": "SkyWalking OAP health",
  "status": "down",
  "url": "http://skywalking.example.com:12800/healthCheck",
  "checked_at": "2026-07-01T12:00:00Z",
  "response_code": 503,
  "response_time_ms": 3200
}

Part 6: Monitoring SkyWalking with Elasticsearch backend

SkyWalking stores trace and metric data in Elasticsearch (or BanyanDB). An Elasticsearch failure causes SkyWalking to become unable to read or write data, even while the OAP process continues running. Monitor your Elasticsearch cluster separately:

1. Add an HTTP(S) monitor for http://your-elasticsearch-host:9200/_cluster/health
2. Set interval to 2 minutes.
3. Add a keyword check: must contain "status":"green" or "status":"yellow".
4. Add your alert channel.

A "status":"red" response means primary shards are unassigned — SkyWalking writes will start failing. Catching this early gives you time to recover Elasticsearch before SkyWalking begins dropping trace data.

Part 7: Alerting recommendations

SkyWalking is your distributed tracing backbone. When it fails, you lose the ability to understand system behavior during incidents — exactly when you need tracing most.

| Monitor | Interval | Alert threshold | Severity | |---------|----------|-----------------|----------| | OAP health check | 1 min | 2 consecutive failures | Critical | | gRPC receiver TCP | 1 min | 2 consecutive failures | Critical | | SkyWalking UI | 2 min | 3 consecutive failures | Warning | | Elasticsearch health | 2 min | 2 consecutive failures | Critical | | SSL certificate | 1 day | 14 days before expiry | Warning |

Set the OAP and gRPC monitors to alert after 2 consecutive failures to avoid false positives from rolling restarts, while keeping the interval at 1 minute to minimize data loss from undetected outages.

Summary

Your Apache SkyWalking deployment now has four layers of monitoring:

1. OAP health endpoint — confirms the SkyWalking server and its storage connection are healthy, polled every 60 seconds by Vigilmon.
2. gRPC receiver TCP monitor — confirms agents can connect to send trace data.
3. UI availability — confirms the Rocketbot dashboard is serving your team.
4. SSL monitors — alerts you 14 days before any certificate expires, preventing silent agent connection failures.

Vigilmon handles check scheduling, multi-region polling, alert routing, and uptime history. You get notified within 60 seconds of any SkyWalking failure, before gaps appear in your distributed trace data.

Monitor your Apache SkyWalking deployment free at vigilmon.online

#skywalking #apm #observability #monitoring #microservices