How to Monitor Falco with Vigilmon

Falco is the cloud-native runtime security tool that watches your Kubernetes workloads and host systems for suspicious behavior — unexpected system calls, container escapes, privilege escalation attempts, file integrity violations, and network anomalies. It's your last line of defense: the sensor that tells you when something is actively going wrong in your production environment.

The irony of a security monitoring tool is that it needs to be monitored itself. When Falco goes down, you don't get an alert — you just lose visibility into runtime threats. An attacker exploiting a container escape vulnerability, a malicious insider running unexpected system calls, a compromised dependency writing to /etc/passwd — all of these go undetected while Falco is down. You won't know your security monitor was offline until you check the logs and find a gap.

In this tutorial you'll set up external uptime monitoring for Falco using Vigilmon, covering the HTTP output endpoint, gRPC health, alert webhook delivery, and Falcosidekick availability.

Why Falco needs external monitoring

Falco can fail in several ways that are completely invisible without external monitoring:

• Falco process crashes — a kernel driver incompatibility, memory exhaustion, or a malformed rule causes the process to exit. Runtime threats go undetected.
• gRPC server becomes unresponsive — Falco's gRPC API, used by clients and Falcosidekick, starts timing out. Alert delivery breaks silently.
• HTTP output endpoint stops responding — if you're using Falco's built-in HTTP output to post alerts to a SIEM or custom endpoint, alerts queue and eventually drop.
• Falcosidekick fails — Falcosidekick is the fan-out daemon that delivers Falco alerts to Slack, PagerDuty, Prometheus, Elasticsearch, and dozens of other destinations. If Sidekick goes down, your entire alert delivery chain breaks regardless of whether Falco is healthy.
• Driver loading failure after kernel update — a node kernel update can break the eBPF or kernel module driver that Falco uses for syscall capture, causing Falco to start without its sensor layer.

Each of these scenarios produces the same user-visible symptom: silence. No errors, no alerts, no indication that your runtime security posture just changed.

What you'll need

• A running Falco installation (Kubernetes DaemonSet or systemd service)
• The Falco HTTP output plugin or gRPC server enabled
• Falcosidekick deployed (strongly recommended for production)
• A free Vigilmon account

Step 1: Enable Falco's HTTP output endpoint

Falco can POST alerts to an HTTP endpoint. We'll use this both for your SIEM integration and as a health signal for external monitoring.

In your falco.yaml configuration:

# falco.yaml
http_output:
  enabled: true
  url: https://your-siem.example.com/falco/events
  # For monitoring purposes, you can also point to a local echo service

# Enable the web server for health checks
webserver:
  enabled: true
  listen_port: 8765
  k8s_healthz_endpoint: /healthz
  ssl_enabled: false

# gRPC server for Falcosidekick
grpc:
  enabled: true
  bind_address: "0.0.0.0:5060"
  threadiness: 8

grpc_output:
  enabled: true

Verify the health endpoint:

# Falco web server health check
curl -s http://falco-node:8765/healthz
# Expected: HTTP 200

# Test the gRPC server is listening
nc -zv falco-node 5060
# Expected: Connection to falco-node 5060 port [tcp/*] succeeded!

Step 2: Deploy Falcosidekick

Falcosidekick is the recommended alert delivery layer for production Falco deployments. It subscribes to Falco via gRPC and fans out alerts to your destinations.

# falcosidekick values for Helm deployment
config:
  slack:
    webhookurl: https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK
    minimumpriority: warning
  pagerduty:
    routingkey: YOUR_PAGERDUTY_ROUTING_KEY
    minimumpriority: critical
  webui:
    enabled: true

listenport: 2801

# Deploy via Helm
helm install falcosidekick falcosecurity/falcosidekick \
  -n falco --create-namespace \
  -f falcosidekick-values.yaml

# Verify Falcosidekick health
kubectl port-forward -n falco svc/falcosidekick 2801:2801
curl -s http://localhost:2801/healthz
# Expected: HTTP 200

# Check the Falcosidekick UI (if enabled)
curl -s http://localhost:2801/ping
# Expected: "pong"

Step 3: Monitor the Falco web server health endpoint

This is your primary liveness check for the Falco process. Expose the Falco web server via a Kubernetes Service:

# falco-health-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: falco-health
  namespace: falco
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: falco
  ports:
    - name: webserver
      port: 8765
      targetPort: 8765
      nodePort: 30765

kubectl apply -f falco-health-service.yaml

Set up the monitor in Vigilmon:

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS
3. Set the URL: http://<node-ip>:30765/healthz
4. Check interval: 1 minute
5. Expected response:
   • Status code: 200
6. Save the monitor

Because Falco runs as a DaemonSet with one pod per node, you should monitor at least your critical nodes individually. A single node going dark means workloads on that node are unmonitored.

Step 4: Monitor Falcosidekick availability

Falcosidekick is the alert delivery backbone. If it's down, Falco alerts are generated but never delivered to Slack, PagerDuty, or your SIEM.

Expose Falcosidekick externally:

# falcosidekick-lb-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: falcosidekick-external
  namespace: falco
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: falcosidekick
  ports:
    - name: http
      port: 2801
      targetPort: 2801

kubectl apply -f falcosidekick-lb-service.yaml
kubectl get svc -n falco falcosidekick-external
# Note the EXTERNAL-IP

Add to Vigilmon:

1. Monitors → New Monitor → HTTP / HTTPS
2. URL: http://<falcosidekick-external-ip>:2801/healthz
3. Check interval: 1 minute
4. Expected response:
   • Status code: 200
5. Save the monitor

Also add a ping monitor to verify Falcosidekick's own liveness response:

1. Monitors → New Monitor → HTTP / HTTPS
2. URL: http://<falcosidekick-external-ip>:2801/ping
3. Expected response:
   • Status code: 200
   • Response body contains: pong
4. Save the monitor

Step 5: Monitor the gRPC server port

Add a TCP monitor for the Falco gRPC port. If this port goes down, Falcosidekick loses its connection to Falco and stops receiving alerts — even if both processes are running:

# Expose gRPC port via NodePort
apiVersion: v1
kind: Service
metadata:
  name: falco-grpc
  namespace: falco
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: falco
  ports:
    - name: grpc
      port: 5060
      targetPort: 5060
      nodePort: 30560

In Vigilmon:

1. Monitors → New Monitor → TCP Port
2. Host: <node-ip>
3. Port: 30560
4. Check interval: 1 minute
5. Save the monitor

A TCP monitor firing on the gRPC port while the HTTP health monitor is up tells you that Falco's gRPC subsystem has failed without crashing the main process — an unusual but real failure mode, typically caused by TLS misconfiguration or thread exhaustion.

Step 6: Configure alert channels

Falco going down is a security incident, not just an operational one. Alert routing should reflect that urgency.

Email alerts

1. Alert Channels → Add Channel → Email
2. Add your security team, on-call rotation, and SOC if applicable
3. Assign to all Falco monitors

Webhook alerts for immediate response

1. Alert Channels → Add Channel → Webhook
2. Configure PagerDuty with a high-urgency policy for Falco health monitors
3. Consider a dedicated Slack channel #falco-health separate from #alerts

Example Vigilmon alert payload:

{
  "monitor_name": "Falco healthz (worker-1)",
  "status": "down",
  "url": "http://worker-1:30765/healthz",
  "started_at": "2026-01-15T14:23:00Z",
  "duration_seconds": 300
}

Alert routing priorities

| Monitor | Priority | Response | |---------|----------|----------| | Falco agent /healthz (any node) | P1 / Security | Page on-call + notify security team | | Falco gRPC port (TCP) | P1 / Security | Page on-call; check Falcosidekick connectivity | | Falcosidekick /healthz | P1 | Page on-call; alerts not being delivered | | Falcosidekick /ping | P2 | Alert; partial delivery failure possible |

For a security tool, treat any downtime as a potential security gap — the longer Falco is down, the longer your runtime threat detection has a blind spot.

Step 7: Verify Falco is actually detecting threats

Monitoring that Falco is alive is necessary but not sufficient. You also want to verify that Falco is actually processing syscalls and firing rules. Use Falco's test event mechanism:

# Send a test event that should trigger a Falco rule
# This runs a command that Falco's default ruleset will detect
kubectl run falco-test --image=alpine --restart=Never --rm -it -- \
  sh -c "cat /etc/shadow"

# Check that Falco detected it
kubectl logs -n falco -l app.kubernetes.io/name=falco --tail 20 | grep "shadow"
# Expected: {"output":"... A shell was spawned... /etc/shadow","priority":"Warning",...}

If the test event doesn't produce a Falco alert within 30 seconds, Falco's syscall capture layer may be broken even though the process is alive and the health endpoint returns 200.

Consider running this test on a schedule (via a Kubernetes CronJob) and alerting if the expected output doesn't appear in Falco's logs.

Complete monitoring setup

| Monitor | Type | Endpoint | Priority | |---------|------|----------|----------| | Falco agent health | HTTP | http://node:30765/healthz | P1 | | Falco gRPC port | TCP | node:30560 | P1 | | Falcosidekick health | HTTP | http://sidekick:2801/healthz | P1 | | Falcosidekick ping | HTTP | http://sidekick:2801/ping | P2 |

For a cluster with 10 nodes, monitor the Falco agent on each node individually. Each node going dark is an independent security gap.

Summary

Falco is your runtime security sensor — when it's down, your production environment is running blind. External monitoring with Vigilmon ensures you know about Falco outages immediately, rather than discovering hours later that you have no security event data for a critical time window.

Start monitoring Falco for free at vigilmon.online — setup takes under five minutes.