tutorial

How to Monitor ProxySQL with Vigilmon (HTTP, TCP, and Alerts)

ProxySQL is a high-performance MySQL-compatible proxy that sits between your application servers and your database tier. It provides query routing, connectio...

ProxySQL is a high-performance MySQL-compatible proxy that sits between your application servers and your database tier. It provides query routing, connection pooling, load balancing across replicas, query caching, and automatic failover — all without any changes to your application code. Teams running MySQL, MariaDB, or Percona XtraDB Cluster at scale use ProxySQL to absorb thousands of application connections into a much smaller pool of backend connections and to route read queries to replicas automatically.

When ProxySQL goes down or becomes degraded, the effects are immediate and severe: every application that reads from or writes to your database loses connectivity, connection pools are exhausted, and query timeouts cascade across your entire stack. In this tutorial you'll set up end-to-end monitoring for ProxySQL using Vigilmon — free tier, no credit card required.

Why ProxySQL needs external monitoring

ProxySQL is a single point through which all database traffic flows. Its failure modes are subtle but catastrophic:

  • ProxySQL process crash — the proxy exits; all application database connections drop simultaneously; the error is "connection refused on 6033" rather than "database down," which confuses developers unfamiliar with the proxy layer
  • Backend host marked OFFLINE — ProxySQL's health check marks your primary MySQL host as OFFLINE_HARD after a spike in latency; all writes fail even though MySQL is healthy and would accept direct connections
  • Connection pool exhaustion — max_connections is set too low; new application connections are queued indefinitely rather than rejected, causing cascading timeouts
  • Hostgroup misconfiguration — a ProxySQL rule update accidentally routes writes to a read replica; the replica silently rejects the writes (read-only mode), returning error 1290
  • Stats interface unreachable — the ProxySQL admin interface on port 6032 becomes unresponsive; you lose visibility into connection stats and can't update routing rules without restarting
  • Memory pressure — ProxySQL's query cache fills up and starts evicting aggressively; cache hit rate drops to zero while memory usage stays maxed out

An external monitor catches ProxySQL outages from the application's perspective — the same perspective your users experience.

What you'll need

  • A running ProxySQL instance (version 2.x recommended)
  • MySQL or MariaDB backend servers registered in ProxySQL
  • A free Vigilmon account — takes 30 seconds to create

Step 1: Expose a health check endpoint for ProxySQL

ProxySQL does not ship with an HTTP health endpoint by default, but you can use its built-in stats interface to build one. The cleanest approach is to deploy a thin health check sidecar that queries ProxySQL's stats and exposes an HTTP endpoint for Vigilmon to probe.

Option A: MySQL stats query sidecar (recommended)

ProxySQL exposes rich statistics via a MySQL interface on port 6032 (the admin port) and its standard proxy port 6033. Query the mysql_server_ping_log table to confirm backend servers are reachable:

#!/usr/bin/env python3
# proxysql_health.py — lightweight health sidecar
from http.server import HTTPServer, BaseHTTPRequestHandler
import mysql.connector
import json
import os

PROXYSQL_HOST = os.getenv('PROXYSQL_HOST', '127.0.0.1')
PROXYSQL_ADMIN_PORT = int(os.getenv('PROXYSQL_ADMIN_PORT', '6032'))
PROXYSQL_ADMIN_USER = os.getenv('PROXYSQL_ADMIN_USER', 'admin')
PROXYSQL_ADMIN_PASS = os.getenv('PROXYSQL_ADMIN_PASS', 'admin')

class HealthHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != '/health':
            self.send_response(404)
            self.end_headers()
            return

        try:
            conn = mysql.connector.connect(
                host=PROXYSQL_HOST,
                port=PROXYSQL_ADMIN_PORT,
                user=PROXYSQL_ADMIN_USER,
                password=PROXYSQL_ADMIN_PASS,
                database='stats',
                connect_timeout=3,
            )
            cursor = conn.cursor(dictionary=True)

            # Check online backend server count
            cursor.execute("""
                SELECT COUNT(*) AS online_backends
                FROM mysql_servers
                WHERE status = 'ONLINE'
            """)
            row = cursor.fetchone()
            online = row['online_backends']

            # Get connection pool stats
            cursor.execute("""
                SELECT SUM(ConnUsed) AS used, SUM(ConnFree) AS free
                FROM stats_mysql_connection_pool
            """)
            pool = cursor.fetchone()

            conn.close()

            if online == 0:
                self.send_response(503)
                self.send_header('Content-Type', 'application/json')
                self.end_headers()
                self.wfile.write(json.dumps({
                    'status': 'error',
                    'reason': 'no_online_backends',
                    'online_backends': 0,
                }).encode())
                return

            self.send_response(200)
            self.send_header('Content-Type', 'application/json')
            self.end_headers()
            self.wfile.write(json.dumps({
                'status': 'ok',
                'online_backends': online,
                'connections_used': pool['used'],
                'connections_free': pool['free'],
            }).encode())

        except Exception as e:
            self.send_response(503)
            self.send_header('Content-Type', 'application/json')
            self.end_headers()
            self.wfile.write(json.dumps({
                'status': 'error',
                'reason': str(e),
            }).encode())

    def log_message(self, format, *args):
        pass  # suppress access logs

if __name__ == '__main__':
    port = int(os.getenv('HEALTH_PORT', '9090'))
    server = HTTPServer(('0.0.0.0', port), HealthHandler)
    server.serve_forever()

Run this sidecar alongside ProxySQL:

pip install mysql-connector-python
PROXYSQL_ADMIN_USER=radmin PROXYSQL_ADMIN_PASS=radmin python3 proxysql_health.py

Option B: Use ProxySQL's built-in ping endpoint

ProxySQL 2.4+ supports a simple TCP ping on a configurable port. Enable it in proxysql.cnf:

admin_variables=
{
    restapi_enabled=true
    restapi_port=6070
}

Once enabled, ProxySQL exposes:

GET http://localhost:6070/metrics   # Prometheus metrics
GET http://localhost:6070/health    # Simple health check

If you're on ProxySQL 2.4+, use the REST API endpoint directly with Vigilmon.

Step 2: Set up TCP port monitoring

The most critical check is whether ProxySQL's proxy port is accepting connections. If port 6033 is unreachable, all application database queries are failing:

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose TCP Port as the type
  3. Enter your ProxySQL host and port: proxysql.yourcompany.com / 6033
  4. Set the check interval to 1 minute
  5. Save the monitor

Also add a TCP monitor for the admin port to confirm management access is available:

  1. Create a second TCP monitor for port 6032
  2. This tells you when the admin interface is unreachable even if the proxy port is still serving cached connections

Step 3: Set up HTTP monitoring for the health sidecar

With the health sidecar running on port 9090:

  1. In Vigilmon, go to Monitors → New Monitor
  2. Choose HTTP / HTTPS as the type
  3. Set the URL to https://proxysql.yourcompany.com:9090/health (or proxy it through nginx)
  4. Set the check interval to 1 minute
  5. Under Expected response:
    • Status code: 200
    • Response body contains: "status":"ok"
  6. Save the monitor

Example nginx proxy to expose the health endpoint over HTTPS:

server {
    listen 443 ssl;
    server_name proxysql-health.yourcompany.com;

    ssl_certificate     /etc/letsencrypt/live/proxysql-health.yourcompany.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/proxysql-health.yourcompany.com/privkey.pem;

    location /health {
        proxy_pass http://127.0.0.1:9090/health;
        proxy_set_header Host $host;
        # Restrict to monitoring IPs only
        allow 185.25.104.0/24;  # Vigilmon monitoring IPs
        deny all;
    }
}

Step 4: Monitor backend MySQL servers directly

ProxySQL's health checks monitor your backend MySQL hosts internally, but you should also monitor them directly from Vigilmon to distinguish between "ProxySQL is down" and "MySQL is down":

  1. Add a TCP monitor for each MySQL backend: mysql-primary.yourcompany.com / 3306
  2. Add a TCP monitor for each replica: mysql-replica-1.yourcompany.com / 3306

When an alert fires, the pattern of which monitors are red tells you exactly what failed:

| ProxySQL TCP | MySQL TCP | Health HTTP | Diagnosis | |---|---|---|---| | Down | Up | Down | ProxySQL process crashed | | Up | Down | Down | MySQL backend went offline | | Up | Up | Down | Backend marked OFFLINE in ProxySQL hostgroup | | Up | Up | Up | Intermittent or resolved |

Step 5: Configure alerts and status page

Slack webhook alert:

  1. Go to Alert Channels → Add Channel → Webhook
  2. Enter your Slack webhook URL
  3. Assign it to all ProxySQL and MySQL monitors
  4. Set severity labels: ProxySQL TCP down = P1 (page immediately), MySQL replica down = P2

Example Slack alert:

{
  "monitor_name": "ProxySQL TCP :6033",
  "status": "down",
  "host": "proxysql.yourcompany.com",
  "port": 6033,
  "started_at": "2024-04-22T03:15:00Z",
  "duration_seconds": 45
}

PagerDuty escalation:

For database proxy outages, consider routing alerts to PagerDuty for on-call escalation:

  1. Go to Alert Channels → Add Channel → Webhook
  2. Use your PagerDuty Events API v2 URL
  3. Assign only to the ProxySQL TCP monitor and health HTTP monitor (these indicate active user impact)

Status page:

  1. Go to Status Pages → New Status Page
  2. Name it "Database Infrastructure"
  3. Add ProxySQL TCP monitors, MySQL TCP monitors, and the health HTTP monitor
  4. Publish it and share with your backend engineering team

ProxySQL Docker Compose with health monitoring

version: "3.9"

services:
  proxysql:
    image: proxysql/proxysql:2.6
    ports:
      - "6033:6033"  # MySQL proxy port
      - "6032:6032"  # Admin interface
    volumes:
      - ./proxysql.cnf:/etc/proxysql.cnf
      - proxysql_data:/var/lib/proxysql
    healthcheck:
      test: ["CMD-SHELL", "mysql -u radmin -pradmin -h 127.0.0.1 -P6032 -e 'SELECT 1' 2>/dev/null"]
      interval: 10s
      timeout: 5s
      retries: 3
    restart: unless-stopped

  proxysql-health:
    build: ./proxysql-health-sidecar
    ports:
      - "9090:9090"
    environment:
      PROXYSQL_HOST: proxysql
      PROXYSQL_ADMIN_PORT: 6032
      PROXYSQL_ADMIN_USER: radmin
      PROXYSQL_ADMIN_PASS: ${PROXYSQL_ADMIN_PASS}
      HEALTH_PORT: 9090
    depends_on:
      proxysql:
        condition: service_healthy
    restart: unless-stopped

  mysql-primary:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: app
      MYSQL_USER: app
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    volumes:
      - mysql_primary_data:/var/lib/mysql
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  proxysql_data:
  mysql_primary_data:

Monitoring checklist

| What to monitor | Type | Target | Alert on | |---|---|---|---| | ProxySQL proxy port | TCP | proxysql-host:6033 | Connection refused | | ProxySQL admin port | TCP | proxysql-host:6032 | Connection refused | | Health sidecar | HTTP | /health | Non-200 or no_online_backends | | MySQL primary | TCP | mysql-primary:3306 | Connection refused | | MySQL replicas | TCP | mysql-replica-N:3306 | Connection refused |

Summary

ProxySQL is the invisible layer between your applications and your database — when it fails, everything fails simultaneously. A three-tier monitoring approach (TCP port check on the proxy, HTTP health check verifying backend server status, and direct TCP checks on MySQL) gives you precise failure attribution within 60 seconds. Vigilmon's free tier supports all these monitors with 1-minute check intervals, Slack alerts, and a public status page for your database infrastructure team.

Start monitoring ProxySQL for free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles