tutorial

How to Monitor Solace PubSub+ with Vigilmon

Solace PubSub+ powers enterprise event meshes — but silent queue buildup, HA failovers, and VPN degradation can stall your integration layer without warning. Learn how to monitor Solace message VPN health, queue depth, throughput, and HA pair status with Vigilmon.

Solace PubSub+ is the backbone of many enterprise integration platforms — routing millions of events per second across financial systems, logistics pipelines, and IoT infrastructure. But when a message VPN degrades, a queue backs up beyond its high-water mark, or an HA pair loses synchronization, your event mesh silently stalls while downstream consumers wait.

Unlike HTTP services, Solace failures don't surface as obvious 500 errors. Queue depth climbs, throughput drops, and consumer group lag spikes — often for minutes before anyone notices. Vigilmon gives you external visibility into Solace health through HTTP probe monitoring against the Solace SEMP (Solace Element Management Protocol) REST API and heartbeat monitors for your consumer applications.

Why Solace Needs External Monitoring

Solace ships with PubSub+ Monitor and the Solace CLI for internal metrics, but these require you to be watching. External monitoring with Vigilmon adds:

  • Proactive alerting when message VPN health endpoints return non-200 or report degraded state
  • Queue depth threshold monitoring before queues hit their maximum message count
  • HA pair status checks so you know immediately if the standby broker loses sync
  • Heartbeat monitoring for consumer applications that must prove they are actively processing
  • Multi-region probe consensus to avoid false alerts from transient network blips

Step 1: Enable the Solace SEMP REST API

Solace exposes broker health and statistics through its SEMP v2 REST API. Enable it on your Solace PS+ software broker or hardware appliance:

# Verify SEMP is enabled (Solace CLI)
solace> show service semp

# SEMP v2 is available at:
# http://<broker-host>:8080/SEMP/v2/monitor/
# https://<broker-host>:943/SEMP/v2/monitor/

Create a read-only SEMP user for monitoring:

solace> configure
solace(configure)> username vigilmon-monitor password <strong-password>
solace(configure)> username vigilmon-monitor global-access-level read-only
solace(configure)> end

Test the SEMP API

# Get message VPN health
curl -u vigilmon-monitor:<password> \
  "http://localhost:8080/SEMP/v2/monitor/msgVpns/default" \
  | jq '.data.enabled, .data.operationalState'

# Get all queues in a VPN
curl -u vigilmon-monitor:<password> \
  "http://localhost:8080/SEMP/v2/monitor/msgVpns/default/queues?select=queueName,msgCount,maxMsgCount,bindCount" \
  | jq '.data[]'

Step 2: Build a Solace Health Endpoint

SEMP responses contain rich data but need interpretation before Vigilmon can probe them. Build a lightweight health sidecar that queries SEMP and surfaces a clean /health endpoint:

# solace_health.py
from flask import Flask, jsonify
import requests, os

app = Flask(__name__)

SEMP_BASE = os.environ.get('SOLACE_SEMP_URL', 'http://localhost:8080/SEMP/v2/monitor')
SEMP_USER = os.environ.get('SOLACE_SEMP_USER', 'admin')
SEMP_PASS = os.environ.get('SOLACE_SEMP_PASS', 'admin')
VPN_NAME  = os.environ.get('SOLACE_VPN', 'default')
QUEUE_FILL_THRESHOLD = float(os.environ.get('SOLACE_QUEUE_FILL_THRESHOLD', '0.80'))

def semp_get(path):
    r = requests.get(f"{SEMP_BASE}{path}",
                     auth=(SEMP_USER, SEMP_PASS), timeout=5)
    r.raise_for_status()
    return r.json()

@app.route('/health/solace')
def health():
    issues = []

    # Check message VPN operational state
    vpn = semp_get(f'/msgVpns/{VPN_NAME}')['data']
    if not vpn.get('enabled'):
        issues.append('message_vpn_disabled')
    if vpn.get('state') != 'Up':
        issues.append(f"vpn_state_{vpn.get('state', 'unknown')}")

    # Check queue depth across all queues in the VPN
    queues = semp_get(f'/msgVpns/{VPN_NAME}/queues'
                      '?select=queueName,msgCount,maxMsgCount,bindCount')['data']
    for q in queues:
        max_msgs = q.get('maxMsgCount', 0)
        if max_msgs > 0:
            fill = q.get('msgCount', 0) / max_msgs
            if fill >= QUEUE_FILL_THRESHOLD:
                issues.append(f"queue_{q['queueName']}_fill_{int(fill*100)}pct")
        if q.get('bindCount', 0) == 0:
            issues.append(f"queue_{q['queueName']}_no_consumers")

    if issues:
        return jsonify(status='degraded', issues=issues), 503

    return jsonify(
        status='ok',
        vpn=VPN_NAME,
        queue_count=len(queues),
        active_consumers=sum(q.get('bindCount', 0) for q in queues),
    ), 200

@app.route('/health/solace/throughput')
def throughput():
    vpn = semp_get(f'/msgVpns/{VPN_NAME}')['data']
    return jsonify(
        status='ok',
        msgs_rx_per_sec=vpn.get('averageRxMsgRate', 0),
        msgs_tx_per_sec=vpn.get('averageTxMsgRate', 0),
        bytes_rx_per_sec=vpn.get('averageRxByteRate', 0),
        bytes_tx_per_sec=vpn.get('averageTxByteRate', 0),
    ), 200

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=3005)

HA Pair Status Endpoint

For high-availability Solace deployments, add a dedicated HA check:

@app.route('/health/solace/ha')
def ha_status():
    try:
        # Solace HA status via SEMP
        redundancy = semp_get('/about/user/msgVpns')  # placeholder path
        # Hardware appliances: check /redundancy
        ha = semp_get('/redundancy') if '/redundancy' in SEMP_BASE else {}
        ha_data = ha.get('data', {})
        mode = ha_data.get('redundancyMode', 'NonRedundant')
        active_standby = ha_data.get('activeStandbyRole', 'Active')

        if mode != 'NonRedundant' and ha_data.get('configSyncState') not in ('Up', None):
            return jsonify(
                status='degraded',
                reason='ha_config_sync_down',
                role=active_standby,
            ), 503

        return jsonify(status='ok', mode=mode, role=active_standby), 200
    except Exception as e:
        return jsonify(status='unknown', error=str(e)), 200

# Run the health sidecar
pip install flask requests
SOLACE_SEMP_URL=http://your-broker:8080/SEMP/v2/monitor \
SOLACE_SEMP_USER=vigilmon-monitor \
SOLACE_SEMP_PASS=your-password \
SOLACE_VPN=production \
python solace_health.py

Step 3: Enable Prometheus Stats Endpoint

Solace PubSub+ can expose Prometheus-format metrics natively (software broker version 9.6+):

# Enable Prometheus telemetry on port 9087
solace> configure
solace(configure)> telemetry
solace(configure/telemetry)> receiver-acl-profile default
solace(configure/telemetry)> no shutdown
solace(configure/telemetry)> end

Your Prometheus endpoint will be at http://<broker>:9087/. To monitor it with Vigilmon, probe a Prometheus-to-HTTP adapter:

@app.route('/health/solace/prometheus')
def prometheus_health():
    r = requests.get('http://localhost:9087/', timeout=5)
    if r.status_code == 200 and 'solace_' in r.text:
        return jsonify(status='ok', metrics_available=True), 200
    return jsonify(status='down', reason='prometheus_endpoint_unavailable'), 503

Step 4: Configure Vigilmon HTTP Monitors

  1. Log in to vigilmon.online and go to Monitors → New Monitor
  2. Choose HTTP / HTTPS
  3. Set the URL to your Solace health endpoint: https://your-app.example.com/health/solace
  4. Set the check interval to 1 minute
  5. Under Expected response, configure:
    • Status code: 200
    • Response body contains: "status":"ok"
    • Response time threshold: 3000ms
  6. Under Alert channels, assign your Slack or PagerDuty integration
  7. Save the monitor

Add additional monitors for granular coverage:

| Monitor URL | Purpose | Interval | |---|---|---| | /health/solace | VPN health, queue depth, consumer bind | 1 min | | /health/solace/throughput | Message and byte rate trends | 1 min | | /health/solace/ha | HA pair sync status | 1 min | | /health/solace/prometheus | Prometheus metrics availability | 5 min |

Step 5: Heartbeat Monitoring for Solace Consumer Applications

Solace broker health is only half the picture — a consumer application can silently stall while the broker reports healthy. Vigilmon heartbeat monitors detect silent consumer failures: your application pings Vigilmon after each successful message batch, and if pings stop, Vigilmon fires an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
  2. Set the name: solace-order-consumer
  3. Set the expected interval: 5 minutes
  4. Set the grace period: 10 minutes
  5. Save — copy the heartbeat URL: https://vigilmon.online/heartbeat/abc123xyz

Wire It Into Your Consumer (Java — Solace JCSMP)

// SolaceConsumer.java
import com.solacesystems.jcsmp.*;
import java.net.http.*;
import java.net.URI;
import java.util.concurrent.atomic.AtomicLong;

public class SolaceConsumer {
    private static final String VIGILMON_HEARTBEAT =
        System.getenv("VIGILMON_HEARTBEAT_URL");
    private static final long HEARTBEAT_EVERY_N = 100;
    private static AtomicLong msgCount = new AtomicLong(0);
    private static final HttpClient http = HttpClient.newHttpClient();

    public static void main(String[] args) throws Exception {
        JCSMPProperties props = new JCSMPProperties();
        props.setProperty(JCSMPProperties.HOST, System.getenv("SOLACE_HOST"));
        props.setProperty(JCSMPProperties.VPN_NAME, System.getenv("SOLACE_VPN"));
        props.setProperty(JCSMPProperties.USERNAME, System.getenv("SOLACE_USER"));
        props.setProperty(JCSMPProperties.PASSWORD, System.getenv("SOLACE_PASS"));

        JCSMPSession session = JCSMPFactory.onlyInstance().createSession(props);
        session.connect();

        ConsumerFlowProperties flowProps = new ConsumerFlowProperties();
        flowProps.setEndpoint(
            JCSMPFactory.onlyInstance().createQueue(System.getenv("SOLACE_QUEUE"))
        );

        FlowReceiver receiver = session.createFlow(
            (BytesXMLMessage msg, JCSMPException ex) -> {
                if (ex != null) return;
                processMessage(msg);
                msg.ackMessage();

                if (msgCount.incrementAndGet() % HEARTBEAT_EVERY_N == 0) {
                    pingVigilmon();
                }
            },
            flowProps
        );
        receiver.start();

        // Keep-alive
        Thread.currentThread().join();
    }

    private static void pingVigilmon() {
        try {
            http.send(
                HttpRequest.newBuilder(URI.create(VIGILMON_HEARTBEAT)).GET().build(),
                HttpResponse.BodyHandlers.discarding()
            );
        } catch (Exception ignored) {}
    }

    private static void processMessage(BytesXMLMessage msg) {
        // your processing logic
    }
}

Python Consumer (solace-pubsubplus)

# consumer.py
import solace.messaging.messaging_service as mms
import requests, os, time

VIGILMON_URL = os.environ['VIGILMON_HEARTBEAT_URL']
last_heartbeat = time.time()
HEARTBEAT_INTERVAL = 60  # seconds

messaging_service = (mms.MessagingService.builder()
    .from_properties({
        "solace.messaging.transport.host": os.environ["SOLACE_HOST"],
        "solace.messaging.service.vpn-name": os.environ["SOLACE_VPN"],
        "solace.messaging.authentication.basic.username": os.environ["SOLACE_USER"],
        "solace.messaging.authentication.basic.password": os.environ["SOLACE_PASS"],
    })
    .build())
messaging_service.connect()

def message_handler(message):
    global last_heartbeat
    process(message)
    if time.time() - last_heartbeat > HEARTBEAT_INTERVAL:
        try:
            requests.get(VIGILMON_URL, timeout=3)
        except Exception:
            pass
        last_heartbeat = time.time()

persistent_receiver = (messaging_service
    .create_persistent_message_receiver_builder()
    .build(solace.messaging.resources.queue.Queue.durable_non_exclusive_queue(
        os.environ["SOLACE_QUEUE"]
    )))
persistent_receiver.start()
persistent_receiver.receive_async(message_handler)

Step 6: Alert Routing

Configure alert channels to match Solace failure severity:

| Monitor | Alert Channel | Priority | |---|---|---| | VPN health /health/solace | Slack + PagerDuty | P1 | | HA pair /health/solace/ha | Slack + PagerDuty | P1 | | Queue depth spike | Slack | P2 | | Heartbeat: consumer app | Slack + email | P2 | | Throughput drop | Email | P3 |

Set response time thresholds for early warning:

  • Alert at 2000ms for SEMP health queries (slowness signals broker load pressure)
  • Alert at 5000ms for HA status (HA sync operations can be heavy during active replication)

Summary

Solace PubSub+ failures are silent: VPN degradation and queue backpressure accumulate before dashboards catch up. External monitoring with Vigilmon closes this gap:

| Monitor Type | What It Covers | |---|---| | HTTP monitor on /health/solace | VPN health, queue depth, consumer bind counts | | HTTP monitor on /health/solace/ha | HA pair synchronization state | | HTTP monitor on /health/solace/throughput | Message and byte rate trends | | Heartbeat monitor | Consumer application liveness and processing health |

Get started free at vigilmon.online — your first Solace monitor is running in under two minutes.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles