tutorial

Monitoring Google Cloud Memorystore with Vigilmon: Redis and Memcached Availability, Latency, and Worker Health

Practical guide to uptime and health monitoring for Google Cloud Memorystore — HTTP health endpoints that probe real instance connectivity, memory pressure detection, and independent external monitoring with Vigilmon.

Google Cloud Memorystore is a fully managed in-memory data service for Redis and Memcached, eliminating the operational burden of self-managing cache infrastructure. But managed does not mean self-monitoring: connection quota exhaustion, memory pressure nearing eviction thresholds, a misconfigured VPC routing rule, and silent worker stalls all cause real application degradation that Cloud Monitoring alone may not surface from your application's perspective. Vigilmon adds an independent external layer — HTTP health checks on your Memorystore-connected services that verify real instance connectivity, surface memory pressure, and alert on application-level failures before your users notice.

This tutorial shows you how to build meaningful monitoring for Memorystore-backed applications using Vigilmon.

What You'll Build

  • A health endpoint that probes real Memorystore connectivity and checks memory pressure
  • Memory usage and eviction rate detection
  • A Vigilmon HTTP monitor with appropriate timeout and assertion settings
  • A heartbeat monitor for Memorystore-backed background workers
  • A multi-instance alerting strategy for Redis and Memcached tiers

Prerequisites

  • A running Google Cloud Memorystore instance (Redis or Memcached)
  • An application connected to Memorystore (any runtime)
  • A free account at vigilmon.online

Step 1: Add a Health Endpoint to Your Memorystore Service

Add a /health endpoint that performs a lightweight operation against Memorystore to verify real connectivity and check memory pressure. Memorystore instances are only reachable from within the same VPC network — your health endpoint runs inside that network alongside your application code.

Node.js (ioredis)

// routes/health.js
import Redis from 'ioredis';

const redis = new Redis({
  host: process.env.MEMORYSTORE_HOST, // e.g. 10.0.0.3 (private IP from GCP console)
  port: parseInt(process.env.MEMORYSTORE_PORT || '6379'),
  connectTimeout: 5000,
  commandTimeout: 5000,
  lazyConnect: true,
  maxRetriesPerRequest: 1,
});

export async function healthHandler(req, res) {
  const checks = {};
  let ok = true;
  const start = Date.now();

  // Check 1: PING — basic connectivity
  try {
    const pong = await redis.ping();
    if (pong !== 'PONG') throw new Error(`Unexpected PING response: ${pong}`);
    checks.connectivity = 'ok';
    checks.pingLatencyMs = Date.now() - start;
  } catch (err) {
    checks.connectivity = `error: ${err.message}`;
    ok = false;
  }

  // Check 2: Memory pressure
  try {
    const info = await redis.info('memory');
    const usedMatch = info.match(/used_memory:(\d+)/);
    const maxMatch = info.match(/maxmemory:(\d+)/);
    const evictedMatch = info.match(/evicted_keys:(\d+)/);

    const used = usedMatch ? parseInt(usedMatch[1]) : 0;
    const max = maxMatch ? parseInt(maxMatch[1]) : 0;
    const evicted = evictedMatch ? parseInt(evictedMatch[1]) : 0;

    checks.usedMemoryBytes = used;
    checks.evictedKeys = evicted;

    if (max > 0) {
      const pct = Math.round((used / max) * 100);
      checks.memoryUsagePct = pct;
      const threshold = parseInt(process.env.MAX_MEMORY_PCT || '85');
      if (pct > threshold) {
        checks.memoryHealth = `high: ${pct}% — eviction risk`;
        ok = false;
      } else {
        checks.memoryHealth = 'ok';
      }
    }
  } catch (err) {
    checks.memoryHealth = `error: ${err.message}`;
    ok = false;
  }

  // Check 3: Connected clients
  try {
    const clients = await redis.info('clients');
    const connectedMatch = clients.match(/connected_clients:(\d+)/);
    const maxMatch = clients.match(/maxclients:(\d+)/);
    const connected = connectedMatch ? parseInt(connectedMatch[1]) : 0;
    const max = maxMatch ? parseInt(maxMatch[1]) : 0;
    checks.connectedClients = connected;
    if (max > 0 && connected / max > 0.9) {
      checks.clientHealth = `connection quota high: ${connected}/${max}`;
      ok = false;
    } else {
      checks.clientHealth = 'ok';
    }
  } catch (err) {
    checks.clientHealth = `error: ${err.message}`;
  }

  res.status(ok ? 200 : 503).json({
    status: ok ? 'ok' : 'degraded',
    service: 'memorystore-service',
    checks,
  });
}

Python (redis-py)

# routers/health.py
import os, time
import redis as redis_lib
from fastapi import APIRouter
from fastapi.responses import JSONResponse

router = APIRouter()

r = redis_lib.Redis(
    host=os.environ['MEMORYSTORE_HOST'],
    port=int(os.environ.get('MEMORYSTORE_PORT', 6379)),
    socket_connect_timeout=5,
    socket_timeout=5,
    decode_responses=True,
)

@router.get("/health")
async def health():
    checks = {}
    ok = True
    start = time.monotonic()

    # Check 1: PING
    try:
        pong = r.ping()
        if not pong:
            raise RuntimeError('PING returned falsy')
        checks['connectivity'] = 'ok'
        checks['pingLatencyMs'] = int((time.monotonic() - start) * 1000)
    except Exception as e:
        checks['connectivity'] = f'error: {e}'
        ok = False

    # Check 2: Memory pressure
    try:
        info = r.info('memory')
        used = info.get('used_memory', 0)
        max_mem = info.get('maxmemory', 0)
        evicted = info.get('evicted_keys', 0)
        checks['usedMemoryBytes'] = used
        checks['evictedKeys'] = evicted
        if max_mem > 0:
            pct = round(used / max_mem * 100, 1)
            checks['memoryUsagePct'] = pct
            threshold = float(os.environ.get('MAX_MEMORY_PCT', '85'))
            if pct > threshold:
                checks['memoryHealth'] = f'high: {pct}% — eviction risk'
                ok = False
            else:
                checks['memoryHealth'] = 'ok'
    except Exception as e:
        checks['memoryHealth'] = f'error: {e}'
        ok = False

    # Check 3: Connected clients
    try:
        client_info = r.info('clients')
        connected = client_info.get('connected_clients', 0)
        max_clients = client_info.get('maxclients', 0)
        checks['connectedClients'] = connected
        if max_clients > 0 and connected / max_clients > 0.9:
            checks['clientHealth'] = f'connection quota high: {connected}/{max_clients}'
            ok = False
        else:
            checks['clientHealth'] = 'ok'
    except Exception as e:
        checks['clientHealth'] = f'error: {e}'

    return JSONResponse(
        status_code=200 if ok else 503,
        content={'status': 'ok' if ok else 'degraded', 'service': 'memorystore-service', 'checks': checks},
    )

Go (go-redis)

// internal/health/handler.go
package health

import (
    "context"
    "encoding/json"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"

    "github.com/redis/go-redis/v9"
)

func Handler(rdb *redis.Client) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
        defer cancel()

        checks := map[string]any{}
        ok := true

        // Check 1: PING
        start := time.Now()
        if err := rdb.Ping(ctx).Err(); err != nil {
            checks["connectivity"] = "error: " + err.Error()
            ok = false
        } else {
            checks["connectivity"] = "ok"
            checks["pingLatencyMs"] = time.Since(start).Milliseconds()
        }

        // Check 2: Memory pressure
        memInfo, err := rdb.Info(ctx, "memory").Result()
        if err == nil {
            checks["memoryHealth"] = parseMemoryHealth(memInfo)
            if checks["memoryHealth"] != "ok" {
                ok = false
            }
        } else {
            checks["memoryHealth"] = "error: " + err.Error()
        }

        // Check 3: Connected clients
        clientInfo, err := rdb.Info(ctx, "clients").Result()
        if err == nil {
            checks["clientHealth"] = parseClientHealth(clientInfo)
            if checks["clientHealth"] != "ok" {
                ok = false
            }
        }

        w.Header().Set("Content-Type", "application/json")
        if !ok {
            w.WriteHeader(http.StatusServiceUnavailable)
        }
        _ = json.NewEncoder(w).Encode(map[string]any{
            "status":  map[bool]string{true: "ok", false: "degraded"}[ok],
            "service": "memorystore-service",
            "checks":  checks,
        })
    }
}

func parseMemoryHealth(info string) string {
    threshold, _ := strconv.ParseFloat(os.Getenv("MAX_MEMORY_PCT"), 64)
    if threshold == 0 {
        threshold = 85
    }
    var used, max int64
    for _, line := range strings.Split(info, "\r\n") {
        if strings.HasPrefix(line, "used_memory:") {
            used, _ = strconv.ParseInt(strings.TrimPrefix(line, "used_memory:"), 10, 64)
        }
        if strings.HasPrefix(line, "maxmemory:") {
            max, _ = strconv.ParseInt(strings.TrimPrefix(line, "maxmemory:"), 10, 64)
        }
    }
    if max > 0 {
        pct := float64(used) / float64(max) * 100
        if pct > threshold {
            return "high: eviction risk"
        }
    }
    return "ok"
}

func parseClientHealth(info string) string {
    var connected, max int64
    for _, line := range strings.Split(info, "\r\n") {
        if strings.HasPrefix(line, "connected_clients:") {
            connected, _ = strconv.ParseInt(strings.TrimPrefix(line, "connected_clients:"), 10, 64)
        }
        if strings.HasPrefix(line, "maxclients:") {
            max, _ = strconv.ParseInt(strings.TrimPrefix(line, "maxclients:"), 10, 64)
        }
    }
    if max > 0 && float64(connected)/float64(max) > 0.9 {
        return "connection quota high"
    }
    return "ok"
}

Deploy and verify manually before wiring it into Vigilmon:

curl -i https://your-app.example.com/health
# HTTP/1.1 200 OK
# {"status":"ok","service":"memorystore-service","checks":{"connectivity":"ok","pingLatencyMs":1,"memoryHealth":"ok","clientHealth":"ok"}}

Step 2: Configure the Vigilmon HTTP Monitor

  1. Log in to Vigilmon → Add Monitor → HTTP.
  2. URL: your service's health endpoint (e.g. https://api.example.com/health).
  3. Check interval: 60 seconds.
  4. Response timeout: 10 seconds.
  5. Expected status: 200.
  6. JSON assertion: path status, expected value ok.
  7. Save the monitor.

Vigilmon probes from multiple geographic regions simultaneously. A single-region blip will not page you; multi-region consensus is required before an incident opens. This matters for Memorystore because VPC peering misconfigurations or quota exhaustion may affect some application pods but not others.

Tip: Expose checks.memoryUsagePct from your health endpoint and add a second Vigilmon monitor with a numeric assertion — alert at 80% as an early warning before the default 85% threshold triggers a 503.

Step 3: Heartbeat Monitoring for Memorystore-Backed Workers

Memorystore is commonly used as a job queue backend (BullMQ, RQ, Celery with Redis broker), session store, and rate-limit counter. Workers that depend on Memorystore will stall silently when the instance becomes unreachable or when the connection quota is exhausted. Vigilmon's heartbeat monitors detect silent worker death: your worker pings Vigilmon after each successful processing cycle, and if pings stop arriving, Vigilmon fires an alert.

Set Up the Heartbeat Monitor

  1. In Vigilmon, go to Monitors → New Monitor → Heartbeat.
  2. Set the name: memorystore-worker.
  3. Set the expected interval: 2 minutes (adjust to your job frequency).
  4. Set the grace period: 5 minutes.
  5. Save — copy the unique heartbeat URL.

Wire It Into Your Worker

Node.js / BullMQ:

import { Worker } from 'bullmq';
import fetch from 'node-fetch';

const worker = new Worker('my-queue', async (job) => {
  await processJob(job);
}, {
  connection: {
    host: process.env.MEMORYSTORE_HOST,
    port: parseInt(process.env.MEMORYSTORE_PORT || '6379'),
  },
});

worker.on('completed', () => {
  fetch(process.env.VIGILMON_HEARTBEAT_URL).catch(() => {});
});

Python / Celery:

from celery import Celery
from celery.signals import task_success
import requests, os

app = Celery('tasks', broker=f"redis://{os.environ['MEMORYSTORE_HOST']}:6379/0")

@task_success.connect
def on_task_success(sender=None, **kwargs):
    requests.get(os.environ['VIGILMON_HEARTBEAT_URL'], timeout=5)

Step 4: Memcached Instance Monitoring

If you use Memorystore for Memcached rather than Redis, the monitoring approach is similar but uses Memcached's text protocol instead of Redis commands.

# Python health check for Memcached via pymemcache
import os, time
from pymemcache.client.base import Client
from fastapi import APIRouter
from fastapi.responses import JSONResponse

router = APIRouter()
mc = Client((os.environ['MEMORYSTORE_HOST'], int(os.environ.get('MEMORYSTORE_PORT', 11211))))

@router.get("/health/memcached")
async def memcached_health():
    checks = {}
    ok = True
    start = time.monotonic()

    # Write and read a probe key
    try:
        mc.set('vigilmon-probe', b'ok', expire=30)
        val = mc.get('vigilmon-probe')
        if val != b'ok':
            raise ValueError('Probe value mismatch')
        checks['connectivity'] = 'ok'
        checks['pingLatencyMs'] = int((time.monotonic() - start) * 1000)
    except Exception as e:
        checks['connectivity'] = f'error: {e}'
        ok = False

    # Stats — check curr_connections and evictions
    try:
        stats = mc.stats()
        curr_conn = int(stats.get(b'curr_connections', 0))
        evictions = int(stats.get(b'evictions', 0))
        checks['currConnections'] = curr_conn
        checks['evictions'] = evictions
        if evictions > 0:
            checks['cacheHealth'] = f'evictions detected: {evictions}'
        else:
            checks['cacheHealth'] = 'ok'
    except Exception as e:
        checks['cacheHealth'] = f'error: {e}'

    return JSONResponse(
        status_code=200 if ok else 503,
        content={'status': 'ok' if ok else 'degraded', 'service': 'memcached-service', 'checks': checks},
    )

Step 5: Alerting Strategy

| Alert channel | When to use | |---|---| | PagerDuty | Connectivity check fails (instance unreachable from VPC) | | Slack webhook | Memory usage > 80% (approaching eviction threshold) | | Slack + Email | Memory usage > 85% and evictions detected | | PagerDuty | Heartbeat missed (worker stopped processing) | | Slack | Connection quota > 90% (client exhaustion risk) | | Status page | Any user-facing service backed by Memorystore |

Set response time thresholds as an early warning:

  • Alert at 5ms for Memorystore operation latency (sub-millisecond is normal; 5ms+ signals instance pressure)
  • Alert at 500ms for health endpoint response time

What Vigilmon Catches That Internal Monitoring Misses

| Scenario | Cloud Monitoring | Vigilmon | |---|---|---| | VPC routing misconfiguration (app cannot reach instance) | No | HTTP probe fails — 503 fires | | Memory approaching eviction threshold | Metric alert only | Health endpoint surfaces percentage | | Connection quota exhausted (new connections refused) | Metric alert only | Health endpoint client check fails | | Worker silently stopped processing | Process appears running | Heartbeat grace period expires → alert | | Memcached evictions silently degrading cache hit rate | No | Stats check surfaces eviction count | | Cloud Monitoring pipeline delayed or down | Alerts silently lost | Vigilmon is external — unaffected |

Google Cloud Memorystore failures cascade fast: a cache miss storm can take down your database tier in seconds. External health checks with Vigilmon give you the independent signal you need — before latency spikes reach your users.

Start monitoring your Memorystore applications in under 5 minutes — register free at vigilmon.online.

Monitor your app with Vigilmon

Free plan — 5 monitors, no credit card required. Up and running in 60 seconds.

Start free →
← Back to all articles