How to Monitor Resque with Vigilmon

Resque is one of the oldest and most widely deployed background job libraries in the Ruby ecosystem. Built on Redis and used at GitHub since 2009, it provides a simple and reliable model for enqueuing and processing background work. Thousands of Rails applications still depend on it for everything from sending emails to processing payments.

But Resque's simplicity is also its blind spot. Workers that die without restart, queues that grow unbounded, or failed jobs sitting in the failure list unacknowledged can all break your application silently while your web tier continues serving traffic. Vigilmon gives you external visibility into Resque health through HTTP monitors for queue health endpoints and heartbeat monitors for worker processes.

Why Resque Needs External Monitoring

Resque ships with a web UI (resque-web), but it's a pull interface — you have to look at it. External monitoring with Vigilmon adds:

• Proactive alerting when worker processes exit and aren't restarted by your process supervisor
• Queue depth threshold checks before backlogs grow unacceptably large
• Failed job count monitoring to catch recurring exceptions before the failure list becomes a graveyard
• Redis connectivity checks — Resque is wholly dependent on Redis; a dropped connection silently stops all processing
• Heartbeat monitoring for scheduled workers that must prove they are executing on time

Step 1: Build a Resque Health Endpoint

Add a Sinatra or Rails controller that queries Resque's Redis state:

# app/controllers/health/resque_controller.rb
module Health
  class ResqueController < ApplicationController
    skip_before_action :authenticate_user!, raise: false

    QUEUED_THRESHOLD = (ENV['RESQUE_QUEUED_THRESHOLD'] || 500).to_i
    FAILED_THRESHOLD = (ENV['RESQUE_FAILED_THRESHOLD'] || 50).to_i

    def show
      issues = []

      # Check Redis connectivity first
      begin
        Resque.redis.ping
      rescue Redis::BaseError => e
        render json: { status: 'down', reason: 'redis_unreachable', error: e.message },
               status: :service_unavailable
        return
      end

      workers       = Resque.workers.count
      working       = Resque.working.count
      failed        = Resque::Failure.count
      queue_stats   = {}

      Resque.queues.each do |queue|
        size = Resque.size(queue)
        queue_stats[queue] = size
        issues << "queue_#{queue}_backlog_#{size}" if size > QUEUED_THRESHOLD
      end

      issues << "failed_jobs_#{failed}" if failed > FAILED_THRESHOLD
      issues << "no_workers"            if workers.zero? && queue_stats.values.sum > 0

      stats = {
        workers:  workers,
        working:  working,
        failed:   failed,
        queues:   queue_stats,
      }

      if issues.any?
        render json: { status: 'degraded', issues: issues, stats: stats }, status: :service_unavailable
      else
        render json: { status: 'ok', stats: stats }
      end
    end
  end
end

Register the route:

# config/routes.rb
Rails.application.routes.draw do
  namespace :health do
    resource :resque, only: :show
  end
  # ...
end

Test locally:

curl http://localhost:3000/health/resque
# {"status":"ok","stats":{"workers":4,"working":1,"failed":0,"queues":{"default":0,"mailers":2}}}

Step 2: Monitor Worker Heartbeats Directly

Resque workers set a processing key in Redis while working. You can check for stale workers that haven't updated their heartbeat:

# app/controllers/health/resque_controller.rb (add stale action)

STALE_WORKER_THRESHOLD_SECONDS = (ENV['RESQUE_STALE_SECONDS'] || 300).to_i

def stale
  stale_workers = []

  Resque.workers.each do |worker|
    next unless worker.working?

    processing  = worker.processing
    run_at      = processing['run_at']
    next if run_at.nil?

    age = Time.now - Time.parse(run_at.to_s)
    stale_workers << worker.to_s if age > STALE_WORKER_THRESHOLD_SECONDS
  end

  if stale_workers.any?
    render json: { status: 'degraded', stale_workers: stale_workers }, status: :service_unavailable
  else
    render json: { status: 'ok', stale_workers: [] }
  end
end

Add the route:

namespace :health do
  resource :resque, only: :show do
    get :stale, on: :collection
  end
end

Step 3: Add Heartbeat Pings to Job Classes

HTTP queue checks confirm Redis and the queue table are healthy. Heartbeats confirm specific jobs are actually executing:

# app/jobs/subscription_renewal_job.rb
class SubscriptionRenewalJob
  @queue = :billing

  def self.perform(subscription_id)
    RenewalService.process(subscription_id)

    # Ping heartbeat only on success — a missed ping IS the alert
    if (url = ENV['VIGILMON_SUBSCRIPTION_RENEWAL_HEARTBEAT'])
      Net::HTTP.get(URI(url)) rescue nil
    end
  end
end

For jobs triggered on a schedule via resque-scheduler:

# config/resque_schedule.yml
nightly_cleanup:
  cron: "0 3 * * *"
  class: NightlyCleanupJob
  queue: maintenance
  description: "Clean up stale records nightly"

hourly_digest:
  cron: "0 * * * *"
  class: HourlyDigestJob
  queue: default

Each scheduled job pings its own heartbeat URL:

class NightlyCleanupJob
  @queue = :maintenance

  def self.perform
    CleanupService.run

    Net::HTTP.get(URI(ENV['VIGILMON_NIGHTLY_CLEANUP_HEARTBEAT'])) rescue nil
  end
end

For high-frequency jobs, ping every N completions instead of every run to avoid excessive outbound requests:

class OrderProcessingJob
  @queue = :orders

  HEARTBEAT_EVERY = (ENV['HEARTBEAT_EVERY'] || '20').to_i
  @@count = Concurrent::AtomicFixnum.new(0)

  def self.perform(order_id)
    OrderService.fulfill(order_id)

    n = @@count.increment
    if n % HEARTBEAT_EVERY == 0
      Net::HTTP.get(URI(ENV['VIGILMON_ORDER_HEARTBEAT'])) rescue nil
    end
  end
end

Step 4: Configure Vigilmon HTTP Monitors

1. Log in to vigilmon.online and go to Monitors → New Monitor
2. Choose HTTP / HTTPS
3. Set the URL: https://your-app.example.com/health/resque
4. Set the check interval to 1 minute
5. Under Expected response, configure:
   • Status code: 200
   • Response body contains: "status":"ok"
   • Response time threshold: 3000ms
6. Under Alert channels, assign your Slack or email integration
7. Save the monitor

Add monitors for each concern:

| Monitor URL | Purpose | Interval | |---|---|---| | /health/resque | Queue depth, failed count, worker presence | 1 min | | /health/resque/stale | Long-running stale worker detection | 2 min |

Step 5: Configure Heartbeat Monitors for Workers

1. In Vigilmon, go to Monitors → New Monitor → Heartbeat
2. Set the name: resque-nightly-cleanup
3. Set the expected interval matching your cron schedule (e.g. 24 hours)
4. Set the grace period: 30 minutes
5. Save — copy the heartbeat URL
6. Set the corresponding environment variable in production

Create one heartbeat monitor per critical scheduled job:

| Job Class | Heartbeat Interval | Grace Period | |---|---|---| | NightlyCleanupJob | 24 hours | 30 min | | HourlyDigestJob | 1 hour | 10 min | | SubscriptionRenewalJob | 6 hours | 15 min |

Step 6: Alert Routing

| Monitor | Alert Channel | Priority | |---|---|---| | Queue health /health/resque | Slack + PagerDuty | P1 | | Stale workers /health/resque/stale | Slack | P2 | | Worker heartbeats | Slack + PagerDuty | P1 |

Key thresholds to tune for your workload:

• Queued threshold: set per queue — your mailer queue and your report queue have very different normal depths
• Failed threshold: keep low — failed jobs in Resque don't auto-retry by default, so each failure is a permanent loss unless you have resque-retry configured
• Stale window: 300 seconds is a reasonable default; increase for jobs that legitimately run for several minutes
• Redis connectivity: always alert immediately — there is no Resque without Redis

Summary

Resque failures are silent — a dead worker process doesn't crash your web tier, a full failure queue doesn't page anyone, and a Redis connection drop leaves jobs sitting in limbo. External monitoring with Vigilmon catches problems before they become incidents:

| Monitor Type | What It Covers | |---|---| | HTTP monitor on /health/resque | Queue depth, failure count, worker presence, Redis health | | HTTP monitor on /health/resque/stale | Long-running stuck worker detection | | Heartbeat monitor | Scheduled job execution on time |

Get started free at vigilmon.online — your first Resque monitor is running in under two minutes.