Testing Sidekiq Jobs with RSpec

Sidekiq's testing API offers three modes that trade realism for speed, and this guide shows which to use for which question, as part of Testing Background Jobs in Backend Frameworks & Worker Scaling. It covers job logic, enqueue contracts, retry behaviour, middleware, and a thin integration layer that runs real jobs through real Redis.

Problem Statement

A Rails application's spec suite uses Sidekiq::Testing.inline! globally, so every perform_async runs immediately in the test. Tests are simple to write but three classes of bug keep reaching production: jobs enqueued with ActiveRecord objects instead of ids (which Sidekiq's JSON serialization turns into strings), a job's sidekiq_retries_exhausted hook that crashed on the message format and never alerted anyone, and a job that double-charged when Sidekiq retried it after a timeout. You want fast specs for logic, explicit assertions about enqueues and argument types, tests for the retry-exhausted path, and a small set of specs that exercise real Redis.

Prerequisites

  • Sidekiq 7+, RSpec 3, and rspec-sidekiq (optional, adds matchers like have_enqueued_sidekiq_job).
  • require "sidekiq/testing" in spec/rails_helper.rb for the testing modes.
  • Redis available in CI for the integration layer.
  • Jobs whose perform arguments are simple JSON types — the tests below enforce that.

Step 1 — Default to Fake Mode

Sidekiq::Testing.fake! pushes jobs onto in-memory arrays per job class instead of Redis. Nothing runs until you drain. Make it the global default; inline mode hides the enqueue boundary that most bugs live on.

# spec/rails_helper.rb
require "sidekiq/testing"
Sidekiq::Testing.fake!                    # default: enqueue into arrays, don't run

RSpec.configure do |config|
  config.before { Sidekiq::Worker.clear_all }          # empty every fake queue per example
  config.around(:each, :sidekiq_inline) { |ex| Sidekiq::Testing.inline! { ex.run } }
  config.around(:each, :sidekiq_disabled) { |ex| Sidekiq::Testing.disable! { ex.run } }
end

Examples can opt into inline or real Redis with metadata, which keeps the choice visible in each spec.

Three testing modes In fake mode, perform_async appends the job to an in-memory array for the job class and nothing runs until the test drains it. In inline mode, perform_async serializes and immediately runs the job in the test process. In disabled mode, perform_async pushes to real Redis, where a real Sidekiq process or a test helper picks it up. Where perform_async goes in each mode fake! (default) array per job class assert, then drain inline! runs immediately no retries, no Redis disable! real Redis push integration specs only Choose per example with metadata; never switch the global mode inside a spec.

Step 2 — Assert Enqueues and Argument Types

In fake mode, MyJob.jobs is the array of enqueued payloads — the same hashes that would go to Redis, with arguments already JSON-round-tripped. That makes argument-type bugs visible.

RSpec.describe InvoicesController, type: :request do
  it "enqueues the email job with the invoice id after create" do
    post "/invoices", params: { invoice: { amount_cents: 1200 } }
    expect(SendInvoiceEmailJob.jobs.size).to eq(1)
    job = SendInvoiceEmailJob.jobs.first
    expect(job["args"]).to eq([Invoice.last.id])       # an Integer id, not a model
    expect(job["queue"]).to eq("mailers")
  end

  it "does not enqueue when the invoice fails validation" do
    post "/invoices", params: { invoice: { amount_cents: -1 } }
    expect(SendInvoiceEmailJob.jobs).to be_empty
  end
end

# With rspec-sidekiq matchers
expect(SendInvoiceEmailJob).to have_enqueued_sidekiq_job(invoice.id).on("mailers")

Sidekiq 7 raises on non-native JSON arguments when Sidekiq.strict_args! is enabled; turn it on in the test environment so passing a model, a Time, or a symbol fails the spec instead of arriving as a surprising string in production.

# config/initializers/sidekiq.rb
Sidekiq.strict_args! if Rails.env.test? || Rails.env.development?

Step 3 — Test Logic by Calling perform

Job logic is a Ruby method; call it directly with the arguments a real enqueue would produce. For idempotency, call it twice.

RSpec.describe ChargeInvoiceJob do
  let(:invoice) { create(:invoice, status: "open", amount_cents: 1200) }

  it "charges once even when run twice" do
    gateway = instance_double(PaymentGateway, charge: double(id: "ch_1"))
    allow(PaymentGateway).to receive(:new).and_return(gateway)

    2.times { described_class.new.perform(invoice.id) }

    expect(gateway).to have_received(:charge).once
    expect(invoice.reload.status).to eq("paid")
  end

  it "passes a stable idempotency key to the gateway" do
    gateway = spy(PaymentGateway, charge: double(id: "ch_1"))
    allow(PaymentGateway).to receive(:new).and_return(gateway)
    described_class.new.perform(invoice.id)
    expect(gateway).to have_received(:charge).with(hash_including(idempotency_key: "invoice-#{invoice.id}"))
  end
end

The idempotency key assertion protects the case a run-twice test cannot: a crash after the gateway call but before status is updated. The pattern is described in preventing duplicate job execution with idempotency.

Two specs, two failure windows The run-twice spec proves that once the invoice is marked paid, a second run skips the charge. The idempotency-key spec covers the window where the gateway charged but the job crashed before marking the invoice paid; the retry sends the same key and the gateway returns the original charge instead of creating another. What each spec protects gateway.charge status = paid job acked crash here: idempotency-key spec crash here: run-twice spec Left window: charged but not marked paid. Right window: marked paid but not acked.

Step 4 — Test Retry Options and the Exhausted Hook

sidekiq_options retry: and sidekiq_retry_in are configuration worth pinning, and sidekiq_retries_exhausted is code that only runs in the worst case — exactly when you need it to work.

class ChargeInvoiceJob
  include Sidekiq::Job
  sidekiq_options queue: "billing", retry: 8

  sidekiq_retry_in do |count, exception, _jobhash|
    case exception
    when PaymentGateway::CardDeclined then :kill          # no retry: straight to the dead set
    when PaymentGateway::RateLimited  then 60 * (count + 1)
    else (count ** 4) + 15 + rand(10 * (count + 1))       # Sidekiq's default-style curve
    end
  end

  sidekiq_retries_exhausted do |job, exception|
    Invoice.find(job["args"].first).update!(status: "payment_failed")
    BillingAlerts.notify(invoice_id: job["args"].first, error: exception.message)
  end
end

RSpec.describe ChargeInvoiceJob do
  it "kills card declines instead of retrying" do
    expect(described_class.sidekiq_retry_in_block.call(0, PaymentGateway::CardDeclined.new, {})).to eq(:kill)
  end

  it "marks the invoice failed and alerts when retries are exhausted" do
    invoice = create(:invoice, status: "open")
    job = { "class" => described_class.name, "args" => [invoice.id], "retry_count" => 8 }
    expect(BillingAlerts).to receive(:notify).with(hash_including(invoice_id: invoice.id))
    described_class.sidekiq_retries_exhausted_block.call(job, RuntimeError.new("boom"))
    expect(invoice.reload.status).to eq("payment_failed")
  end

  it "retries at most 8 times" do
    expect(described_class.get_sidekiq_options["retry"]).to eq(8)
  end
end

It helps to see the three paths a failure can take, because each one needs its own spec. A card decline returns :kill and goes straight to the dead set — the exhausted hook still runs, so the invoice is marked failed immediately. A rate limit returns a linear delay and retries. Anything else follows the polynomial curve until the eighth retry, and only then reaches the hook. A spec suite that exercises only the "anything else" path leaves the other two untested, and those are the paths with business meaning.

Three failure paths, three specs The sidekiq_retry_in block classifies the exception. CardDeclined returns kill, sending the job to the dead set and running the exhausted hook at once. RateLimited returns a delay of sixty seconds times the attempt number. Any other error uses a polynomial backoff, and after eight retries the exhausted hook marks the invoice failed and alerts. sidekiq_retry_in decides the path perform raises CardDeclined: :kill, dead set, exhausted hook now RateLimited: retry in 60 s x (count + 1) other: polynomial backoff, hook after retry 8

The exhausted-hook spec feeds the hook a job hash in the same shape Sidekiq passes — string keys, args as an array — which is the format mismatch that broke it in the problem statement. Retry design in general is covered in setting retry budgets and max attempts.

Step 5 — Test Server Middleware in Isolation

Server middleware wraps every job — tenancy, logging context, prioritisation — and a bug there affects all jobs. Test it by calling call with a job hash and a block.

class TenantMiddleware
  include Sidekiq::ServerMiddleware
  def call(_job_instance, job, _queue)
    Current.tenant_id = job["tenant_id"] or raise ArgumentError, "job missing tenant_id"
    yield
  ensure
    Current.reset
  end
end

RSpec.describe TenantMiddleware do
  it "sets and resets the tenant around the job" do
    seen = nil
    described_class.new.call(nil, { "tenant_id" => "t-9" }, "default") { seen = Current.tenant_id }
    expect(seen).to eq("t-9")
    expect(Current.tenant_id).to be_nil
  end

  it "refuses jobs without a tenant" do
    expect { described_class.new.call(nil, {}, "default") { } }.to raise_error(ArgumentError)
  end
end

The ensure block matters: Sidekiq threads are reused, and a tenant left set from one job leaks into the next. Middleware-based prioritisation is discussed in Sidekiq middleware for job prioritization.

Step 6 — Add a Thin Integration Layer with Real Redis

A handful of specs should push to real Redis and process with a real Sidekiq processor, to catch serialization, middleware ordering, and unique-job behaviour. Sidekiq's Sidekiq::Testing.disable! plus draining with a test launcher, or simply running Sidekiq::Job::Setter into Redis and processing with Sidekiq::Processor, both work; the simplest robust option is a real sidekiq process started by the CI job.

RSpec.describe "billing pipeline", :sidekiq_disabled, :integration do
  it "charges an invoice end to end" do
    invoice = create(:invoice, status: "open")
    ChargeInvoiceJob.perform_async(invoice.id)          # real Redis push
    wait_for(timeout: 15) { invoice.reload.status == "paid" }   # processed by CI's sidekiq
  end
end
# CI step for the integration layer
- run: bundle exec sidekiq -C config/sidekiq.yml -e test &
- run: bundle exec rspec --tag integration

Make sure the test database is committed (not wrapped in a per-example transaction) for these specs, or the Sidekiq process cannot see records the spec created.

Verification

bundle exec rspec --tag ~integration --profile 10    # fast layer: well under a second each
bundle exec rspec --tag integration                  # slow layer: seconds each, separate CI job

Then reintroduce each production bug temporarily: pass a model to perform_async (strict args spec fails), break the exhausted hook's argument handling (hook spec fails), remove the idempotency key (gateway spec fails).

Gotchas & Edge Cases

Leftover jobs between examples. Without Sidekiq::Worker.clear_all in a before hook, fake queues carry jobs across examples and assertions on counts become order-dependent.

drain runs jobs enqueued by jobs. MyJob.drain also runs jobs that the drained jobs enqueue for the same class; Sidekiq::Worker.drain_all runs everything. Use them deliberately.

Active Job vs native Sidekiq. Jobs defined with Active Job use ActiveJob::TestHelper (assert_enqueued_with, perform_enqueued_jobs) instead of Sidekiq's arrays. Mixed codebases need both helpers.

Sidekiq Pro/Enterprise features. Batches, unique jobs, and rate limiters have their own test helpers and some behaviour only with real Redis; cover them in the integration layer.

FAQ

Why not keep inline mode for simplicity? Inline mode runs jobs at enqueue time inside the request's transaction, hides serialization, skips retries, and makes it impossible to assert that something was enqueued without running it. Fake mode costs one drain call and removes all four problems.

How do I test scheduled jobs (perform_in)? In fake mode, perform_in records the job with an at timestamp. Assert on it, then use Timecop/travel_to and drain to run it.

How do I stop specs from depending on the order jobs run? Assert on end state rather than on execution order, clear all fake queues before each example, and drain explicitly in the order the production flow implies. If a spec only passes when job A happens to run before job B, the production code has the same hidden dependency — make it explicit with a batch callback or a chained enqueue instead of relying on queue order.

Should specs cover Sidekiq's own retry mechanics? No — test your configuration and hooks, not Sidekiq's scheduler. One integration spec that a failing job lands in the retry set is enough.

Related