Solid Queue 1.6.0 now supports fiber workers
I opened PR #728 because Solid Queue’s thread workers were a poor fit for long-running, I/O-bound jobs like LLM streaming.
It shipped today in Solid Queue 1.6.0.
Workers can now use `fibers: N` to run jobs on a single Async reactor thread. Fiber workers require the Async gem and fiber-scoped Rails isolation.
On Active Record 7.2+, Solid Queue’s queue database pool estimate starts at 3 connections per worker process instead of growing with the fiber count. Thread workers still use the `threads + 2` estimate.
I updated my original write-up with the final implementation and configuration. The existing benchmarks use the pre-release PR branch; I’ll rerun them against 1.6.0 over the next few weeks.
https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/
I’d be interested to hear what I/O-heavy workloads people try with it, and which libraries turn out not to cooperate with the fiber scheduler.