Add fiber worker execution mode

[crmne]

Summary

Hi @rosa, I finally had some time to work on this after our earlier conversation about async worker execution mode.

This PR is a first implementation of async worker execution mode for Solid Queue. I'm also running benchmarks in parallel and can follow up with numbers once I have stable results.

Workers can now be configured with execution_mode: :async (or :fiber), which runs claimed jobs as fibers on a single async reactor thread and bounds concurrency with capacity / fibers instead of a thread pool.

This is separate from supervisor async mode. Supervisor mode still controls whether managed processes run in forks or threads; this change adds an async execution backend for workers themselves.

What Changed

• added an execution pool abstraction for workers
• introduced SolidQueue::ExecutionPools::AsyncPool
• moved existing thread-pool behavior into SolidQueue::ExecutionPools::ThreadPool
• added execution_mode: :async and :fiber as a configuration alias
• added capacity / fibers as the clearer async-worker concurrency options
• updated workers to build execution pools after fork/boot rather than during initialize
• updated worker metadata to report execution mode, capacity, and inflight work across both backends
• wake workers when capacity becomes available again so async execution integrates cleanly with the poll loop
• treat async pool cancellation as fatal so reactor failures are surfaced instead of being silently ignored

Configuration / Validation

This PR also tightens async worker validation:

• async workers require fiber-scoped isolated execution state
• async workers fail early if the async gem is not available
• async workers reject threads: and require capacity or fibers instead

Database pool guidance is now Rails-version-aware:

• Rails 7.1 keeps the conservative async DB-pool validation
• Rails 7.2+ uses the lower async-worker bound, since ordinary Active Record query paths can release connections between async waits

The README now documents the version-specific DB-pool guidance and calls out sticky Active Record APIs that can still pin connections.

Why

The goal is to support cooperative, mostly I/O-bound job execution with lower thread overhead and a clearer concurrency model, while keeping the feature explicit and safe to configure.

Tests

Added and updated coverage for:

• async pool behavior
• async worker execution
• worker metadata and wake-up behavior
• async boot/shutdown/error handling
• configuration validation
• DB-pool validation and docs around Rails version differences

[seanharmer]

I've pushed my initial implementation for this to #729 for comparison. Feel free to take anything you deem useful. :)

[crmne]

Hi @rosa, I've published a benchmark harness that answers three questions:

1. Within Solid Queue: how much do I/O-heavy workloads benefit from async execution compared to thread mode?
2. DB pool ceiling: at what concurrency does thread mode exhaust the database connection pool, and how much further does async go?
3. Across backends: how does Solid Queue compare to Async::Job + Redis when both run through ActiveJob?

https://github.com/crmne/solid_queue_bench

Solid Queue: async vs thread

async wins the majority of headline tests. Strongest gains by workload:

Workload	Best delta	Win rate
sleep	+27.2%	6/9
async_http	+26.0%	5/9
cpu	+5.1%	7/9
ruby_llm_stream	+20.2%	9/9

The cleanest result is ruby_llm_stream (real RubyLLM streaming + Turbo broadcasts) -- async wins every test. cpu staying roughly neutral is the expected control, which makes the I/O gains credible.

DB pool ceiling (stress suite)

The headline suite caps total concurrency to keep comparisons fair. The stress suite removes that cap. thread mode hit the DB pool wall after the baseline cap=25, proc=2 test and failed every higher-concurrency cell. async completed all 10/10 planned tests per workload -- it multiplexes fibers over a much smaller connection pool, so it survives where threads cannot.

Async::Job vs Solid Queue

Async::Job + Redis is faster across all shared tests (+7% to +213%), but that's a different backend entirely -- a throughput ceiling reference, not a same-backend comparison.

Bottom line

The main async win is not raw throughput. It's good I/O performance without thread-sized DB pools and the connection pressure that comes with them. Under stress, that difference becomes binary: threads fail, fibers keep going.

[crmne]

I'm running this code in production at Chat with Work right now. Switched from Async::Job largely because of operational advantages, especially visibility from Mission Control Jobs when debugging failures.

[bradgessler]

Also running this in production and can confirm it works very well.

[trevorturk]

I wanted to highlight something to make sure it's more visible to others... it looks like this setup would allow for separately configured worker pools, so one could imagine putting "mostly waiting on LLMs or slow APIs" jobs into an async pool, and the "regular" work could be left on a thread pool. For example, in the PR changes here, you can see a config example like so:

- queues: "llm*"
  execution_mode: async
- queues: [ real_time, background ]
  execution_mode: thread

[crmne]

Just a heads-up: I'm planning to write a blog post about this PR! :)

[rosa]

Hey @crmne, sorry for the delay here. I'm swamped with other work and haven't had the time to look into this at all. It's a big diff so I need time. I also need to figure out the naming around this, because we already have an async execution mode unrelated to this, so the naming needs to be adjusted.

I'm really thankful for the contribution, but I'm afraid I won't have time to review this in the near term.

Of course, you're welcome to write whatever blog post you want 😊

[crmne]

Hey @rosa, didn't mean to add any pressure - I totally get being swamped with other committments. Does execution_mode: :fiber work better for you? I can do the change pretty quickly and before the blog post goes out!

[rosa]

Oh, no, no worries! I didn't think you were trying to add pressure, it's just that I felt really bad about seemingly ignoring this 😅

Yes! :fiber is what I had in mind! But also execution_mode... not completely sure about that... 🤔 Thinking of another possible names that reflect what this is doing.

[seanharmer]

How about:

- queues: llm
      threads: 2
      concurrency_model: fiber
      fibers: 50
      polling_interval: 0.1

[crmne]

How about:

- queues: llm
      threads: 2
      concurrency_model: fiber
      fibers: 50
      polling_interval: 0.1

I like that, but mixing fibers and threads wouldn't make sense.

[seanharmer]

How about:

- queues: llm
      threads: 2
      concurrency_model: fiber
      fibers: 50
      polling_interval: 0.1

I like that, but mixing fibers and threads wouldn't make sense.

Right, sorry, that was a copy/paste from my early implementation. If this one doesn't expose threads, just ignore that line. :)

[rosa]

Maybe it could just be threads: N or fibers: N, make it so that only one is allowed, and that determines whether you use threads or fibers?

[crmne]

Maybe it could just be threads: N or fibers: N, make it so that only one is allowed, and that determines whether you use threads or fibers?

I like it. Will implement that during the weekend.

[crmne]

I updated it accordingly, avoiding the need of a separate execution_mode config option.

I also renamed the worker implementation/docs away from "async", using "fiber" and "worker concurrency model" to avoid confusion.

[crmne]

https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/ in case people are interested

[gobijan]

Not sure about monkey patching here. Maybe just raise the required ruby version?

[gobijan]

This creates a busy-poll loop at 100hz per worker?
Would it be better to raise the min required async gem version to async >= 2.29 (thread safety improvements) so you can replace the poll loop with Async::Notification or Async::Queue?

[gobijan]

Raise async gem version to avoid busy-poll loop?

[ollym]

Similarly would be cool to see this work be introduced into Rails core so we could set config.active_record.async_query_executor = :global_fiber_pool for async queries. Good work otherwise, looking forward to seeing this and batch support merged.

[rsamoilov]

In my opinion, if the configuration is called fibers, it needs to be backend-agnostic. Which means checking Fiber.scheduler instead of defined?(Async), using Fiber.schedule {} instead of Async {}, and SizedQueue instead of Async::Semaphore - thus completely relying on the standard library and being implementation-agnostic.

Good design is the one that survives change, but this executor can't survive change by definition, as it locks the implementation to a specific backend. Async is the de facto standard, but there are other implementations, and hopefully there will be many more. IMO using Async belongs to the application layer, and tying the executor to the specific implementation of the scheduler would lock users out from using other (perfectly functional) implementations in the future. To me, this coupling doesn't make sense in the framework code, especially when the standard library exposes clean hooks that allow the executor to remain implementation-agnostic.

Essentially, I'd suggest either using a backend-agnostic implementation or updating the fibers configuration name to specifically point out that it's powered by Async.

[rosa]

That's a good point, @rsamoilov 🤔

[crmne]

Thanks @rsamoilov. I agree that a backend-agnostic fiber executor could be worth exploring, but I don't think that should block this PR.

To me, fibers: N describes the concurrency model exposed to Solid Queue users, just like threads: N does. It says jobs will run concurrently using fibers rather than only threads. I don't think it necessarily means the first implementation must also expose a scheduler abstraction.

This PR implements the concrete version we have today: a fiber worker backed by Async. That implementation is done, tested, documented, benchmarked, and already running in production. It also handles the Solid Queue-specific parts that matter here: bounded in-flight jobs, lifecycle, shutdown, error propagation, worker wakeups, and the DB-pool guidance around fiber execution.

I agree that making the Async-specific pieces backend-agnostic may be worth exploring. But I don't think that's a mechanical naming fix; it's a separate design exercise. It should be proven against at least one real alternative scheduler, with the same lifecycle/shutdown/error behavior and benchmarks showing what users gain from the extra abstraction.

So my preference is to land this PR as the initial fiber worker implementation, keeping the threads: N / fibers: N user-facing API that was discussed above. If backend-agnostic scheduling is important, I think the right next step is a follow-up PR from someone who wants to drive that work, with a concrete implementation and tradeoffs.

I'm happy to make the docs explicit that the current fiber worker is powered by Async, but I don't think we should hold up the concrete implementation people can use today for an abstraction that hasn't been demonstrated yet.

[rsamoilov]

fibers: N describes the concurrency model exposed to Solid Queue users, just like threads: N does. It says jobs will run concurrently using fibers

It also introduces a level of indirection, as the current implementation doesn't really use fibers. It uses Async, which uses fibers under the hood.

I work with fibers a lot, so I thought I'd share my perspective - for something that can end up in a Rails default job backend, that's a meaningful architectural choice.

[crmne]

the current implementation doesn't really use fibers.

That's not accurate. Async is built on Ruby fibers; this implementation provides fiber-based concurrency via Async.

I agree we should be explicit about that, so I'm happy to document the current worker as Async-backed. But I don't think that turns this PR into a generic scheduler abstraction. That's a separate design with real tradeoffs: lifecycle, shutdown, error propagation, DB pool behavior, and benchmarks against a concrete alternative.

So I think the decision here is simple: land the Async-backed fiber worker now, or require a broader adapter layer before any fiber worker can ship. My preference is strongly the former. If someone wants scheduler pluggability, that should be a follow-up PR with code, tradeoffs, and numbers, not a blocker for this one.

[will-s-stone]

Were there any updates on the ETA for this PR? It would be really helpful to have this functionality!

[rosa]

Hey @will-s-stone, no ETA. I'm still swamped with other work and not sure when I'll have time to work on this.

[abuisman]

This seems like exactly what we need. Thanks @crmne for the work!

[rosa]

Hey! I'm finally catching up with PRs and issues this week, so I'll get to this one in the coming days. Thanks @crmne and everyone else for your patience 🙏🏻

[rosa]

I think I'm going to stop supporting Ruby 3.1. It's time, it's been EOL since 2025-03-26. In that way, we can simplify this I think.

[rosa]

@rsamoilov, @crmne, I've thought about the explicit Async vs backend-agnostic question, and I agree with @crmne here. Async is already tested and benchmarked for this PR, so I think we should go with it. The only exposed configuration, fibers: N, doesn't prevent going for that backend-agnostic approach in the future if needed, so that people can use other backends when they're available. If/when this is needed, we can work on that in a separate, future PR. If this PR made that impossible or a breaking change or something like that, it'd be different but it doesn't seem to be the case.

Thanks a lot for your perspective, @rsamoilov, really appreciate it! I don't work with fibers, so your insight has been very useful for me.

[rosa]

I think this belongs in the configuration validation.

[rosa]

I think we can simplify all this, and just rely on ClaimedExecution records, instead of having to keep this denormalized metadata up to date. I don't think it's worth it. I'll push some changes to this.

[rosa]

@crmne, I prepared a few changes on top of your work. It was getting too big to just push it here as I did with a few smaller changes before, so I've opened crmne#1 for you to check. Core functionality doesn't change except for some defensive code and nicer errors for private APIs, which I think we can do without. The largest change is removing the dynamic metadata from workers and the need to keep the inflight capacity in sync.
The rest is mostly refactoring and reorganizing code. Feel free to review that one and let me know if you have objections over on that PR. I'll tidy up the history once everything is over here, before merging.