Skip to content

Investigate: can training run in parallel in Ruby at all? #23

Description

@Selectus2

This issue produces evidence, not code. It closes with findings, not a PR against lib/.

Ruby offers three concurrency primitives and they are not interchangeable for this workload:

  • Ractors give true parallelism, but a C extension is only callable from a Ractor if it opts in via rb_ext_ractor_safe. Major gems — Nokogiri, protobuf — still have open requests to do so. If torch-rb has not, a tensor operation inside Ractor.new raises Ractor::UnsafeError and this path is closed.
  • Threads give real parallelism only for the duration of a C call that releases the GVL. If libtorch operations release it, Ruby threads parallelise training compute despite the GVL. This is the most likely path to a positive result.
  • Fibers give no compute parallelism at any time. Their only plausible role here is overlapping data loading and tokenisation I/O with compute.

The honest workload is also worth naming up front. Data-parallel training of a single model needs gradient synchronisation across workers, which Ractor isolation makes expensive and awkward. The achievable win is N independent training runs in one process — hyperparameter sweeps, per-tenant fine-tunes, the shape that actually shows up behind a Rails app running Sidekiq.

Acceptance criteria

  • Ractor.new { Torch.tensor([1, 2, 3]).sum } — result recorded, exact error text if it raises
  • Whether torch-rb calls rb_ext_ractor_safe, established by reading its source, with a link to the line or a statement that it does not appear
  • Whether libtorch calls release the GVL, established by reading torch-rb for rb_thread_call_without_gvl or equivalent
  • Wall-clock for the same synthetic fine-tune at 1 / 2 / 4 threads, on stated hardware — three runs each, median reported
  • The same measurement for Ractors if they are viable at all
  • Peak RSS at each level. Ractor isolation means one model copy per Ractor; if that is N × model size, it caps the useful N and that is a finding, not a footnote
  • A fiber-based data-loading prototype, or a written statement of why it is not worth measuring
  • Findings written up in this issue in a form readable by someone who was not there

Out of scope

Any change to lib/. Multi-GPU, distributed training, gradient synchronisation across workers.

Notes

Ractors still emit an experimental-feature warning. Say so in the writeup rather than suppressing it — "this is experimental and here is what that cost me" is more useful than a clean transcript.

Run the benchmarks after issue 11 (LR schedule) and issue 12 (seed) land. Numbers taken on a trainer whose scheduler is silently wrong are measuring the bug, and numbers taken without a working seed are not reproducible.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions