Skip to content

Stop DataLoaderShard taking a device mesh it never reads - #4252

Open
vineethsaivs wants to merge 1 commit into
huggingface:mainfrom
vineethsaivs:dataloader-shard-unused-device-mesh
Open

Stop DataLoaderShard taking a device mesh it never reads#4252
vineethsaivs wants to merge 1 commit into
huggingface:mainfrom
vineethsaivs:dataloader-shard-unused-device-mesh

Conversation

@vineethsaivs

Copy link
Copy Markdown
Contributor

What breaks

DataLoaderShard(dataset, torch_device_mesh=mesh) accepts the mesh and ignores it.

Its sibling does not:

class DataLoaderDispatcher(...):
    def __init__(self, ..., torch_device_mesh=None, ...):
        self.torch_device_mesh = torch_device_mesh
        ...
        if self.torch_device_mesh and "tp" in self.torch_device_mesh.mesh_dim_names:
            self.submesh_tp = self.torch_device_mesh["tp"]

DataLoaderShard.__init__ takes the same argument, never stores it, and never reads it. Its docstring documents every other parameter and not this one.

Scope

prepare_data_loader only hands the mesh to the dispatcher, so nothing inside accelerate is affected. The trap is for anyone building a DataLoaderShard directly, which is a public, exported class: they get silence rather than either an effect or an error.

What changed

Drop the parameter. It now falls into **kwargs, where DataLoader rejects it, so an unknown argument fails the same loud way any other one does.

Tests

test_dataloader_shard_does_not_take_a_device_mesh asserts the shard rejects it and that the dispatcher still accepts one.

1 passed in 1.69s

On main:

with self.assertRaises(TypeError):
E  AssertionError: TypeError not raised

ruff 0.13.1, the pin in setup.py, is clean on both files.

Related

Same class of thing as #4251, which removes a split_batches argument Accelerator.__init__ never reads. Separate file and separate call path, so they are separate PRs, but happy to fold them together if you would rather review one.

DataLoaderDispatcher stores torch_device_mesh and builds its tp, dp and fsdp
submeshes from it. DataLoaderShard takes the same argument, documents none of it
in a docstring that covers every other parameter, and never reads it.

prepare_data_loader only hands the mesh to the dispatcher, so nothing in
accelerate is affected. Anyone constructing DataLoaderShard directly with a mesh
gets silence instead of either an effect or an error.

Drop the parameter. It now lands in **kwargs and DataLoader rejects it, which is
the same loud failure any other unknown argument gets.

Test: test_dataloader_shard_does_not_take_a_device_mesh, which also pins that the
dispatcher still accepts one. It fails on main with "TypeError not raised".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant