accelerate

05bf97ef - Do not import torch.distributed in model_has_dtensor on a non-distributed build (#4250)

Do not import torch.distributed in model_has_dtensor on a non-distributed build (#4250) * fix(dtensor): do not import torch.distributed on a non-distributed build 🧩 Fixes #4249. model_has_dtensor() imports torch.distributed.tensor unconditionally. On a PyTorch build compiled without a distributed backend that import raises rather than returning False, and since v1.15.0 every prepare_model() call reaches it: accelerator.py:1802 device_placement = (... and not model_has_dtensor(model)) utils/other.py:243 from torch.distributed.tensor import DTensor ... torch/distributed/distributed_c10d.py:25 ModuleNotFoundError: No module named 'torch._C._distributed_c10d' AMD's official Windows ROCm wheels are exactly that build: torch.distributed imports, torch.distributed.is_available() is False, and torch._C._distributed_c10d does not exist. So all training on AMD GPUs on Windows dies at trainer start. In 1.14.0 the sole call site (accelerator.py:1878) sat behind `if self.multi_device and not (...)`, so a single-GPU run never reached it. The second call site added in #4181 is unguarded, which is what turned a multi-device-only path into one every run executes. Guarding model_has_dtensor itself rather than restoring the guard on the new call site, because the invariant belongs to the function: DTensor lives under torch.distributed, so a build without a distributed backend cannot hold DTensor parameters at all. That also covers any future call site. is_torch_distributed_available() is the existing idiom for this. It is already imported in this same file and already used as a guard 70 lines further down, and accelerator.py:2958 already computes `is_dtensor_available = torch.distributed.is_available() and ...`, so the "DTensor requires a distributed build" relationship is established. 2 tests added. The first reproduces the reported failure by making the import raise the way it does on those wheels, so it fails against unpatched sources with the bug report's own error: ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package The second pins that the guard does not short-circuit the real check on a normal build. tests/test_utils.py: 48 passed, 4 skipped. ruff check and ruff format clean. * Update src/accelerate/utils/other.py --------- Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>