Fix wait_for_everyone() crashing on MULTI_CPU when the machine has an accelerator (#4354)
* Fix wait_for_everyone() crashing on MULTI_CPU when the machine has an accelerator
`PartialState.wait_for_everyone()` passes `device_ids=[local_process_index]` to
`torch.distributed.barrier()` for every distributed type, including MULTI_CPU.
When `device_ids` is given, torch runs the barrier on the machine's accelerator
(`torch._C._get_accelerator()`) instead of on the gloo/CPU process group. On a
machine with a non-CUDA accelerator (e.g. Apple Silicon / MPS) this raises
`NotImplementedError: The distributed operator 'c10d::barrier' is not currently
implemented for the MPS device`, so any multi-process CPU run that calls
`wait_for_everyone()` / `main_process_first()` crashes.
Only pass `device_ids` when the distributed type is not MULTI_CPU; on CPU let
torch pick the device of the process group, as it did before #3744.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Use elif for MULTI_CPU barrier and drop mocked tests per review
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>