transformers
26bb358e - [Trainer] Add ddp_static_graph option (#45519)

Commit
116 days ago
[Trainer] Add ddp_static_graph option (#45519) * [Trainer] Add ddp_static_graph option Expose PyTorch DDP's `static_graph` flag as a new `ddp_static_graph: Optional[bool]` field on `TrainingArguments`, forwarded through `Trainer._build_accelerator_args` into Accelerate's `DistributedDataParallelKwargs` (which already supports it). Completes the set of DDP flags partially exposed today (`ddp_find_unused_parameters`, `ddp_bucket_cap_mb`, `ddp_broadcast_buffers`). Defaults to `None`; when unset, the kwarg is never added to `ddp_kwargs`, so Accelerate's own default (`False`) applies — strictly additive, no existing behavior changes. See issue #45518 for full motivation: users with frozen trainable submodules (e.g. the LLM-frozen head-tuning pattern) today either hit `Expected to have finished reduction in the prior iteration...` or must pay per-iteration `find_unused_parameters=True` traversal cost. `static_graph=True` is the performance-optimal third option that Accelerate/PyTorch already support but that Trainer couldn't expose. Tests: positive (True, False) plus regression guard (None must not leak the kwarg). Fixes #45518 * Address review: shorten ddp_static_graph docs
Author
Parents
Loading