transformers
caed4a7a - Trainer reads the load-time DistributedConfig instead of mirrored model attributes

Commit
26 days ago
Trainer reads the load-time DistributedConfig instead of mirrored model attributes The model carried `_fsdp_size` and `_expert_parallel_dispatch`, plus a public `fsdp_size` property, whose only reader was the Trainer, while the config they were copied from was already stored on the model. Read it through one helper that looks through PEFT wrappers, and drop the mirrors before the property becomes public API.
Author
Parents
Loading