transformers
29985e67 - Add distributed runtime utils and DistributedMixin (#47352)

Commit
66 days ago
Add distributed runtime utils and DistributedMixin (#47352) * Add distributed runtime utils and DistributedMixin (FSDP orchestration 1/3). Introduce distributed/utils.py and DistributedMixin, defer DistributedConfig validation to load time, and refactor PreTrainedModel plan properties without changing the from_pretrained distributed_config API yet. * addd ep_plan * restore validate module * Wire DistributedConfig through from_pretrained and save_pretrained. Route distributed load/save orchestration through DistributedMixin so TP and FSDP paths share the same entry points. * revert * inline distribute_model * revert * remove saving/loading * leaner mixin * downgrade torch version guarding * remove * linting * post_init() parallel plan move to mixin * revert tp mixin
Author
Parents
Loading