transformers
95e96cb3 - [distributed] Refuse context parallelism in Trainer

Commit
18 hours ago
[distributed] Refuse context parallelism in Trainer `Trainer` would give a different batch to the ranks of a cp group, which expect the same one. Use a plain training loop until its dataloader shards the sequences instead.
Author
Committer
Parents
Loading