transformers
95e96cb3
- [distributed] Refuse context parallelism in Trainer
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
18 hours ago
[distributed] Refuse context parallelism in Trainer `Trainer` would give a different batch to the ranks of a cp group, which expect the same one. Use a plain training loop until its dataloader shards the sequences instead.
Author
3outeille
Committer
3outeille
Parents
8051af49
Loading