transformers
a92e0ad2 - Enable training Llama with model or pipeline parallelism (#22329)

Commit
2 years ago
Enable training Llama with model or pipeline parallelism (#22329) * Llama - Move target tokens to final pipeline device if needed * Update src/transformers/models/llama/modeling_llama.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Update src/transformers/models/llama/modeling_llama.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> --------- Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Author
Parents
Loading