DeepSpeed
ee7db483 - autoTP for Qwen (#4902)

Commit
1 year ago
autoTP for Qwen (#4902) Enabled autoTP for the Qwen model, added some module matching, and adjusted TP-related variables. Verification was conducted on Qwen-1_8B and Qwen-72B-chat. --------- Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Author
Parents
Loading