DeepSpeed
beed962c - [Bug fix] Add rope_theta for llama config (#4480)

Commit
1 year ago
[Bug fix] Add rope_theta for llama config (#4480) * Add rope_theta for llama config * Add rope_theta to bias_add_transform_0213 * Fix CI problems * Add rope_theta to linear layer --------- Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com> Co-authored-by: Lev Kurilenko <113481193+lekurile@users.noreply.github.com>
Author
Parents
Loading