text-generation-inference
7f54b733 - Test Marlin MoE with `desc_act=true` (#2622)

Commit
1 year ago
Test Marlin MoE with `desc_act=true` (#2622) Update the Mixtral GPTQ test to use a model with `desc_act=true` and `group_size!=-1` to ensure that we are checking activation sorting/non-full K (with tensor parallelism). The `desc_act=false` case is already checked by the Mixtral AWQ test.
Author
Parents
Loading