text-generation-inference
14980df2 - Support AWQ quantization with bias (#2117)

Commit
1 year ago
Support AWQ quantization with bias (#2117) When the AWQ quantizer was used with a layer that uses a bias, the bias tensor was not correctly passed/used. Instead, the value `true`/`1.0` was added to the linear transformation. Correctly pass through the bias when it is not `None`. Fixes #2106.
Author
Parents
Loading