text-generation-inference
64142489 - Add support for fused MoE Marlin for AWQ (#2616)

Commit
1 year ago
Add support for fused MoE Marlin for AWQ (#2616) * Add support for fused MoE Marlin for AWQ This uses the updated MoE Marlin kernels from vLLM. * Add integration test for AWQ MoE
Author
Parents
Loading