text-generation-inference
1c84a30f
- MoE Marlin: support `desc_act` for `groupsize != -1` (#2590)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
1 year ago
MoE Marlin: support `desc_act` for `groupsize != -1` (#2590) This change uses the updated Marlin MoE kernel from vLLM to support MoE with activation sorting and groups.
References
#2590 - MoE Marlin: support `desc_act` for `groupsize != -1`
Author
danieldk
Parents
d1f257ac
Loading