vllm
[Model] Enable FP8 QKV in MoE and refine kernel tuning script
#5039

Merged

[Model] Enable FP8 QKV in MoE and refine kernel tuning script #5039

LiuXiaoxuanPKU merged 6 commits into vllm-project:main from comaniac:moe-lienar

wip

487c724d

8c06d5f2

script

b7cf8968

add 8x22b tp=8

a1c886f1

fix

0af621db

comaniac changed the title ~~[Model] Enable FP8 QKV in MoE~~ [Model] Enable FP8 QKV in MoE and refine kernel tuning script 1 year ago

finish tuning

81399b65

robertgshaw2-redhat approved these changes on 2024-05-31

LiuXiaoxuanPKU merged e9899fb7 into main 1 year ago

comaniac deleted the moe-lienar branch 1 year ago

Reviewers

robertgshaw2-redhat

Assignees

No one assigned

Labels

None yet

Milestone

No milestone