vllm
[ Misc ] `fp8-marlin` channelwise via `compressed-tensors`
#6524
Merged
Overview
Commits
12
Changes
View On GitHub

[ Misc ] `fp8-marlin` channelwise via `compressed-tensors` #6524

simon-mo merged 12 commits into main from ct-fp8-marlin
robertgshaw2-redhat
first implementation
6ad34e35
github-actions
format
ca80c4ab
robertgshaw2-redhat
robertgshaw2-redhat commented on 2024-07-17
robertgshaw2-redhat
robertgshaw2-redhat commented on 2024-07-17
add file :)
6b1a941e
renamed
eecfd129
updated
7716aeef
mgoin
mgoin commented on 2024-07-17
mgoin mgoin changed the title [ Misc ] `fp8-marlin `channelwise` via `compressed-tensors` [ Misc ] `fp8-marlin` channelwise via `compressed-tensors` 2 years ago
mgoin Merge branch 'main' into ct-fp8-marlin
29804d30
mgoin Format
4787555e
mgoin Finish implementation
1a56c76a
mgoin mgoin added ready
mgoin Add lm-eval test
b279c18a
mgoin Fix model name
2125900e
mgoin Add smoke test for fp8 kv cache
05a23982
mgoin Revert "Add smoke test for fp8 kv cache"
b5d6381d
mgoin
mgoin approved these changes on 2024-07-25
mgoin mgoin enabled auto-merge (squash) 2 years ago
disabled auto-merge 2 years ago
Manually disabled by user
simon-mo simon-mo merged 889da130 into main 2 years ago
RonanKMcGovern
simon-mo simon-mo deleted the ct-fp8-marlin branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
ready
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum