vllm
[Bugfix] Fix `kv_cache_dtype=fp8` without scales for FP8 checkpoints
#6761
Merged
Merged
[Bugfix] Fix `kv_cache_dtype=fp8` without scales for FP8 checkpoints #6761
simon-mo
merged
3 commits into
main
from
fix-unloaded-fp8-kv-scales
Fix fp8 kv cache without scales
452bfeb5
mgoin
changed the title
Add smoke test
fe9648b0
Format
eda606ed
robertgshaw2-redhat
approved these changes
on 2024-07-25
comaniac
approved these changes
on 2024-07-25
disabled auto-merge 1 year ago
Manually disabled by user
Manually disabled by user
Reviewers
Assignees
No one assigned
Labels
ready
Milestone
No milestone
Login to write a write a comment.
Login via GitHub