vllm
[ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing
#12501
Merged
Merged
[ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing #12501
simon-mo
merged
9 commits into
vllm-project:main
from
EmbeddedLLM:ptpc-fp8-rocm-2
Add ptpc-fp8 quantization
8c615443
Enable torch._scaled_mm rowwise gemm fp8
d86df2d2
Update PyTorch version in Dockerfile.rocm_base; Update AMD GPU instal…
4d578810
add ptpc fp8 unittests
ef98cef6
mgoin
commented
on 2025-01-28
fix test_fp8.py::test_kv_cache_model_load_and_run; remove unnecessary…
0f309c2f
tjtanaa
changed the title
Merge remote-tracking branch 'origin/main' into ptpc-fp8-rocm-2
004dadbe
format lint code
73d7bd10
mgoin
approved these changes
on 2025-02-07
Merge remote-tracking branch 'origin/main' into ptpc-fp8-rocm-2
12f42de8
introduce USE_ROWWISE_TORCH_SCALED_MM
881ce38d
disabled auto-merge 316 days ago
Manually disabled by user
Manually disabled by user
Reviewers
Assignees
No one assigned
Labels
documentation
rocm
ready
ci/build
Milestone
No milestone
Login to write a write a comment.
Login via GitHub