vllm
10f6ab01 - [Perf] Use value-only reduction for native per-token FP8 quantization (#59800)

Commit

13 days ago

[Perf] Use value-only reduction for native per-token FP8 quantization (#59800) Signed-off-by: jackLei0901 <841747318@qq.com> Co-authored-by: Misha Goin <mgoin64@gmail.com>

References

#59800 - [Perf] Use value-only reduction for native per-token FP8 quantization

Author

jackLei0901

Parents

8882b83b

vllm 10f6ab01 - [Perf] Use value-only reduction for native per-token FP8 quantization (#59800)

vllm
10f6ab01 - [Perf] Use value-only reduction for native per-token FP8 quantization (#59800)