Add FP8 KV-cache support #28392
ggerganov
merged
24 commits into
master
from
osimons/ocp_fp8_kv
pwilkin
commented
on 2026-10-13
pwilkin
approved these changes
on 2026-10-23
Base automatically changed from
osimons/ocp_fp8
to
master
12 days ago
am17an
commented
on 2026-11-02
am17an
approved these changes
on 2026-11-06
Add FP8 KV-cache support
2809b375
Add tests for scaled, FP8 KV-cache to test-backend-ops
71f46526
Fix get_*_scale mapping and dequnatize FP8 also before rope shift
8e02eb94
Normalizes cache-scale names independently of projection layout.
6dc7807e
Enable Gemma 4 Assistant with FP8 by storing and fetching scale ptrs
5a377e51
Update comment wording
9950a405
Move E2E FP8 KV cache tests to test-llama-archs
c8361300
Increase support surface, remove seed
f54ef5ca
Fix CI failings for test-kv-cache
d35e68ab
CI: execute kv-cache-scale tests
e9bebbe6
exclude fp8-kv tests from OpenVINO
b57c5a78
Fix linting errors of test_kv_cache_scales
612cc29c
Disable kv-ffload=false + multi-GPU tests for FP8
c17e30c5
Adjust test label for test-kv-cache.cpp
06be8c3f
Fix conflict-resolution mistakes in test-kv-cache.cpp
2313af1d
Move KV tests to test-fp8.cpp
a8ce8c86
Remove tmp file for test-kv-cache-scales.py on Windows as well
8847745f
Fix fp8 kv cache test by population kv_source_impl
b8da77e0
Exclude models that consume KV cache directly
43d6929d
Remove obsolete comment
a66395dd
Allow BF16 scales as well for KV cache
430e514d
Rename *_cache_scale to *_cache_s
d9ac6556
Move `is_type_scaled()` to `llama-kv-cache.cpp`
9ebe7f3d
ggerganov
approved these changes
on 2026-11-10
am17an
approved these changes
on 2026-11-10
Login to write a write a comment.
Login via GitHub