CUDA: Add OCP FP8 support #28413
ORippler
wants to merge
28 commits into
osimons/ocp_fp8_kv
from
osimons/ocp_fp8_cuda
0cc4m
commented
on 2026-09-18
Initial CUDA WIP version
9fb51175
Use fp8 intrinsics
ce8176f5
Do W8A16 for FP8 in GEMV case
9519047d
Optimize GEMV kernel for FP8
ce0cbd9f
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.2
130111ee
Remove trailing whitespace
f0042e8b
Add supported for strided copies to/from FP8 in CUDA
ac3fa177
Prepare Q_reg for FP8 FlashAttention in fattn-vec.cuh
11697527
Fix odd-rows for convert/getrows, possible with FP8
69cb9753
Fix FP8 fallback in cuBlas
348de606
Gate mmvf_f8x2_e4m3_to_bf162 on FP8 availability, i.e. CTK >= 11.8
2ec5eaf9
Restrict (fused) MMVF to 2*ts alignment, and cublas FP8 fo 16-byte
869118c8
Add todo about accumulator/precision honoring
4181ffee
Avoid races in fp8_fallback
38c119e3
Allow Hopper and other BW SKUs for FP8
462030ac
Use intrinsic over explicit manual conversions
448c9545
Resolve TODO
9a210cc3
Add commend on cublaslt workspaces
b09dbfc3
Use convert_unary_cont_cuda over block dequant in convert.cu
d6633514
Use get_rows_cuda_float for fp8 kv cache
22e83368
Make FP8 copies work on all IHV's
5ac8bf07
More cleanup
ced2aa78
Simplify by falling back to shared cuBLAS helper
e282a87e
Fuse weight_scale multiplication for FP8 also
3f7cdbfe
More code-cleanup
1a608da4
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.4
c6558f2d
Use F32 activations in fallback mmvf path
3760c495
Explicitly link against cublaslt
d35a3ede
Login to write a write a comment.
Login via GitHub