ggerganov/llama.cpp

Pull Requests Commits
WIP src1/prec for fp8 in MMVF
ORippler committed 4 days ago
6842be1e
Move FP8 conversion intrinsics into ggml_cuda_cast
ORippler committed 4 days ago
a5f2306e
Explicitly link against cublaslt
ORippler committed 4 days ago
b280371b
Use F32 activations in fallback mmvf path
ORippler committed 4 days ago
fa6d382a
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.4
ORippler committed 4 days ago
28c10b60
More code-cleanup
ORippler committed 4 days ago
6d7c3f17
Fuse weight_scale multiplication for FP8 also
ORippler committed 4 days ago
609fe489
Simplify by falling back to shared cuBLAS helper
ORippler committed 4 days ago
bcc0df28
More cleanup
ORippler committed 4 days ago
4e810841
Make FP8 copies work on all IHV's
ORippler committed 4 days ago
479eca62
Use get_rows_cuda_float for fp8 kv cache
ORippler committed 4 days ago
dce88571
Use convert_unary_cont_cuda over block dequant in convert.cu
ORippler committed 4 days ago
18dec227
Add commend on cublaslt workspaces
ORippler committed 4 days ago
713ef18a
Resolve TODO
ORippler committed 4 days ago
32c67a7f
Use intrinsic over explicit manual conversions
ORippler committed 4 days ago
90f917a8
Allow Hopper and other BW SKUs for FP8
ORippler committed 4 days ago
38af9e02
Avoid races in fp8_fallback
ORippler committed 4 days ago
4591fd3c
Add todo about accumulator/precision honoring
ORippler committed 4 days ago
5165f42b
Restrict (fused) MMVF to 2*ts alignment, and cublas FP8 fo 16-byte
ORippler committed 4 days ago
3ce429f2
Gate mmvf_f8x2_e4m3_to_bf162 on FP8 availability, i.e. CTK >= 11.8
ORippler committed 4 days ago
47267e0c
Fix FP8 fallback in cuBlas
ORippler committed 4 days ago
1eaf2e51
Fix odd-rows for convert/getrows, possible with FP8
ORippler committed 4 days ago
4a513f3d
Prepare Q_reg for FP8 FlashAttention in fattn-vec.cuh
ORippler committed 4 days ago
da5cc4ac
Add supported for strided copies to/from FP8 in CUDA
ORippler committed 4 days ago
1570cfb8
Remove trailing whitespace
ORippler committed 4 days ago
61b7f9cd
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.2
ORippler committed 4 days ago
cb7a6faf
Optimize GEMV kernel for FP8
ORippler committed 4 days ago
5d66d30b
Do W8A16 for FP8 in GEMV case
ORippler committed 4 days ago
f22a92b7
Use fp8 intrinsics
ORippler committed 4 days ago
8cbc56ac
Initial CUDA WIP version
ORippler committed 4 days ago
c188d0c0
Terms Of Use Privacy Policy Refund Policy Impressum