ggerganov/llama.cpp

Pull Requests Commits
WIP src1/prec for fp8 in MMVF
ORippler committed 1 day ago
92df19a0
Move FP8 conversion intrinsics into ggml_cuda_cast
ORippler committed 1 day ago
e2aee3bf
Explicitly link against cublaslt
ORippler committed 1 day ago
ce7cf28c
Use F32 activations in fallback mmvf path
ORippler committed 1 day ago
d5183472
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.4
ORippler committed 1 day ago
e7a654e6
More code-cleanup
ORippler committed 1 day ago
c3a0e727
Fuse weight_scale multiplication for FP8 also
ORippler committed 1 day ago
13225fc2
Simplify by falling back to shared cuBLAS helper
ORippler committed 1 day ago
31bf99ee
More cleanup
ORippler committed 1 day ago
3fb41158
Make FP8 copies work on all IHV's
ORippler committed 1 day ago
721ecf8f
Use get_rows_cuda_float for fp8 kv cache
ORippler committed 1 day ago
7a69e3af
Use convert_unary_cont_cuda over block dequant in convert.cu
ORippler committed 1 day ago
733735f2
Add commend on cublaslt workspaces
ORippler committed 1 day ago
bf93911a
Resolve TODO
ORippler committed 1 day ago
d16ac145
Use intrinsic over explicit manual conversions
ORippler committed 1 day ago
605c6ee0
Allow Hopper and other BW SKUs for FP8
ORippler committed 1 day ago
3f879b23
Avoid races in fp8_fallback
ORippler committed 1 day ago
e8b3c71b
Add todo about accumulator/precision honoring
ORippler committed 1 day ago
c34a5ef3
Restrict (fused) MMVF to 2*ts alignment, and cublas FP8 fo 16-byte
ORippler committed 1 day ago
145166d7
Gate mmvf_f8x2_e4m3_to_bf162 on FP8 availability, i.e. CTK >= 11.8
ORippler committed 1 day ago
59d911db
Fix FP8 fallback in cuBlas
ORippler committed 1 day ago
a3364a5f
Fix odd-rows for convert/getrows, possible with FP8
ORippler committed 1 day ago
47725a8b
Prepare Q_reg for FP8 FlashAttention in fattn-vec.cuh
ORippler committed 1 day ago
ea362b70
Add supported for strided copies to/from FP8 in CUDA
ORippler committed 1 day ago
a20f6be8
Remove trailing whitespace
ORippler committed 1 day ago
bb0db82e
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.2
ORippler committed 1 day ago
4a763a43
Optimize GEMV kernel for FP8
ORippler committed 1 day ago
e907f6ad
Do W8A16 for FP8 in GEMV case
ORippler committed 1 day ago
125a7826
Use fp8 intrinsics
ORippler committed 1 day ago
a34afec4
Initial CUDA WIP version
ORippler committed 1 day ago
2003658f
Terms Of Use Privacy Policy Refund Policy Impressum