ggerganov/llama.cpp

Pull Requests Commits
vulkan: add fp8 and scaled matmul support
0cc4m committed 16 hours ago
329c8bf0
add optional activation scales to mul_mat and mul_mat_id tensors
0cc4m committed 16 hours ago
22d59e91
add meta backend scale split
0cc4m committed 16 hours ago
cb41adea
ggml: add tensor scaling (per-tensor, per-channel, per-expert, per-expert-per-channel) support
0cc4m committed 16 hours ago
8060c074
Add commend on cublaslt workspaces
ORippler committed 16 hours ago
fd3d4118
Resolve TODO
ORippler committed 16 hours ago
56d06e53
Use intrinsic over explicit manual conversions
ORippler committed 16 hours ago
93e7c560
Allow Hopper and other BW SKUs for FP8
ORippler committed 16 hours ago
d1b35b31
Avoid races in fp8_fallback
ORippler committed 16 hours ago
8a0c9bd1
Add todo about accumulator/precision honoring
ORippler committed 16 hours ago
ba3383a5
Restrict (fused) MMVF to 2*ts alignment, and cublas FP8 fo 16-byte
ORippler committed 16 hours ago
18eb236a
Gate mmvf_f8x2_e4m3_to_bf162 on FP8 availability, i.e. CTK >= 11.8
ORippler committed 16 hours ago
2217433b
Fix FP8 fallback in cuBlas
ORippler committed 16 hours ago
47ca4851
Fix odd-rows for convert/getrows, possible with FP8
ORippler committed 16 hours ago
e5f3207e
Prepare Q_reg for FP8 FlashAttention in fattn-vec.cuh
ORippler committed 16 hours ago
a1288cfe
Add supported for strided copies to/from FP8 in CUDA
ORippler committed 16 hours ago
443c2528
Remove trailing whitespace
ORippler committed 16 hours ago
e37bbd6c
Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.2
ORippler committed 16 hours ago
53ccb2e2
Optimize GEMV kernel for FP8
ORippler committed 16 hours ago
253dc250
Do W8A16 for FP8 in GEMV case
ORippler committed 16 hours ago
55a612e6
Use fp8 intrinsics
ORippler committed 16 hours ago
d313ac91
Initial CUDA WIP version
ORippler committed 16 hours ago
4fa4293f
Fix CI failings for test-kv-cache
ORippler committed 16 hours ago
49a6dfb6
make scaled kv generic
ORippler committed 16 hours ago
706ead5f
Increase support surface, remove seed
ORippler committed 16 hours ago
252e360b
Move E2E FP8 KV cache tests to test-llama-archs
ORippler committed 16 hours ago
88fb22d5
Update comment wording
ORippler committed 16 hours ago
b8a4359e
Enable Gemma 4 Assistant with FP8 by storing and fetching scale ptrs
ORippler committed 16 hours ago
8e3c5ee8
Normalizes cache-scale names independently of projection layout.
ORippler committed 16 hours ago
f63a8783
Fix get_*_scale mapping and dequnatize FP8 also before rope shift
ORippler committed 16 hours ago
130dc026
Terms Of Use Privacy Policy Refund Policy Impressum