ggerganov/llama.cpp

Pull Requests Commits

add optional activation scales to mul_mat and mul_mat_id tensors

0cc4m committed 5 days ago

b8a4e2cb

add meta backend scale split

0cc4m committed 5 days ago

ba784086

ggml: add tensor scaling (per-tensor, per-channel, per-expert, per-expert-per-channel) support

0cc4m committed 5 days ago

4589c2fc

Avoid races in fp8_fallback

ORippler committed 5 days ago

021df05a

Add todo about accumulator/precision honoring

ORippler committed 5 days ago

fd867ef5

Restrict (fused) MMVF to 2*ts alignment, and cublas FP8 fo 16-byte

ORippler committed 5 days ago

0f0f57d6

Gate mmvf_f8x2_e4m3_to_bf162 on FP8 availability, i.e. CTK >= 11.8

ORippler committed 6 days ago

26fd61b6

Fix FP8 fallback in cuBlas

ORippler committed 6 days ago

0a297ed8

Fix odd-rows for convert/getrows, possible with FP8

ORippler committed 6 days ago

f2ec3f1c

Prepare Q_reg for FP8 FlashAttention in fattn-vec.cuh

ORippler committed 6 days ago

7c53b968

Add supported for strided copies to/from FP8 in CUDA

ORippler committed 6 days ago

98d0007c

Remove trailing whitespace

ORippler committed 6 days ago

c18fda7c

Guard __nv_cvt_fp8x2_to_bf162raw on CTK 13.2

ORippler committed 6 days ago

f2d29850

Optimize GEMV kernel for FP8

ORippler committed 6 days ago

e1eadb6a

Do W8A16 for FP8 in GEMV case

ORippler committed 6 days ago

642a9655

Use fp8 intrinsics

ORippler committed 6 days ago

82c64364

Initial CUDA WIP version

ORippler committed 6 days ago

a81b6782

Fix get_*_scale mapping and dequnatize FP8 also before rope shift

ORippler committed 6 days ago

4602e642

Add tests for scaled, FP8 KV-cache to test-backend-ops

ORippler committed 6 days ago

9c61732b

Add FP8 KV-cache support

ORippler committed 6 days ago

fc6335f1

Add supported for strided copies to/from FP8 in CPU

ORippler committed 6 days ago

574eb984

More FP8 conversion fixes

ORippler committed 6 days ago

aa742bb6

Emit warnings when not fusing FP8 during conversion

ORippler committed 6 days ago

471b26dd

Avoid fusing FP8 for --fuse-gate-up-exps as we may have different ws

ORippler committed 6 days ago

e703584f

Fix conversion of fused qkvz & FP8 for Qwen3-Next

ORippler committed 6 days ago

b2860f80

Disallow QKV-fusion for FP8 temporarily

ORippler committed 6 days ago

5f5aedd5

Fix and test gguf-py behavior for FP8_E4M3

ORippler committed 6 days ago

7ba386e0

Allow tensor-strategy for FP8-only compressed-tensor ckpts

ORippler committed 6 days ago

1c4cde0e

Allow remaining FP8 tensors to be dequantized during conversion

ORippler committed 6 days ago

19361354

Fix conversion of https://huggingface.co/RedHatAI/Qwen3.8-27B-NVFP4

ORippler committed 6 days ago

9e12cdad

Older