ggml-org/llama.cpp

Pull Requests Commits

0cc4m committed 1 day ago

291a674e

add f8 copy pipelines

0cc4m committed 1 day ago

5bbe39fd

remove supports_op rejection, reject unaligned src, fix get_rows issue

0cc4m committed 1 day ago

20279333

vulkan: add fp8 and scaled matmul support

0cc4m committed 1 day ago

32d193fe

Wire activation scales and disallow broadcasting over K

ORippler committed 1 day ago

09b4eab8

Also remove IQ from needs_scale related tests/behvaior

ORippler committed 1 day ago

7ebce19d

Add separate tests for scaled and unscaled DSV2

ORippler committed 1 day ago

38f78bd9

Disable dedicated kernels also for kleidiai in requant test

ORippler committed 1 day ago

dc19ddd8

Revert "tq quants also need scales"

ORippler committed 1 day ago

c8a3d735

Fix conversion edge-case for FP8 block-scales

ORippler committed 1 day ago

7cbec596

llama: use map for scale tensors instead of manual handling

0cc4m committed 1 day ago

cc04d420

tq quants also need scales

0cc4m committed 1 day ago

2d0c9a04

Enforce scale side-cars for scaled data-types during conversion

ORippler committed 1 day ago

26382ae8

Fix pre-existing view-handling failure in meta backend

ORippler committed 1 day ago

879a3844

Add comment explaining intended use of activation scales

ORippler committed 1 day ago

a44e082b

Transform companion FP8 scales

ORippler committed 1 day ago

a36e7e77

Use ggml_need_scale in code-base

ORippler committed 1 day ago

b0d8bf8a

Make needs_scale=true for FP8

ORippler committed 1 day ago

caa289d8

Make scale tensors required for ggml_type with needs_scale=true

ORippler committed 1 day ago

088f9a5a

Make *scale tensors use existing GGML_OP_MUL buffer probe

ORippler committed 1 day ago

e9b12ccf

Handle strided views on scales

ORippler committed 1 day ago

83112f58

Also apply scales in CPU backend 's tiled mul_mat

ORippler committed 1 day ago

bf20c193

Temp workaround for moe-cache with scaled data-types

ORippler committed 1 day ago

f142cfff

Don't treat single expert as per-channel-per-expert

ORippler committed 1 day ago

13ec5523

handle bitnet scales too, disable in backends

0cc4m committed 1 day ago

45564200

add optional activation scales to mul_mat and mul_mat_id tensors

0cc4m committed 1 day ago

e773d226

add meta backend scale split

0cc4m committed 1 day ago

336051a3

ggml: add tensor scaling (per-tensor, per-channel, per-expert, per-expert-per-channel) support

0cc4m committed 1 day ago

e13a94a5

WIP src1/prec for fp8 in MMVF

ORippler committed 1 day ago

92df19a0

Move FP8 conversion intrinsics into ggml_cuda_cast

ORippler committed 1 day ago

e2aee3bf

Older