ggerganov/llama.cpp

Pull Requests Commits

batch : require non-coupled batch with sequential split_equal

ggerganov committed 106 days ago

Verified 61795789

batch : optional requirement for sequential sequence ids

ggerganov committed 106 days ago

Verified 5eb1a88d

kv-cache : rework kv_idxs, support seq_cp

ggerganov committed 106 days ago

Verified 66631284

kv-cache : simplify set_rows logic

ggerganov committed 107 days ago

Verified 0bb1da58

graph : support iSWA virtual sequences

ggerganov committed 107 days ago

Verified 165d8220

ggml : extend support for n_seq for soft_max and fattn

ggerganov committed 107 days ago

Verified 1b74b9d7

kv-cache : fix non-FA path with virutal sequences

ggerganov committed 107 days ago

Verified 8c682198

metal : extend ggml_soft_max_ext() to support n_seq dim

ggerganov committed 107 days ago

Verified 7c6487b2

cont : fix build

ggerganov committed 107 days ago

Verified 401c13e3

tools : tmp adjustments (TMP)

ggerganov committed 107 days ago

Verified 13214393

llama : add "virtual sequences"

ggerganov committed 107 days ago

Verified 52b90071

kv-cache : utilize ggml_set_rows broadcast

ggerganov committed 108 days ago

Verified 36f8e20d

cont : support non-continuous slots

ggerganov committed 108 days ago

Verified 332f0735

cont : kv-cells cp/set for non-cont slots

ggerganov committed 108 days ago

Verified 39d0b1e8

cont : migrate to using set of indices instead of slot head

ggerganov committed 108 days ago

Verified f875d6cb

cont : gate the ggml_set_rows usage with env var

ggerganov committed 108 days ago

Verified db2bb378

kv-cache : use ggml_set_rows

ggerganov committed 108 days ago

Verified 79dac3c8

ggml : fix supports_op

rgerganov committed 108 days ago

Verified 1f647b59

ggml : simplify forward_dup_f32

rgerganov committed 108 days ago

Verified eba97574

metal : add ggml_set_rows implementation

ggerganov committed 108 days ago

Verified c0cfc2f7

tests : add ggml_set_rows

ggerganov committed 108 days ago

Verified 828e5d2f

ggml : ggml_set_rows update comment + better index name

ggerganov committed 108 days ago

Verified e73690a6

ggml : support GGML_TYPE_F32 ".from_float" trait

ggerganov committed 108 days ago

Verified e8970972

ggml : ggml_set_rows support quantized dst

ggerganov committed 108 days ago

Verified 630c84a2

ggml : ggml_set_rows support broadcast

ggerganov committed 108 days ago

Verified df71c803

ggml : add ggml_is_contiguous_rows

ggerganov committed 108 days ago

Verified 313a444b

ggml : add repeat impl for i64

ggerganov committed 108 days ago

Verified 695b6b70

use I64 for indices

rgerganov committed 108 days ago

Verified f2cd962f

ggml : add ggml_set_rows

rgerganov committed 108 days ago

Verified c1a581a1

kv-cells : fix tracking of seq_pos (#14339)

ggerganov committed 108 days ago

Verified 7b50d589

Older