ggerganov/llama.cpp

Pull Requests Commits

cache q values into registers for KQ

0cc4m committed 4 hours ago

ac0a161c

add padding to mask shmem buffer

0cc4m committed 13 hours ago

9544cc68

0cc4m committed 14 hours ago

ac6648e0

add medium rows FA shader Br size

0cc4m committed 1 day ago

0146dff0

optimize masksh use

0cc4m committed 1 day ago

377fc217

Revert "move mask cache from shared memory to registers"

0cc4m committed 1 day ago

8e344af3

move mask cache from shared memory to registers

0cc4m committed 1 day ago

94e4b164

fix amd workgroup size issue

0cc4m committed 1 day ago

36acd86c

use f32 scalar FA if f16 is not supported by device

0cc4m committed 1 day ago

c562553a

use row_split when Br >= 4, change reductions to use shared memory if row_split == 1

0cc4m committed 1 day ago

b7632b27

split rows inside of subgroups for faster synchronization

0cc4m committed 1 day ago

a6f1f8a6

vulkan: allow using fp16 in coopmat1 flash attention shader

0cc4m committed 1 day ago

ab9851f1

vulkan: For coopmat2 FA, use fp16 accumulators for the final result (#19376)

jeffbolznv committed 1 day ago

Verified 1946e46f

vulkan: make FA mask/softcap enables spec constants (#19309)

jeffbolznv committed 1 day ago

Verified f9bd518a

metal : skip loading all-zero mask (#19337)

ggerganov committed 1 day ago

Verified 7fcf1ef4

llama : rename llama-sampling to llama-sampler (#19363)

danbev committed 1 day ago

Verified e696cfc0

cuda : cuda graphs now compare all node params (#19383)

ggerganov committed 1 day ago

Verified 3e216476

metal : adaptive CPU/GPU interleave based on number of nodes (#19369)

ggerganov committed 2 days ago

Verified 22cae832

vulkan: Preprocess FA mask to detect all-neg-inf and all-zero. (#19281)

jeffbolznv committed 2 days ago

Verified 449ec2ab

benches : update models + numbers (#19359)

ggerganov committed 2 days ago

Verified 3795cc1e

docker : fix vulkan build (#19352)

CISC committed 2 days ago

Verified b828e18c

vendor : update BoringSSL to 0.20260204.0 (#19333)

angt committed 2 days ago

Verified a4ea7a18

metal : add diag (#19330)

ggerganov committed 2 days ago

Verified 7a4f97d1

vulkan: fix GPU deduplication logic. (#19222)

okuvshynov committed 2 days ago

Verified a498c75a

vulkan: Set k_load_shmem to false when K is too large (#19301)

jeffbolznv committed 2 days ago

Verified 3409ab84

vulkan: fix non-contig rope (#19299)

jeffbolznv committed 2 days ago

Verified c342c3b9

metal : add missing includes (#19348)

will-lms committed 2 days ago

Verified af252d07

vendor : add missing llama_add_compile_flags (#19322)

CISC committed 2 days ago

Verified 11fb327b

vendor: update cpp-httplib version (#19313)

taronaeo committed 2 days ago

Verified e6e934c5

codeowners : add danbev for examples/debug (#19332)

danbev committed 3 days ago

Verified b536eb02

Older