llama.cpp
metal : per-device tuned (Q, NE) for flash-attn vec (replay)
#26570

Open

metal : per-device tuned (Q, NE) for flash-attn vec (replay) #26570

forforever73 wants to merge 12 commits into stack/metal-split from stack/metal-fa-vec

forforever73 requested a review from

ggerganov 9 days ago

forforever73 requested a review 9 days ago

forforever73 changed the base branch from master to stack/metal-split 9 days ago

forforever73 force-pushed the stack/metal-split branch from 864c08ed to 2934b5b3 9 days ago

forforever73 force pushed from 5d79b775 to aa4b5996 9 days ago

github-actions added testing

github-actions added ggml

github-actions added Apple Metal

metal : per-device tuned (Q, NE) for flash-attn vec (#25750)

0f279350

add device id -> token lookup for the offline tuning tool

eb052925

add ggml-metal-tuning skeleton

55318e3b

add op-agnostic perf cell + median timing for the tuner

cf80ea68

add FA-vec graph build + tensor init to the tuner

8e7831fd

tools : add FA-vec (Q,NE) sweep, compression and table emit

f44d2ee2

cool down and re-measure the dirty window on thermal drift

35c08cd0

test-backend-ops : replace the FA vec tune mode with a bounded (Q,NE)…

26edf081

tools : document the Metal tuner, point the table comment at it

ff024685

abort on unknown KV type, single-source fa_vec_legal_ne

9be171f5

cleanup

8045779c

forforever73 force-pushed the stack/metal-split branch from 2934b5b3 to 55224983 9 days ago

forforever73 force pushed from aa4b5996 to 0f279350 9 days ago

github-actions added documentation

github-actions added examples

ggerganov commented on 2026-08-11

honor -o in the FA vec (Q,NE) slice

b2d43d62

Reviewers

ggerganov

Assignees

No one assigned

Labels

documentation testing examples ggml Apple Metal

Milestone

No milestone

llama.cpp metal : per-device tuned (Q, NE) for flash-attn vec (replay) #26570 Open

metal : per-device tuned (Q, NE) for flash-attn vec (replay) #26570

llama.cpp
metal : per-device tuned (Q, NE) for flash-attn vec (replay)
#26570

Open