llama.cpp
metal : optimize FA vec for large sequences and BS <= 8
#15566

Merged

metal : optimize FA vec for large sequences and BS <= 8 #15566

ggerganov merged 4 commits into master from gg/metal-fa-vec-opt-2

github-actions added examples

github-actions added ggml

github-actions added Apple Metal

Base automatically changed from gg/metal-mmid-opt to master 290 days ago

metal : optmize FA vec for large heads and sequences

ef681866

metal : adjust small-batch mul mv kernels

9c0fe8ed

batched-bench : fix total speed computation

6d0b2222

ggerganov force pushed from aed06a93 to 6d0b2222 290 days ago

cont : add comments

a92bdd9a

ggerganov merged b3964c1e into master 290 days ago

ggerganov deleted the gg/metal-fa-vec-opt-2 branch 290 days ago

Reviewers

No reviews

Assignees

No one assigned

Labels

examples ggml Apple Metal

Milestone

No milestone