llama : rotate activations for better quantization #21038
ggerganov
merged
12 commits into
master
from
gg/attn-rot
CISC
approved these changes
on 2026-03-28
pwilkin
approved these changes
on 2026-03-31
llama : rotate activations for better quantization
4d68f97e
cont : rotate V more + refactor
cb6e21d8
cont : rotate caches separately + support non-power-of-2 head sizes
d467d3d0
cont : simplify
898a8fe6
cont : add reference for V rotation
eaefe0f0
cont : refactor
66b72b41
cont : support context shift
62adb6a1
cont : consolidate
d424a885
cont : dedup + allow different types for the rotation matrix
69e476d3
cont : add env variable to disable rotation
29f41968
cont : simplify attn rot kv cache logic + rename env
8df24c07
cont : pre-compute the Hadamard matrices
a0c4a2a7
Login to write a write a comment.
Login via GitHub