llama.cpp
7fce5439 - ggml : add BF16 support to ROPE and SET_ROWS (CPU, CUDA) (#32387)

Commit
1 day ago
ggml : add BF16 support to ROPE and SET_ROWS (CPU, CUDA) (#32387) * ggml : add BF16 support to ROPE and SET_ROWS (CPU, CUDA) * cuda : SET_ROWS from BF16 into quantized types and F8_E4M3 * cuda : fuse BF16 rope into a BF16 cache * tests : trim the BF16 rope cases to one per code path
Author
Parents
Loading