llama.cpp
a49db739 - cuda : raise shared mem limit only for the launched kernel (#32392)

Commit
1 day ago
cuda : raise shared mem limit only for the launched kernel (#32392) This commit removes a call of the CUDA_SET_SHARED_MEMORY_LIMIT macro as the fallback decision was moved out of launch_mul_mat_q. launch_mul_mat_q now has the fallback template parameter which was added in Commit 6eddde06a4f25d55d538b5d15628dcc2b6882147 ("CUDA: refactor MMQ kernel configuration (#24127)"). This code was then refactored in Commit c49edf0963a76d5e78b2a7278c9bd5b71d26c84f ("CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948)") which added the has_fusion template parameter but it kept both macro calls. launch_mul_mat_q only launches one type of kernel now (this is now decided earlier in the templated call chain) and each specialization will set the limit itself. The motivation for this is to simplify the code and improve readabilty.
Author
Parents
Loading