llama.cpp
befddd0f - Vulkan Bugfixes and Improvements (#7084)

Commit
1 year ago
Vulkan Bugfixes and Improvements (#7084) * Modify mat mat mul shader for mul_mat_id, modify mat vec mul shaders for single call batch operation * Further work towards MoE, disabled for now * Disable MoE code (not ready yet), fix a number of bugs in shaders and Vulkan code * Add softmax with f16 mask and pos buffer support * Disable mul_mat_id shaders for now * Fix flake8 * Fix validation errors caused by empty buffers on larger batch sizes
Author
Parents
Loading