llama.cpp
7c7836d9 - Vulkan Shader Refactor, Memory Debugging Option (#7947)

Commit

357 days ago

Vulkan Shader Refactor, Memory Debugging Option (#7947) * Refactor shaders, extract GLSL code from ggml_vk_generate_shaders.py into vulkan-shaders directory * Improve debug log code * Add memory debug output option * Fix flake8 * Fix unnecessary high llama-3 VRAM use

References

#7947 - Vulkan Shader Refactor, Memory Debugging Option

Author

0cc4m

Parents

0c7b3595

Files54

CMakeLists.txt
Makefile
ggml-vulkan-shaders.hpp
ggml-vulkan.cpp
ggml_vk_generate_shaders.py
vulkan-shaders
- add.comp
- argsort.comp
- clamp.comp
- copy.comp
- dequant_f32.comp
- dequant_funcs.comp
- dequant_head.comp
- dequant_q2_k.comp
- dequant_q3_k.comp
- dequant_q4_0.comp
- dequant_q4_1.comp
- dequant_q4_k.comp
- dequant_q5_0.comp
- dequant_q5_1.comp
- dequant_q5_k.comp
- dequant_q6_k.comp
- dequant_q8_0.comp
- diag_mask_inf.comp
- div.comp
- gelu.comp
- generic_binary_head.comp
- generic_head.comp
- generic_unary_head.comp
- get_rows.comp
- get_rows_quant.comp
- mul.comp
- mul_mat_split_k_reduce.comp
- mul_mat_vec.comp
- mul_mat_vec_base.comp
- mul_mat_vec_nc.comp
- mul_mat_vec_p021.comp
- mul_mat_vec_q2_k.comp
- mul_mat_vec_q3_k.comp
- mul_mat_vec_q4_k.comp
- mul_mat_vec_q5_k.comp
- mul_mat_vec_q6_k.comp
- mul_mm.comp
- norm.comp
- relu.comp
- rms_norm.comp
- rope_head.comp
- rope_neox.comp
- rope_norm.comp
- scale.comp
- silu.comp
- soft_max.comp
- square.comp
- sum_rows.comp
- types.comp

llama.cpp 7c7836d9 - Vulkan Shader Refactor, Memory Debugging Option (#7947)

llama.cpp
7c7836d9 - Vulkan Shader Refactor, Memory Debugging Option (#7947)