llama.cpp
Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing
#10597
Merged

Commits
  • Vulkan: Implement VK_KHR_cooperative_matrix support in the matrix matrix multiplication shader
    0cc4m committed 1 year ago
  • Improve performance with better q4_k and q5_k dequant and store unrolling
    0cc4m committed 1 year ago
  • Add Vulkan MUL_MAT and MUL_MAT_ID accumulator precision selection
    0cc4m committed 1 year ago
  • Rework mulmat shader selection and compilation logic, avoid compiling shaders that won't get used by device
    0cc4m committed 1 year ago
  • Vulkan: Implement accumulator switch for specific mul mat mat shaders
    0cc4m committed 1 year ago
  • Vulkan: Unroll more loops for more mul mat mat performance
    0cc4m committed 1 year ago
  • Vulkan: Add VK_AMD_shader_core_properties2 support to read Compute Unit count for split_k logic
    0cc4m committed 1 year ago
  • Disable coopmat support on AMD proprietary driver
    0cc4m committed 1 year ago
  • Remove redundant checks
    0cc4m committed 1 year ago
  • Add environment variable GGML_VK_DISABLE_COOPMAT to disable VK_KHR_cooperative_matrix support
    0cc4m committed 1 year ago
  • Fix rebase typo
    0cc4m committed 1 year ago
  • Fix coopmat2 MUL_MAT_ID pipeline selection
    0cc4m committed 1 year ago
Loading