llama.cpp
5c278dce - llama, metal : support BF16 src1 for MUL_MAT and MUL_MAT_ID

Commit
1 day ago
llama, metal : support BF16 src1 for MUL_MAT and MUL_MAT_ID Add GGML_PREC_BF16 handling for src1 on the Metal backend when the tensor API is available: MUL_MAT with an F32 src0 stages the A tile as bfloat, and MUL_MAT_ID stages src1 as bfloat instead of F16. The tensor path keeps src1 in F32 when it reads the operand directly. Add a temporary LLAMA_MOE_AMAX_SKIP env var that disables the graph-level amax rescaling for the MoE down projection and sets the BF16 src1 hint instead. Apply the hint inside build_lora_mm_id so it reaches the matmul before the per-expert scale and LoRA nodes wrap it. Add MUL_MAT_ID src1 BF16 test cases to test-backend-ops. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
Author
Parents
Loading