onnxruntime
1077c77e - [MLAS] Fix MatMulNBits CPU LUT GEMM path to apply optional bias (#28742)

Commit
67 days ago
[MLAS] Fix MatMulNBits CPU LUT GEMM path to apply optional bias (#28742) ### Description The MatMulNBits LUT GEMM path (enabled by session config mlas.use_lut_gemm=1) silently dropped the optional bias input, causing incorrect outputs whenever the graph optimizer's MatMulNBits+Add fusion produced MatMulNBits nodes with bias populated. This is the common case for transformer models with fused linear+bias layers. Fuses the bias broadcast-add into the existing per-tile parallel loop in MlasLutGemm so the addition is multi-threaded and operates on data already hot in cache. Also gates prefer_lut_gemm_ to T1==float since MlasLutGemm interprets A/C as float* internally. Adds MLAS-level (test_sqlutgemm SQLutGemmBiasTest) and op-level (matmul_2bits_test MatMulNBitsLutGemm.*Bias) coverage for the bias-present case across BlkLen 32/64/128, M in {1,32}, both symmetric and asymmetric quantization. ### Motivation and Context Fix correctness issues with LUT Gemm --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Author
Parents
Loading