[MLAS] Add AVX512 (+VNNI) 2-bit weight CPU kernels (#29064)
### Description
Today the only high-performance MLAS kernels for 2-bit weights are the
LUT-based kernels. They under-utilize AVX-512 (e.g. they don't issue
VNNI dot-product instructions), come with shape constraints (most
notably `N` must be a multiple of 128), and — most importantly —
under-perform on prefill regimes where the GEMM is compute-bound rather
than weight-bandwidth-bound. Some of the shape limitations are being
chipped away at incrementally (e.g.
[#28528](https://github.com/microsoft/onnxruntime/pull/28528)), but the
underlying performance gap remains.
This PR addresses the gap directly by adding a "traditional" non-LUT
2-bit weight AVX-512 (+VNNI) GEMM kernel to MLAS, modeled on the
existing 4-bit and 8-bit kernels in the same family. It supports
`BlkLen` ∈ {32, 64, 128} and runs on any shape (no `N % 128 == 0`
constraint). A real customer model heavily benefits from this change.
The customer's model has 120 W2 `MatMulNBits` nodes. ~80 of them have
shapes the LUT kernel supports (`N % 128 == 0`); the remaining ~40 do
not. On `main`, those 40 nodes always fell through to the slow fp32
dequant + SGEMM fallback regardless of the LUT session option. This PR
replaces that fallback with a native AVX-512(+VNNI) W2 kernel and adds a
fast non-LUT path for all 120 nodes (the new kernel beats W2 LUT on the
overlapping 80 nodes at prefill shapes and roughly matches it on decode
shapes).
### Per-node routing
| Branch | LUT mode | LUT-supported nodes (~80) | LUT-unsupported nodes
(~40) |
|---|---|---|---|
| main | LUT off (default) | fp32 dequant + SGEMM | fp32 dequant + SGEMM
|
| main | LUT on | LUT kernel | fp32 dequant + SGEMM |
| This PR | LUT off (default) | new W2 AVX-512(+VNNI) kernel | new W2
AVX-512(+VNNI) kernel |
| This PR | LUT on | LUT kernel | new W2 AVX-512(+VNNI) kernel |
### End-to-end throughput (tokens/sec, higher is better)
| Branch | LUT mode | seq=32 | seq=64 | seq=128 |
|---|---|---:|---:|---:|
| main | LUT on (80 LUT + 40 fp32 fallback) | 587 | 538 | 554 |
| This PR | LUT on (80 LUT + 40 new kernel) | 681 | 722 | 764 |
| This PR | LUT off (120 new kernel) | 1179 | 1338 | 1487 |
| W4 baseline | n/a | 1245 | 1445 | 1647 |
**Headline:** With this PR + the non-LUT path, W2 throughput is now
within ~5–10% of W4. The remaining gap is the W2 "unpack tax" inherent
to 2-bit weight kernels (more bytes-per-output-element to unpack vs.
4-bit).
## Behavior with `mlas.use_lut_gemm` opt-in
User-facing dispatch precedence is unchanged. When the user opts into
LUT via the session config, LUT is still preferred for the shapes where
it is available (`MlasIsLutGemmAvailable` returns true). The new W2
native kernel takes over in two cases:
* **Default sessions (no LUT opt-in)** — this is the headline behavior
change. Previously these sessions had no fast 2-bit path on AVX-512 and
would fall back to dequant + SGEMM; now they run the new native W2
kernel.
* **LUT-enabled sessions on shapes where LUT is unavailable** (e.g. `N`
not a multiple of 128). Previously these shapes hit the dequant+SGEMM
fallback inside the LUT path; now they run the much faster native W2
kernel.
Net effect: LUT-enabled users get a faster fallback path, default users
get a fast default path, and there is no regression in either case.
## fp16 fallback path for 2-bit weights
Semi-related addition in
`onnxruntime/contrib_ops/cpu/quantization/matmul_nbits.cc`:
* `MatMulNBits<MLFloat16>::ComputeBUnpacked` previously only handled
`nbits_ == 4`, with everything else falling into an `ORT_ENFORCE(nbits_
== 8)` branch — meaning fp16-activation 2-bit models would fail at
runtime. Added the missing `nbits_ == 2` arm that calls
`MlasDequantizeBlockwise<float, 2>` and then runs standard fp32 MatMul
(same dequant-then-SGEMM pattern as the existing 4-bit/8-bit fp16
paths).
* Added a `!prefer_lut_gemm_` guard in `MatMulNBits<T1>::PrePack` so we
don't re-pack scales/zero-points in the W2 layout on top of an
already-LUT-packed `packed_b_` buffer (which corrupts the LUT layout and
crashes the LUT compute path). The guard is gated to `T1 == float` via
the constructor, so the fp16 path is unaffected.
### Motivation and Context
Adds a native AVX-512 (+VNNI) 2-bit weight CPU GEMM kernel to MLAS so
that 2-bit quantized models have a competitive non-LUT path — closing
the prefill perf gap versus 4-bit and removing LUT's shape-multiple
constraints — and fixes a missing fp16 fallback so fp16-activation 2-bit
models load and run on CPU.
