[MLAS] Riscv64 rvv kernel optimizations (#32540)
### Description
Improve FP32 inference in the MLAS riscv64 RVV backend with five
commits:
1. **Softmax:** accumulate across each row, reduce once at the end, and
use LMUL 4.
2. **NCHWc convolution:** compute four adjacent outputs with independent
accumulators and shared filter loads.
3. **NCHWc register groups:** select LMUL 1/2/4 from the vector length,
preserving the sixteen-float block layout.
4. **Depthwise convolution:** extend the existing
`sconv_depthwise_kernel_rvv.cpp` to additional shapes and strides,
with unit dilation and kernel width up to sixteen. Enable the wider
route only when RVV kernels are installed.
5. **Activation:** add an RVV `MlasActivation` routine, including the
clamp used by ReLU6.
Changes are limited to MLAS and its CMake source list. Public interfaces
remain unchanged.
**Performance**
A210/C920V2: VLEN=128; SpacemiT K3/X100: VLEN=256.
**Baseline** is unmodified upstream `main` at `9f913ae52`, which already
includes #32406.
**Optimized** is that baseline with this PR's five RVV kernel changes.
Both use the CPU Execution Provider, `ORT_ENABLE_ALL`, and RVV kernels
enabled, with the same toolchain
and build configuration. Runs use one pinned core, one thread, warmup,
fixed frequency, and best of three.
Speedup is baseline latency divided by optimized latency. These
measurements precede the final depthwise
file consolidation and latest rebase.
| Model | Input shape | C920V2 baseline (ms) | C920V2 optimized (ms) |
Speedup | X100 baseline (ms) | X100 optimized (ms) | Speedup |
|---|---|---|---|---|---|---|---|
| [retinaface](https://github.com/serengil/retinaface) | 1x640x640x3
NHWC | 22300.2 | 7404.1 | **3.01x** | 11327.5 | 4037.0 | **2.81x** |
|
[resnet50-v1-7](https://github.com/onnx/models/blob/main/validated/vision/classification/resnet/model/resnet50-v1-7.onnx)
| 1x3x224x224 NCHW | 1398.1 | 567.3 | **2.46x** | 816.3 | 271.6 |
**3.01x** |
|
[mobilenetv2-12](https://github.com/onnx/models/blob/main/validated/vision/classification/mobilenet/model/mobilenetv2-12.onnx)
| 1x3x224x224 NCHW | 148.2 | 70.7 | **2.10x** | 93.9 | 30.4 | **3.09x**
|
|
[mobilevit-s](https://github.com/andestech/ModelZoo/blob/master/MobileViT-Small/Model/mobilevit-s.onnx)
| 1x3x256x256 NCHW | 845.8 | 419.7 | 2.02x | 356.8 | 200.2 | 1.78x |
|
[shufflenet-v2-10](https://github.com/onnx/models/blob/main/validated/vision/classification/shufflenet/model/shufflenet-v2-10.onnx)
| 1x3x224x224 NCHW | 68.5 | 47.2 | 1.45x | 41.9 | 20.3 | 2.06x |
|
[efficientnet-b0](https://huggingface.co/qualcomm/EfficientNet-B0/blob/main/EfficientNet-B0.onnx)
| 1x3x224x224 NCHW | 244.1 | 189.5 | 1.29x | 124.3 | 67.1 | 1.85x |
RetinaFace uses the official serengil TensorFlow/ResNet50 implementation
and weights, exported with tf2onnx
at opset 13.
**Validation**
Before the latest rebase: XuanTie cross-compilation passed under
`-Werror`. After depthwise consolidation,
the K3 convolution/activation subset passed **1109 tests with RVV** and
**565 with forced scalar**,
with zero failures and three HalfConv skips in each mode.
Softmax/log-softmax differential checks against
a double-precision reference passed on both boards.
### Motivation and Context
Follow-up to #28261 and #28411, building on the runtime gate restored in
#32406.
The existing NCHWc kernels reached only 0.20x–0.84x of the alternative
convolution path's throughput
across seven measured shapes. Independent accumulators and appropriately
sized register groups address
these bottlenecks while preserving the blocked layout.
Broader depthwise routing avoids im2col plus a single-row GEMM per
channel. It benefits convolutions
that cannot use NCHWc, including some ShuffleNet layers at the default
optimization level, as well as
`ORT_ENABLE_EXTENDED`. Fewer softmax reductions and vectorized
activation address the remaining overhead.
