onnxruntime
7001d9a1 - Add weightless hyper-connection fusion operators (#32687)

Commit
14 days ago
Add weightless hyper-connection fusion operators (#32687) ### Description Adds four `com.microsoft` operators for weightless hyper-connections: - `BranchwiseRMSNorm` - `ScaledSiLU` - `HyperConnectionPreMix` - `HyperConnectionPostMix` Key changes: - Adds schemas, CPU/CUDA/WebGPU registrations, and per-operator source files. - Reuses existing SimplifiedLayerNormalization implementations for compatible `BranchwiseRMSNorm` paths. - Retains targeted fallbacks for optional, mixed-type, and branch-specific scales. - Adds dedicated symbolic shape-inference handlers for each operator. - Adds operator coverage and generated documentation. ### Motivation and Context Provides portable fused primitives for weightless hyper-connection models while sharing established normalization infrastructure where semantics permit. Dedicated schemas and shape-inference handlers preserve each operator’s distinct contract without coupling them to unrelated operators. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: kunal-vaishnavi <115581922+kunal-vaishnavi@users.noreply.github.com> Co-authored-by: Kunal Vaishnavi <kvaishnavi@microsoft.com>
Author
Parents
Loading