onnxruntime
d889b8bd - [WebGPU] optimizations for audio models (DFT, STFT, ConvTranspose, LSTM, GRU) (#33978)

Commit
10 hours ago
[WebGPU] optimizations for audio models (DFT, STFT, ConvTranspose, LSTM, GRU) (#33978) ### Description This introduces important optimizations for audio models, which tends to rely heavily on nodes such as DFT, STFT, ConvTranspose and for some LSTM & GRU. ### Motivation and Context WebGPU didn't have those node implemented (such as STFT) or they were not really optimized (DFT only optimized for sizes <=512, slow ConvTranspose, LSTM & GRU). The speed gain was tested across 4 popular audio model architectures: HTDemucs, UVR, SCNet, MP-SENet on 2 different hardware accelerators: NVIDIA RTX 3090, and Apple M3. The measured speed gain before and after these changes: NVIDIA RTX 3090 on Windows 11 (CPU: Intel 12700K): Model | WebGPU before | WebGPU after | Gain -- | -- | -- | -- | HTDemucs | 1132 ms | 265 ms | 4.3x SCNet | 12440 ms | 525 ms | 23.7x UVR | 457 ms | 342 ms | 1.34x MP-SENet | 203 ms | 41 ms | 4.95x Apple M3 on macOS 14.7: Model | WebGPU before | WebGPU after | Gain | Output gap before vs after -- | -- | -- | -- | -- HTDemucs | 3308 ms | 1436 ms | 2.30× | rel_l2 1.2e-6 SCNet | 38043 ms | 3114 ms | 12.2× | rel_l2 7.7e-6 UVR | 2130 ms | 2014 ms | 1.06× | rel_l2 4.7e-7 MP-SENet | 489 ms | 261 ms | 1.87× | rel_l2 1.4e-6 Op usage: Model | DFT | STFT | ConvTranspose | LSTM| GRU -- | -- | -- | -- | -- | -- HTDemucs | ✅ | ✅ | ✅ | –| – SCNet | ✅ | ✅ | ✅ | ✅| – UVR | – | – | – | ✅| – MP-SENet | – | – | – | -| ✅ --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Author
Parents
Loading