onnxruntime
Add FP8/FP4 block-quantized support to GatherBlockQuantized contrib op
#32480
Merged
Merged
Add FP8/FP4 block-quantized support to GatherBlockQuantized contrib op #32480
tianleiwu
merged
76 commits into
main
from
copilot/research-onnx-runtime-support
Add Engram contrib op schemas
99d55176
Add CPU Engram contrib kernels
b44523f6
Add CUDA Engram contrib kernels
819e6959
Add WebGPU Engram contrib kernels
608c54f3
Add Engram contrib op tests
3ad34909
Fix Engram CPU threadpool usage
3fd904bd
Split Engram ops into standalone files
1c82a474
Add CPU FP16 Engram kernels, shared EP helpers, and docs
b8a41e7c
Rename NgramHashMapping to NGramHashMapping
fcb7ca07
Address PR review: shared-gate/RMS passes and sign(0) fix for Engram ops
2ddc3069
Fix CI failures: CUDA GetComputeStream, WebGPU namespace, test skip trap
e22b5c81
Merge remote-tracking branch 'origin/copilot/add-op-support-for-deeps…
7bb11f30
Extend GatedRMSNorm with configurable silu/sigmoid gate activation
9eecb7df
Extend NGramHashMapping with past/present tokens, EOS reset, segment_…
08546f92
Extend EngramGate with conv_norm_scale input and gated_value_normed o…
01454fd4
Fix NGramHashMapping generated docs
9c60b274
Clarify ShortConv activation flag
cf85ec46
Fix NGram segment reset review comments
bc2732fd
Fix lint formatting issues
a13d68ab
Extend GatedRMSNorm with configurable silu/sigmoid gate activation
23048f84
Extend NGramHashMapping with past/present tokens, EOS reset, segment_…
a17b7a2a
Extend EngramGate with conv_norm_scale input and gated_value_normed o…
1e8ad6c7
Update Engram generated kernel docs
751a6d07
Format Qwen Engram updates
410de129
Update Qwen PR for merged DeepSeek Engram changes
f7dda340
Fix EngramGate test MSVC shadow warning
d49cbdea
Update generated contrib operator docs
bee74479
Fix CUDA plugin user stream graph test lifetime
dcf54dc4
Add GatherQuantized contrib op (CPU EP) for FP8 block-scaled gather
a132d055
Rename GatherQuantized to GatherFpQuantized and add FP4 (Float4E2M1x2…
39ba0cc7
Document GatherFpQuantized in ContribOperators.md and OperatorKernels.md
c7f63c3e
Add eos_token_id segment-reset to NGramHashMapping (CPU/CUDA/WebGPU)
0095b3ba
Apply lintrunner formatting fixes
0fb86e8c
Support broadcastable (per-tensor) scales in GatherFpQuantized
e661b8a8
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
188cf5d5
kunal-vaishnavi
changed the base branch from
main
to
copilot/add-op-support-for-qwen-3-8-flash-next
7 days ago
kunal-vaishnavi
changed the title
kunal-vaishnavi
changed the title
Merge remote-tracking branch 'origin/main' into copilot/add-op-suppor…
e9b317b0
kunal-vaishnavi
changed the title
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
26d11e77
Address PR review comments: guard FP8 tests, fix block_size validatio…
7a7f0cc7
Fix CI failures: WASM -Wshorten-64-to-32 in GatherFpQuantized kernel,…
c2a76dd3
Merge remote-tracking branch 'origin/main' into copilot/add-op-suppor…
d1d90dff
Merge remote-tracking branch 'origin/main' into copilot/research-onnx…
d2f974ec
Revert "Merge remote-tracking branch 'origin/main' into copilot/resea…
d41fbdb9
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
03ad2f9a
Remove GatedRMSNorm activation changes
f89b2a60
Merge remote-tracking branch 'origin/main' into copilot/add-op-suppor…
da2b6174
Resolve merge conflicts: rebuild tree onto latest copilot/add-op-supp…
b792d550
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
e961436a
Merge GatherFpQuantized schema into GatherBlockQuantized
038990ac
Add FP8/FP4 support to WebGPU GatherBlockQuantized
d678bd0c
Fix OpTester::Run excluded-provider type in migrated GatherBlockQuant…
359d32b7
Apply remaining changes
1b927f27
Fix WebGPU GatherBlockQuantized FP8/FP4 dtype and broadcast-scale han…
902685f3
kunal-vaishnavi
changed the title
Remove stale GatherFpQuantized doc entries; merge into GatherBlockQua…
356d3ced
Apply remaining changes
388be7b6
Fix unrelated GEMM_FLOAT8_TYPES formatting drift flagged by bot review
b874d73a
Fix CI: exclude WebGPU from FP8/FP4 GatherBlockQuantized tests; fix s…
ef3ccd7d
Fix CUDA GatherBlockQuantized FP8/FP4 bugs and wire up kernel registr…
47f185a2
Fix WebGPU divide-by-zero, withhold FP8/FP4 type registration, adjust…
a46941fe
Fix CUDA build error: qualify TArray as onnxruntime::cuda::TArray in …
82ebc35f
apsonawane
commented
on 2026-09-14
apsonawane
commented
on 2026-09-14
apsonawane
commented
on 2026-09-14
apsonawane
commented
on 2026-09-14
Address GatherBlockQuantized review feedback
0477f8cb
Handle empty tensors and high-rank WebGPU scales
dc04cd65
Restore canonical schema macro formatting
2a9e812f
Apply remaining changes
ff695e18
Merge remote-tracking branch 'origin/main' into copilot/add-op-suppor…
fac8b712
Clean up generated docs whitespace
34502f96
Remove NGramHashMapping docs trailing whitespace
e5f2b154
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
1dfac6db
Clean up NGramHashMapping review fixes
33b8c7db
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
a853e5df
Fix NGramHashMapping shape inference docs
c0a37f42
Fix GatherBlockQuantized CI failures
dc635dad
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
b063ff7b
Update generated contrib docs
a03c05ac
Merge remote-tracking branch 'origin/copilot/add-op-support-for-qwen-…
4a58ce19
Sync target and fix formatting
afe56885
Fix WebGPU FP8 lookup shader
8d992e80
Base automatically changed from
copilot/add-op-support-for-qwen-3-8-flash-next
to
main
18 hours ago
tianleiwu
approved these changes
on 2026-09-15
apsonawane
approved these changes
on 2026-09-15
Assignees
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub