Finegrained kernels: unified quantization kernels api #48058
IlyasMoutawwakil
wants to merge
37 commits into
main
from
finegrained-unified
finegrained kernels
64708a24
deepgemm and tp/ep fixes
7b21bdb7
finegrained: adopt the interleaved gate|up layout
05bd0f35
finegrained: keep DeepGEMM Mega MoE on the stacked gate|up layout
686238ed
fix
641fcb2c
Merge remote-tracking branch 'origin/main' into finegrained-unified
e6b20c4a
finegrained: build the per-call kernel objects instead of caching them
cb04cbe0
finegrained: fuse both MoE biases into the GEMMs
e1c7a921
Merge origin/main into finegrained-unified
0a1b49b7
Merge remote-tracking branch 'origin/main' into finegrained-unified
90413201
finegrained: expert layouts as conversion ops; experts forwards on th…
eac0ee87
style
1d94e557
finegrained: on-the-fly quantization for every format; loader keeps c…
fb16472d
finegrained: pass input_recipe to the dispatchers, drop the Quantizat…
07dae500
Revert "finegrained: pass input_recipe to the dispatchers, drop the Q…
0b6ca4c9
finegrained: pass activation_format straight through; let deep-gemm s…
513e2314
hub_kernels: finegrained-kernels ships as version 0
240abc9b
hub_kernels: load finegrained-kernels from the kernels-staging build …
30ab2c21
finegrained: consume modelopt's calibrated globals, and fuse a model'…
a562fb8c
finegrained: address the integration review
89267bf3
finegrained: freeze the mxfp4 and nvfp4 integrations, and serve what …
09ea21c0
Merge branch 'main' into finegrained-unified
d59f9037
fix style
e139d2a0
FineGrainedFP8Config is its own class again, not an alias of the new one
c6656441
FineGrainedFP8Config: drop the frozen-notice docstring line
29cca042
Tolerate an experts module without the norm, and a machine without CUDA
f63ad97d
Pin that every modelopt converter target is a slot the module holds
1265f0cc
is_sm100: answer False without CUDA, and fake both halves in the tests
97dfa106
fix
788d02dd
finegrained: drop the DTensor unwrap, and pass the kernel operands by…
53a8970a
deepgemm: only Mega MoE cannot apply a post-expert norm
e8e6b879
Shard quantized expert companions under EP and TP
43b9a748
fixes for the new model
fe0b7182
fix multimodal and add some testing
22b1001c
static
f9302cc2
Load a calibrated checkpoint's input_scale, and fix what per-tensor w…
daa24205
fix
b4dcb7b0
Login to write a write a comment.
Login via GitHub