DeepGEMM BF16 + mixed FP8/FP4 + MegaMoE + refactor #45634
IlyasMoutawwakil
merged
164 commits into
main
from
deepgemm-isolation
init
94a2786b
Merge branch 'main' into deepgemm
8fc43d72
style
357a0355
full support
741b5eb7
support EP better using offsets !
9fc3662d
comments
84552ae9
get rid of neutralize_ep_sentinels
1d9f319b
remove deepgemm stuff
9b860434
Merge branch 'main' into deepgemm
8a3f88bb
fix
996d67d0
prefix
d033a830
move
e15cfe6a
fix
10b6d904
remove comment
d4a6b305
fix unintilized outputs leaking
1d6054ff
revert unnecessary changes
137393cd
more unnecessary changes
774f9018
revert downcast
81230fee
keep it simple
9f2ff089
guard deepgemm cuda version
c55b7b78
fix style
20858db8
update
bfea94f0
IlyasMoutawwakil
changed the title
add deepgemm testing
eada47e1
moe sentinel support
89d2f0bb
Merge branch 'main' into deepgemm
ef72b0e4
fix
60db1ca0
Merge branch 'main' into deepgemm
e732af1d
compilable sonicmoe
68b7b0fe
mega moe kernel support attempt
faaa7aaa
use package for now
b502cd62
IlyasMoutawwakil
changed the title
Merge branch 'deepgemm' into deepgemm-isolation
cc753ca5
skip ep router and experts pre/post processing
1c174528
simpler
5a8ceaea
fix
a05aa39b
Merge branch 'main' into deepgemm-isolation
854f4fb2
fix
053c9df6
fix
80a6fe5a
dtensor support
ad8226ce
more dtensor
a663f4d7
simpler
74c3f2e3
remove comment
d3cae33e
Merge branch 'deepgemm' into deepgemm-isolation
c1bfa0f0
Merge branch 'main' into deepgemm-isolation
6c436114
revert
0528e0e2
bc order
2bfd0298
revert extra indent
368db007
revert unnecessary change
28fbb842
update
6ef27abe
less defensive
e436b774
allow all kernels
8e8f0ee2
alow all kernels
c494e35b
hub only
03b04421
Merge branch 'main' into deepgemm-isolation
527d8b47
fix
2e51b3cb
fix
82c5fb59
test
99fdf71d
test
21398a8b
sync
f21db7cd
check nvcc
1fe37680
probe
bbfda35e
fix
68a373cd
test psum
825f6d49
test
028a39f1
test
ee173a5f
probe
900e984d
fix
09a711f3
test
a0d49400
nan issue
653b7b3b
repro
ed8af6bb
repro
fb8d3388
fix
562eb512
simplifications
481e5e68
fix
d3dbd325
fix
9f168ff1
fix
7274c22e
fix
a800b8c3
fix
804c9885
fix
7f365628
fix
6df9f47e
empty
c3432c94
simplify
77f09fe5
test deepseek
7dfbedde
dsv4 only
689cc296
download dsv4
8de089ce
fix test
cb4d6f9e
Merge branch 'main' into deepgemm-isolation
389eee8d
push
8f29ed6a
test
946e2007
fix
6619647d
fix ep plan
913339ac
fix attempt
7cfc2b26
debug
4c12f6ea
attempt
92e63378
debug
a9997f59
fixes in modeling
197a8f62
more modeling changes
30d0780d
more modeling changes
3b1c470a
initial megamoe works but wrong output, deepgemm fully functional
51a9425b
mega moe works !
a4130db8
simplification
b140f7c1
comments
9ff656bc
ArthurZucker
commented
on 2026-05-12
Merge branch 'main' into deepgemm-isolation
b5b741e5
fix merge
12312581
fix rank divergence
a8cc32ac
use _keep_in_fp32_modules
c6486cbc
simplify quantizer
bd68b6f8
introduce indexerscorer
8630f9d1
remove testing file
f3b933f6
style
029a4c91
weight rename for scorer
85b57686
style
7f19516d
address review comments
e395e4c8
style
1b94261e
align with laguna
af51a6e1
Merge branch 'main' into deepgemm-isolation
ca934bf5
stale comments and styling
f18ab28c
ArthurZucker
approved these changes
on 2026-05-13
ArthurZucker
commented
on 2026-05-13
use grouped finegrained fp8 for grouped linear
26f3ba70
Merge branch 'main' into deepgemm-isolation
4039ed69
comments and docs
9f2c97b8
Revert "[DeepSeek V4] Fix MoE converter substring-matching FP8 scale …
8dc75bd6
Merge remote-tracking branch 'origin/main' into deepgemm-isolation
57081ccf
fix quantizer and use upstream deepgemm
78fcee66
remove force al kernels
2e4203fa
[deepseek_v4] fix FP8 save-side substring match + fp32 gate through s…
5f9e5cdf
always floats scal inv with triton
afd1b708
minimal changes i quantizer
619b5e9b
force stable abi build for now
3e76df5e
support transposed
f0e2859e
fix yarn
d0c4a6df
unnecessary change
c1adcd70
add flash base test and fix the incorrect and crash issue during enab…
143835f3
merge
b47b30f0
better errors across deepgemm
8bd9619e
more better errors for deepgemm
751c8b5c
ep tests
70c7c3b9
Apply suggestions from code review
464512e1
scope supports_round_trip to quantization ops only
d6fd64db
Apply suggestions from code review
6e175094
Update src/transformers/core_model_loading.py
32596183
style: ruff format
2c06aefc
custom ep/tp plan for megamoe
7dfe6bda
process megamoe after weight loading
18891635
fix megamoe process
46dc2897
up
e9ccaa05
the actual fix???
7d746846
up
87d2ee99
nit comment for futur
7be266b7
style: drop unused exception variable in Fp8Dequantize._dequantize_one
5e102f16
up
8631e6fb
style: apply make fix-repo docstring formatting
b05bda48
Merge pr-46265 into deepgemm-isolation, drop redundant scale-merge wo…
7481aa67
simplify after merge
f7698471
review comments
2b8bba05
Merge branch 'main' into deepgemm-isolation
beac5f22
revert unncessary
f72bc20c
avoid name contraction
b5f2d451
fixes
014707d7
dynamic tp/ep plan swap
2a77aa76
fix deepgemm single error and device assertion docs
6922e3a9
properly support bias in grouped linear
74711095
simplify
b63ce4fd
fix dist error handling and min cuda compatbility
c12e8924
fix least cuda compat
1ec0147e
get alignment from deepgemm and cache it
4fd793d3
Login to write a write a comment.
Login via GitHub