transformers
🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan
#49778

Open

🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan #49778

IlyasMoutawwakil wants to merge 14 commits into fg-split/5-fp8 from fg-split/6-mxfp4

IlyasMoutawwakil force pushed from 0e0531aa to ae62b387 6 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 15bcb879 to 551185c0 6 days ago

IlyasMoutawwakil force pushed from ae62b387 to 5b3c9106 6 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 551185c0 to 4a0d3717 6 days ago

IlyasMoutawwakil force pushed from 5b3c9106 to f41b6f51 6 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 4a0d3717 to 1f9a2f66 6 days ago

IlyasMoutawwakil changed the title ~~MXFP4 on the fine-grained machinery, with DeepSeek-V4's FP8 + MXFP4 plan~~ MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan 6 days ago

IlyasMoutawwakil force pushed from f41b6f51 to d0129d79 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 1f9a2f66 to 909f2cec 5 days ago

IlyasMoutawwakil force pushed from d0129d79 to b1979e6c 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 909f2cec to 50cb07cc 5 days ago

IlyasMoutawwakil force pushed from b1979e6c to 4991a3bb 5 days ago

IlyasMoutawwakil force pushed from 4991a3bb to b6b30744 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 8d50a391 to ef5e58c2 5 days ago

IlyasMoutawwakil force pushed from b6b30744 to d1069758 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from ef5e58c2 to c39d6ae5 5 days ago

IlyasMoutawwakil force pushed from d1069758 to 3837fcac 5 days ago

IlyasMoutawwakil force pushed from 3837fcac to a1dd17d2 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from c39d6ae5 to cd37228f 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from cd37228f to bc2c9912 5 days ago

IlyasMoutawwakil force pushed from a1dd17d2 to 7183af2b 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from bc2c9912 to e3a43ca1 5 days ago

IlyasMoutawwakil force pushed from 7183af2b to 75e78c65 5 days ago

IlyasMoutawwakil force pushed from 75e78c65 to c81614cc 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from e3a43ca1 to 32b23c9f 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 32b23c9f to e3a89a2a 5 days ago

IlyasMoutawwakil force pushed from c81614cc to e4ac490c 5 days ago

IlyasMoutawwakil force pushed from e4ac490c to 941661e2 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from e3a89a2a to b802f37b 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from b802f37b to 5df081e0 5 days ago

IlyasMoutawwakil force pushed from 941661e2 to b1939494 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 5df081e0 to a807b250 5 days ago

IlyasMoutawwakil force pushed from b1939494 to c3d0d8b2 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from a807b250 to 3acb4399 5 days ago

IlyasMoutawwakil force pushed from c3d0d8b2 to 15ee3366 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 3acb4399 to 35113a47 5 days ago

IlyasMoutawwakil force pushed from 15ee3366 to 9c003afe 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 35113a47 to f3394183 5 days ago

IlyasMoutawwakil force pushed from 9c003afe to f8f9371e 5 days ago

IlyasMoutawwakil changed the title ~~MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan~~ 🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan 5 days ago

IlyasMoutawwakil force pushed from f8f9371e to d9efb5bb 5 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from f3394183 to 5d804e34 5 days ago

IlyasMoutawwakil force pushed from d9efb5bb to 0eb9021a 4 days ago

IlyasMoutawwakil force pushed from 0eb9021a to 1aafb732 4 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from e0fa2b7c to 507c0d36 4 days ago

IlyasMoutawwakil force pushed from 0eb9021a to 1aafb732 4 days ago

IlyasMoutawwakil force pushed from 1aafb732 to ba15098d 4 days ago

IlyasMoutawwakil force-pushed the fg-split/5-fp8 branch from 507c0d36 to 2eb35389 4 days ago

IlyasMoutawwakil force pushed from ba15098d to 0b94bcb1 3 days ago

Move Mxfp4Config onto the fine-grained machinery, with DeepSeek-V4's …

1c1d6fac

Document MXFP4 on the fine-grained machinery and DeepSeek-V4's plan

8c4e1312

Test that Mxfp4Config takes main's positional arguments

aa7faff3

Require compute capability 8.9 for every MXFP4 activation format

e4e0ee00

Dequantize MXFP4 experts for use_kernels on every device

3b1979e5

Explain why MXFP4 defaults to DeepSeek's layout and W4A4 experts

4d1cc69d

Keep DeepGEMM grouped experts out of DeepSeek-V4's SM100 sweeps

afeff1c8

Keep DeepSeek-V4's Triton sweeps on Triton linears through prefer_dee…

acfa6992

Read GPT-OSS's experts keys through get_weight_conversions

ea7a1ece

Keep main's MXFP4 integration for GPUs below compute capability 8.9

0411198f

Read DeepSeek-V4's MXFP4 experts plan in the fp8 reader

6c840c54

Leave MXFP4's ModelOpt layout to the producer readers' PR

3e3b270e

Give the DeepSeek-V4 load-path fixture its plan explicitly

c3e232f5

Restack on the simpler plan resolution and merging

b0750fd3

IlyasMoutawwakil force pushed from 0b94bcb1 to b0750fd3 3 days ago

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers 🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan #49778 Open

🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan #49778

transformers
🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan
#49778

Open