microsoft/DeepSpeed

Pull Requests Commits

fix: open the IPC handle on our device context (vLLM-style)

delock committed 3 days ago

a7e999fa

fix: explicitly enable peer access on IPC connect

delock committed 3 days ago

1f66b8f3

fix: connect via raw IPC data pointer (storage reports producer device)

delock committed 3 days ago

c2591ca2

fix: pass the full IPC state tuple to _new_shared_cuda

delock committed 3 days ago

36cfca3f

fix: torch-2.13 CUDA-IPC reconstructor (UntypedStorage._new_shared_cuda)

delock committed 3 days ago

b9f35d11

fix: explicit bf16/half intrinsics (torch build disables conversions)

delock committed 3 days ago

fe9570e9

fix: custom_ar queries topology via torch.distributed (cdb may be unset)

delock committed 3 days ago

5165cb74

feat(comm): one-shot P2P inference all-reduce for CUDA pairs

delock committed 3 days ago

a9bfc6d3

docs: vLLM 0.32 A100 profile — gap decomposition (AR 1.2ms, attn 0.6ms, glue 2.5ms)

delock committed 3 days ago

9bd7c337

Add a v1 multi-rank CPU suite (manual dispatch first) (#8926)

delock committed 3 days ago

Verified 8f9db2c6

docs: final night validation — 186/186 tests, 866/2566/3951 (60/53/52% of vLLM)

delock committed 4 days ago

87acd19b

fix: handle flat GEMV 1-D output in the concatenated attn qkv path

delock committed 4 days ago

5d769303

fix: remove the NT=128 grouped dispatch — numerics bug, no perf win

delock committed 4 days ago

c948a196

docs: loops 9/9b + night summary — 876/2550/3942 (60/52/52% of vLLM)

delock committed 4 days ago

54273dad

fix: materialize concat-sliced k/v for decode_attn contiguity (Loop 9b)

delock committed 4 days ago

64bd29d6

perf(tp): cache concatenated attention qkv weight — 3 GEMMs to 1 (Loop 9)

delock committed 4 days ago

e1baace7

docs: loop 8 — vocab-parallel lm_head +0.8-3.4%, 59/51/52% of vLLM

delock committed 4 days ago

7c8b82e6

perf(tp): vocab-parallel lm_head decode — half weight traffic per rank (Loop 8)

delock committed 4 days ago

2c3c3a04

docs: loop 7 — chunked GLU +0.6% at bs=64

delock committed 4 days ago

30fd45c4

perf: chunked fused GLU for M in (32,64] — avoids composite fp32 roundtrip (Loop 7)

delock committed 4 days ago

0e01795e

docs: loop 6 wash — cast caching kept

delock committed 4 days ago

c0bc8189

perf: cache static A_log/dt_bias fp32 casts — 48 launches off the decode step (Loop 6)

delock committed 4 days ago

1b2bdbd3

docs: loop 5 — weight concat +3-6%, crossed 50% of vLLM everywhere

delock committed 4 days ago

aee67169

perf(tp): cache per-rank concatenated GDN weights — 6 GEMMs to 2 (Loop 5)

delock committed 4 days ago

a1f97781

docs: loop 4 lesson — recurrent kernel loses at TP head counts

delock committed 4 days ago

18d0307e

revert: keep FLA for TP recurrent steps — our kernel loses at H<=16 (Loop 4)

delock committed 4 days ago

a64d46b9

fix: materialize strided TP q/k/v views for the in-place GDN recurrent step

delock committed 4 days ago

d4d7cadd

diag: one-shot scan routing print (temp)

delock committed 4 days ago

c59611fa

docs: loops 3/3b — GLU routing lesson, recovered + bs64 best

delock committed 4 days ago

9968267f

fix: fused GLU exempt from arch threshold — wins on both archs (Loop 3b)

delock committed 4 days ago

06666a02

Newer Older