feat(rollout): segment-based kernel injection for hybrid-engine greedy decode #8602
delock
force pushed
from
f1f9beb4
to
b67e55c5
54 days ago
delock
force pushed
from
11231a7f
to
d41c9149
53 days ago
delock
changed the title feat(rollout): HybridEngineRollout graph capture + segment-KI for Qwen3.5 hybrid — 67 tok/s (1.14× of vLLM) feat(rollout): segment-based kernel injection for hybrid-engine greedy decode 53 days ago
delock
marked this pull request as ready for review 51 days ago
delock
marked this pull request as draft 31 days ago
delock
marked this pull request as ready for review 28 days ago
perf(rollout): TP2 decode optimizations — GLU combo, custom-AR handsh…
0a6a0c55
fix: CI compliance — arch guards, hook literals, mock-safe guard, scr…
4bcc86b8
delock
force pushed
from
1811118c
to
4bcc86b8
17 days ago
fix: CI compliance — remove torch.distributed import, fix ignore-cuda…
9e7e8dd3
delock
force pushed
from
b0c0c441
to
9e7e8dd3
17 days ago
feat(segki): arena/reshard handshake — segki_refresh drops derived-we…
9cc11f5a
fix: import RolloutRequest in the arena tests
8b811a03
test: ZeRO-3 trainer -> arena-sync -> segKI rollout end-to-end
4e384a20
docs: mark the refresh_weights contract provisional pending final han…
8111716c
fix: gate arena tests' graph capture on real accelerators; CI compliance
bd736d3d
fix: route the arena e2e script through get_accelerator() instead of …
e3a46527
chore: drop dead accelerator graph-capture probes; use DeepSpeed lice…
5f44d619
refactor(comm): adopt the custom_collective from #8997, drop the loca…
11525511
test: reference anchors for the post-original glue kernels
cef3ec90
refactor: drop the compile integration — segKI is hand-written kernels
47d426d9
revert: restore the TP dataloader consistency check in engine.py
d154866c
Merge remote-tracking branch 'origin/master' into gma/qwen_ki_expr
1c7717ed
refactor(interface): all Python call sites route through the portable…
38ee22cd
test: route kernel-reference tests through grouped_linear + fix asser…
7a449232
refactor(reference): kernel_reference aligned to the 7 interface methods
1d3948e6
fix: reference test alignment — all call sites through grouped_linear…
e3563429
chore: drop accidentally committed pre-commit cache dirs
fb0351e4
fix(kernels): decode_attn honors the q row stride
9c24dba1
fix(segment_ki): pack FLA recurrent bypass inputs; gate by parity probe
5712d44f
fix(cache): bridge the HF >=5.19 GDN cache protocol on the static cache
169577e3
sync(custom_collective): align with the validated #8997 implementation
7b3ea179
fix(segment_ki): correct GDN TP layout — slice z/b/a per rank, valida…
a4edcac3
merge: adopt master's _token_selector (#8939), drop _sample_from_logits
2d834e9f
delock
force pushed
from
2b304f40
to
2d834e9f
10 days ago
fix: merge fallout — drop duplicate get_query_offset, unused import
a9a6ac1e
fix(rollout): sample the prefill first token and the b>1 decode steps
4fd31d6e
fix(cache): restore the transformers>=5.20 seq-length protocol on Dee…
39210bf4
test(cache): align seq-length expectations with the restored +1 protocol
300e5b8d
fix(rollout): route graph capture through the accelerator-gated variable
43b624fa
fix(rollout): define the accelerator-gated use_graph_capture; drop is…
5b70a0be
fix(rollout): adopt master's _continuous_next_tokens with token selector
16b6fc67
fix: drop leftover @staticmethod on _continuous_next_tokens
30ce8d77
fix: add module-level logger; defensive num_attention_heads access
d8340393
fix: _continuous_prefill_cache sampling param; drop greedy-only restr…
e53e2dc2
fix: add sampling param to _continuous_prefill; update call sites
6d273de1
fix(rollout): restore master's graph-capture gates lost in the merge
eff04b38
Merge branch 'master' into gma/qwen_ki_expr
e2ad9437
Merge branch 'master' into gma/qwen_ki_expr
2553e95c
Merge branch 'master' into gma/qwen_ki_expr
aed28987
fix(comm): restrict the CPU shm inference all-reduce to world-wide gr…
2d8f4094
Merge remote-tracking branch 'origin/master' into gma/qwen_ki_expr
00353b7b
test: keep GDN in_proj z/b/a replicated in the arena e2e specs
d43f5a2e
fix(rollout): drop fossil do_sample kwarg duplicated by _hf_sampling_…
9b4b7206
Merge master: adopt the #8997 evolution of the custom-AR files
f0001ab9
perf(rollout): amortize the b>1 EOS check like the b=1 path
8b05554f
fix(rollout): MRoPE position shape + visible fallback when installs d…
c77736dd
fix(rollout): check EOS every step in the b>1 decode loop
90175e9a
perf(module_inject): bf16-native silu epilogue in the GLU reference
b5298ed9
Merge master: adopt #8786's adaptive prefill cost model
2603bd3d
delock
force pushed
from
23adb533
to
2603bd3d
3 days ago
fix(rollout): keep use_segki out of the legacy positional config slots
de059c11
tohtana
requested changes
on 2026-11-16
test: drop the scratch-repo sys.path insert from the arena e2e script
7423b144
fix(rollout): repair refresh_weights and wire it into sync_weights
bd69fc79
fix(rollout): release the continuous graph across weight syncs (inter…
9bf48c86
Merge branch 'master' into gma/qwen_ki_expr
01257b71
fix(refresh): reach the backend cat cache on GLU-only (dense) models
ed86967a
fix: yapf style + skip the GLU-cache test without a registered FusedG…
0aeb3db6
delock
force pushed
from
36d8a2ad
to
0aeb3db6
13 hours ago
fix: the GLU-cache skip misread an instantiated NotImplementedBuilder
012bea77
tohtana
approved these changes
on 2026-11-19
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub