DeepSpeed
feat(rollout): align decode fronts and trim dead zones
#8497
Merged

feat(rollout): align decode fronts and trim dead zones #8497

nathon-lee
nathon-lee nathon-lee requested a review from tjruwase tjruwase 68 days ago
nathon-lee nathon-lee requested a review from tohtana tohtana 68 days ago
nathon-lee nathon-lee requested a review from loadams loadams 68 days ago
chatgpt-codex-connector
chatgpt-codex-connector commented on 2026-09-13
delock delock requested a review from delock delock 68 days ago
nathon-lee nathon-lee force pushed from 4c768545 to 095c8838 67 days ago
delock
delock commented on 2026-09-14
delock
delock commented on 2026-09-14
delock
nathon-lee
nathon-lee
nathon-lee nathon-lee force pushed from 2519d70b to 9ddbdac3 61 days ago
nathon-lee feat(rollout): align decode fronts and trim dead zones
c527d26f
nathon-lee fix(rollout): simplify continuous batch admission
fd52dfd7
nathon-lee fix: fix some format err by yapf
55d7edd0
nathon-lee fix(rollout): preserve greedy generation semantics in continuous batc…
7e26f35e
nathon-lee fix(rollout): preserve continuous profiling after rebase
440bc14d
nathon-lee nathon-lee force pushed from 9ddbdac3 to 440bc14d 61 days ago
nathon-lee docs(rollout): clarify decode front alignment semantics
24e96bf1
nathon-lee
nathon-lee nathon-lee marked this pull request as draft 58 days ago
nathon-lee
delock
nathon-lee
delock
nathon-lee
nathon-lee fix(rollout): make cache trimming opt-in Add configurable cache capac…
9eb84bba
nathon-lee nathon-lee marked this pull request as ready for review 57 days ago
nathon-lee nathon-lee marked this pull request as draft 57 days ago
nathon-lee fix(rollout): handle unset repetition penalty
4a314db4
nathon-lee fix(rollout): reclaim dead cache prefixes at capacity
8bd28019
nathon-lee nathon-lee force pushed from faf4eb06 to 8bd28019 55 days ago
nathon-lee nathon-lee marked this pull request as ready for review 55 days ago
nathon-lee
nathon-lee Merge branch 'master' into feat/cb-decode-align-trim
bcda012e
delock
delock approved these changes on 2026-09-28
delock
nathon-lee nathon-lee force pushed from 248bf899 to bcda012e 48 days ago
nathon-lee Merge upstream/master into PR-8497 and resolve inference docs conflict
8fdb53e6
nathon-lee test: add StaticCache per_layer_config compatibility to rollout Cache…
c3954e25
nathon-lee
delock delock enabled auto-merge 47 days ago
delock
delock delock merged a77aeb67 into master 47 days ago
tohtana tohtana added needs-gpu-test
tohtana tohtana removed needs-gpu-test
gpu-test-labeler gpu-test-labeler added skip-gpu-test

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone