perf(rollout): select generation and prefill buckets with a tunable cost model #8786
delock
approved these changes
on 2026-11-11
delock
approved these changes
on 2026-11-13
perf(rollout): select generation and prefill buckets with calibrated …
27eac0c8
feat(rollout): combine bucketed prefill with fixed-slot graph decode
9368450c
fix(rollout): accept aligned fronts with Auto graph decode
27ffac85
fix(rollout): reject sampling configs from custom generation preparers
3b04d7de
fix(rollout): reject any request setting changed by a generation prep…
e6c7f14b
fix(rollout): count mask lengths exactly and reject replaced preparer…
50aaca43
fix(rollout): reject in-place mask edits by custom generation preparers
2d71f2ce
test(rollout): check route-planning profile time with a frozen clock
b4ea3624
test(rollout): inject planning delay through the model preparer
8bb25e5a
fix(rollout): match Transformers generation defaults and padding sema…
b40cac42
fix(rollout): prepare batched configs from their original settings
3e651b9c
Merge origin/master into feature/adaptive-cb-prefill
0b71e5c6
Merge origin/master into feature/adaptive-cb-prefill
9f7c14b5
delock
enabled auto-merge 3 days ago
delock
merged
a7f1e4d0
into master 3 days ago
Assignees
No one assigned
Login to write a write a comment.
Login via GitHub