DeepSpeed
9b4b7206 - fix(rollout): drop fossil do_sample kwarg duplicated by _hf_sampling_kwargs

Commit
7 days ago
fix(rollout): drop fossil do_sample kwarg duplicated by _hf_sampling_kwargs The fallback prefill branch of _continuous_prefill_cache passed do_sample=False explicitly while also unpacking **_hf_sampling_kwargs(sampling), whose return value always contains do_sample -- so every execution of that branch raised "GenerationMixin.generate() got multiple values for keyword argument 'do_sample'". The explicit kwarg is a leftover from the greedy-only era (e53e2dc2f); dropping it lets _hf_sampling_kwargs own the value, which also keeps first-token selection consistent with the static-cache path's _token_selector(sampling). Surfaced by TestAutoTPContinuousRollout::test_matches_generate_on_sharded_llama [False] from #9008 after the 2026-11-13 master merge: it drives the no-cache-class fallback branch that our own tests (static-cache models) never reach. Also collapse a stray blank line in tests/unit/utils/test_static_cache.py flagged by the Formatting check (yapf). Validated locally: yapf repo-clean, pre-commit green on changed files, unit/comm and test_static_cache unchanged vs baseline; the rollout test itself requires the CI transformers environment. Signed-off-by: Ma, Guokai <guokai.ma@gmail.com>
Author
Parents
Loading