transformers
2b2bcb01 - Remove redundant past_key_values reassignment in _static_sample

Commit
1 day ago
Remove redundant past_key_values reassignment in _static_sample StaticCache updates KV tensors in-place via index_copy_(), so the object reference from outputs.past_key_values never changes. Grab it once after prefill instead of reassigning every decode step. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Author
Committer
Parents
Loading