transformers
d50db469 - Generation: only search for eos_token if set (#22875)

Commit

2 years ago

Generation: only search for eos_token if set (#22875) Generation: only check for eos_token if set The check for unfinished_sequences.max(), which is to find sequences that have ended early via eos_token_id, creates a synchronization point even when there is no eos_token, which slows inference down. This change moves the calculation to inside the condition checking for eos_token, so that such slowdown may be removed by disabling this token. Co-authored-by: John Doe <john.doe@example.com>

References

#22875 - Generation: only search for eos_token if set

#27720 - Add common processor tests

#29969 - [SigLIP] Add fast tokenizer

#32831 - [Docs] Update resources

#33111 - [Backbone] Remove out_features everywhere

#33174 - [Zero-shot image classification pipeline] Remove tokenizer_kwargs

#39821 - Support MetaCLIP 2

#59 - Fix attention mask handling in EoMT-DINOv3 converter

#62 - Add initial DEIMv2 model implementation

#65 - Fix RTDetrV2 sine position embedding ordering

Author

xloem

Parents

a438a094

transformers d50db469 - Generation: only search for eos_token if set (#22875)

transformers
d50db469 - Generation: only search for eos_token if set (#22875)