transformers
[Paged-Attention] Handle continuous batching for repetition penalty
#39457

Merged

[Paged-Attention] Handle continuous batching for repetition penalty #39457

ArthurZucker merged 9 commits into main from penalty-cb

Handle continuous batching for repetition penalty

635bc875

kashif requested a review from

ArthurZucker 273 days ago

ArthurZucker approved these changes on 2025-07-17

fix last scores and with token mask creation

84cf493f

add test

e34f1f49

Merge branch 'main' into penalty-cb

0cfcee20

kashif requested a review from

ArthurZucker 271 days ago

Merge branch 'main' into penalty-cb

622e68bf

ArthurZucker commented on 2025-07-21

ArthurZucker approved these changes on 2025-07-22

Update src/transformers/generation/continuous_batching.py

4e9cc2f6

Update src/transformers/generation/logits_process.py

2393d7b1

fix formatting

57aa84b9

kashif commented on 2025-07-22

remove unneeded cast

0909e421

ArthurZucker merged 2936902a into main 267 days ago

ArthurZucker deleted the penalty-cb branch 267 days ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone