transformers
[CB] Changes for long generation
#45530
Merged
Overview
Commits
17
Changes
View On GitHub

[CB] Changes for long generation #45530

ArthurZucker merged 17 commits into main from cb-very-long-gen
remi-or
remi-or Fix KV dedup for decode batches
1c076c60
remi-or Fix memory estimation
7eec987f
remi-or Change default
b4b74ff0
remi-or Added write-only fast path
88287d19
remi-or Take both peaks into account
dd00e9bd
remi-or Revert unused config field
1599e249
HuggingFaceDocBuilderDev
remi-or Review 1
7cd08da7
remi-or Fix p1s
86406269
remi-or Fix p2s and p3s that needed it
b853c854
remi-or Added a TODO
f39b68f5
remi-or Fix test, lower max cached graph, add TODO
4f814b2b
remi-or Fix fragmentation with big warmup
28ca9ed3
remi-or Add more space for logits processors
878f469d
remi-or Merge branch 'main' into cb-very-long-gen
5854ad11
remi-or remi-or marked this pull request as ready for review 54 days ago
remi-or remi-or requested a review from ArthurZucker ArthurZucker 54 days ago
ArthurZucker
ArthurZucker approved these changes on 2026-04-23
remi-or Merge branch 'main' into cb-very-long-gen
7bd12b33
remi-or Fix
1695d375
remi-or Merge branch 'main' into cb-very-long-gen
6b45f178
ArthurZucker ArthurZucker merged 07e38311 into main 52 days ago
ArthurZucker ArthurZucker deleted the cb-very-long-gen branch 52 days ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum