transformers
Fixes for continuous batching
#40828
Merged
Overview
Commits
22
Changes
View On GitHub

Fixes for continuous batching #40828

Cyrilvallez merged 22 commits into huggingface:main from remi-or:cb-fix
remi-or
remi-or remi-or requested a review from ArthurZucker ArthurZucker 94 days ago
remi-or remi-or requested a review from McPatate McPatate 94 days ago
HuggingFaceDocBuilderDev
remi-or Fix for CB attn mask and refactor
2c9c94ea
remi-or Tests for CB (not all passing)
1d4fd516
remi-or Passing tests and a logger fix
f1beeff9
remi-or Fixed the KV metrics that were broken when we moved to hybrid alloc
97a4248d
remi-or Fix circular import and style
c8f82576
remi-or Added tests for FA
e9e04cc6
remi-or Unfolded test to have device expectations
d7694e85
remi-or Fixes for H100
146f4fc2
remi-or more fixes for h100
2acd1d56
remi-or H100 are good
f35ae0c4
remi-or Style
04893a43
remi-or Adding some comments from #40831
f7a44487
remi-or remi-or force pushed from ad4ab44a to f7a44487 93 days ago
remi-or Rename test
0d5cb00b
ArthurZucker
ArthurZucker approved these changes on 2025-09-12
remi-or Avoid 1 letter variables
ed2d6474
remi-or
remi-or Dictonnary is only removed during kwargs
7e2fb212
remi-or Merge branch 'main' into cb-fix
880372a7
remi-or Test for supported sample
17fb99a7
remi-or Fix a unvoluntary slice
1266d4d6
remi-or Fixes for non-sliced inputs and small example improvments
0792dce0
remi-or Slice inputs is more understandabe
49ae0616
remi-or Merge branch 'main' into cb-fix
add2bb8b
remi-or Style
06781835
Cyrilvallez Cyrilvallez merged ef053939 into main 93 days ago
McPatate
McPatate commented on 2025-09-12

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum