PR #41421 Restore cuda graphs to continuous batching

Restore cuda graphs to continuous batching #41421

remi-or merged 18 commits into huggingface:main from remi-or:cb-cg

remi-or force pushed from 7703b505 to 32597cc7 245 days ago

remi-or force pushed from 5513b4ae to b9d2ffdb 245 days ago

ArthurZucker approved these changes on 2025-10-10

remi-or force pushed from 8ee70587 to 0df1abb3 240 days ago

Type hints and small fixes

7ec0fdd9

Remove unusued params

175883dd

Made slice inputs the default

25d0ae1d

ruffed

8f40947e

Updated some var name and moved index slicing

550025b5

Logging arg in example

0529c0f4

Added some padding debug var and reformat out cg

da5abdcc

First working CG, fixe size

9dcdef22

Working flexible CG

8b2043a4

CG are compatible with all implementations

71053408

Fixed CG API

408daa4c

Update example

16ac34ee

Documentation

73bba326

Fix padding tokens in FA

29093f82

Review compliance

2a5ec312

Better doc around weird bug

d68560c9

Style

11f60f28

remi-or force pushed from 0df1abb3 to 11f60f28 240 days ago

Fix for sliding with CG

1443d62e

remi-or merged cf1e9834 into main 240 days ago

Reviewers

ArthurZucker

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

transformers Restore cuda graphs to continuous batching #41421 Merged

Restore cuda graphs to continuous batching #41421

transformers
Restore cuda graphs to continuous batching
#41421

Merged