transformers
cf1e9834 - Restore cuda graphs to continuous batching (#41421)

Commit
194 days ago
Restore cuda graphs to continuous batching (#41421) * Type hints and small fixes * Remove unusued params * Made slice inputs the default * ruffed * Updated some var name and moved index slicing * Logging arg in example * Added some padding debug var and reformat out cg * First working CG, fixe size * Working flexible CG * CG are compatible with all implementations * Fixed CG API * Update example * Documentation * Fix padding tokens in FA * Review compliance * Better doc around weird bug * Style * Fix for sliding with CG
Author
Parents
Loading