[CB] Add a "max_requests_per_batch" parameter (#46434)
* Broaden the safety margin and add it to the config
* Plumbing
* IO take attributes from the CB config
* Cap max req per batch to max tokens per batch
* Update cfg is now a fn
* Finalize to model runner and IO
* Rename to be grammaticaly correct
* Auto review
* Small fixes
* Fix imports and remove debub print
* Fix wrong max value for padding
* Solved an aleady solved TODO
* Add progress bar now that there is less info
* Typos
* Add a warning for persistent + sized manager
* Add TODO
* Fix wrong bound
* Fix logit processors
* Fix eager crash of FA2
* Test an edge case
* Better comment
