text-generation-inference

a04356fb - Attempt for cleverer auto batch_prefill values (some simplifications). (#2808)

Attempt for cleverer auto batch_prefill values (some simplifications). (#2808) * Attempt for cleverer auto batch_prefill values (some simplifications). * Less flaky tests. * Fixing typo insertion. * Update launcher/src/main.rs Co-authored-by: Daniël de Kok <me@danieldk.eu> * Adding small comment for source of calculation. * Adding L40. * Adding L40s. --------- Co-authored-by: Daniël de Kok <me@danieldk.eu>