transformers

01846b52 - Wait for the first request in the async continuous batching bootstrap (#48304)

Wait for the first request in the async continuous batching bootstrap (#48304) * Wait for the first request in the async continuous batching bootstrap instead of crashing The async-batching path bootstraps the first batch unconditionally at background-thread start and raises 'Failed to bootstrap the first batch' when the request queue is still empty. Whether a manager works therefore depends on a race between start() and the first add_request; user code that starts the manager and submits from another thread wins or loses it nondeterministically. Wait for the first request with the same stop conditions as the main loop instead. * Trim comment * Reduce the amount of duplicate code * Style --------- Co-authored-by: remi-or <remi.pierre_o@orange.fr> Co-authored-by: Rémi Ouazan <83456801+remi-or@users.noreply.github.com>