cog
994b2f3f - Allow cancelation on async models

Commit
284 days ago
Allow cancelation on async models Fixes #2212. Previously, we had `runner.is_busy()`, which for sync models returned True if a prediction was currently being processed and False otherwise. But for concurrent models, we can be partially busy. The cancel method had a fail-fast check to return if we is_busy() returned False. But with concurrent models, is_busy() has been generalized to mean "is completely full; cannot accept more work". This meant we could only cancel predictions if we were currently processing concurrency.max predictions. As it turns out, this fail-fast check is unnecessary. runner.cancel() checks that the targeted prediction exists and isn't done already. So we can just drop this check.
Author
Committer
Parents
Loading