cog
a4b86cd5 - Poison model healthcheck on shutdown

Commit
1 year ago
Poison model healthcheck on shutdown We have a problem in production where a broken model is not correctly shutting down when requested, which means that director comes back up, sees a healthy model (status READY/BUSY) and starts sending it new predictions, even though it's supposed to be shutting down. For now, try and improve the situation by poisoning the model healthcheck on shutdown. This doesn't solve the underlying problem but it should stop us losing more predictions to a known-broken pod.
Author
Parents
Loading