patch cancel bug: immediately mark predictions as cancelled (#1798)
* unconditionally mark predictions as cancelled without waiting for cancellation to succeed
while I struggle with #1786, we see queue spikes because not responding to cancellation promptly causes the pod to get restarted. this is a dirty hack to pretend like cancellation works immediately. as soon as we fix the race condition (and possibly any issues with task.cancel() behaving differently from signal handlers), we can drop this.
Signed-off-by: technillogue <technillogue@gmail.com>
* make Mux.write a sync method
this makes the cancel patch cleaner, but might be a small speedup for high-throughput outputs
Signed-off-by: technillogue <technillogue@gmail.com>
* gate this behind a feature flag
---------
Signed-off-by: technillogue <technillogue@gmail.com>
