omnibus actual concurrency and major refactor (#1530)
* add concurrency to config
* more descriptive names for predict functions
* don't cancel from signal handler if a loop is running. expose worker busy state to runner
* move handle_event_stream to PredictionEventHandler
* make setup and canceling work
* keep track of multiple runner prediction tasks to make idempotent endpoint return the same result and fix tests somewhat
* drop Runner._result, comments
* move create_event_handler into PredictionEventHandler.__init__
* break out Path.validate into value_to_path and inline get_filename and File.validate
* split out URLPath into BackwardsCompatibleDataURLTempFilePath and URLThatCanBeConvertedToPath with the download part of URLFile inlined
* let's make DataURLTempFilePath also use convert and move value_to_path back to Path.validate
* drop should_cancel
* prediction->request
* split up predict/inner/prediction_ctx into enter_predict/exit_predict/prediction_ctx/inner_async_predict/predict/good_predict as one way to do it. however, exposing all of those for runner predict enter/coro exit still sucks, but this is still an improvement
* bigish change: inline predict_and_handle_errors
* inline make_error_handler into setup
* move runner.setup into runner.Runner.setup
* add concurrency to config in go
* try explicitly using prediction_ctx __enter__ and __exit__
* relax setup argument requirement to str
* glom worker into runner
* add logging message
* fix prediction retry and improve logging
* split out handle_event
* use CURL_CA_BUNDLE for file upload
* dubious upload fix
* skip worker and webhook tests since those were erroring on removed imports. fix or xfail runner tests
* validate prediction response to raise errors, but return the unvalidated output to avoid converting urls to File/Path
* expose concurrency in healthcheck
* mediocre logging that works like print
* COG_DISABLE_CANCEL to ignore cancelations
* COG_CONCURRENCY_OVERRIDE
* add ready probe as an http route
* encode webhooks only after knowing they will be sent, and bail our of upload type checks early for strs
* don't validate outputs
* add AsyncConcatenateIterator
* should_exit is not actually used by http
* format
* codecov
* describe the remaining problems with this PR and add comments about cancelation and validation
* add a test
---------
Signed-off-by: technillogue <technillogue@gmail.com>
Co-authored-by: Mattt <mattt@replicate.com>
