cog
fea08169 - Introduce the concept of "context" to PredictionRequest (#2330)

Commit
1 year ago
Introduce the concept of "context" to PredictionRequest (#2330) This allows a prediction request to include additional per-prediction context alongside the `input`. This will then be made available to the predictor via `current_scope().context`. For now this is limited to `dict[str, str]` but we could expand it later. The primary benefit is to allow prediction scoped configuration to be passed alongside user input without changing the input signature. Usage: ``` POST /predictions { "input": { ... }, "context": { "procedure_source_url": "file:///foo/bar/baz", "replicate_api_token": "r8_foop", }, } ``` Then in the predictor: ``` def predict(): context: dict[str, str] = cog.current_scope().context assert context["foo"] == "bar" ```
Author
Parents
Loading