cog

25aa6b9e - feat(Metrics API): custom prediction metrics via current_scope() (#2752)

feat(Metrics API): custom prediction metrics via current_scope() (#2752) * feat(coglet): add Metric IPC variant with accumulation modes and dot-path nesting Add SlotResponse::Metric {name, value, mode} to the slot socket protocol. Metrics flow per-prediction alongside logs/outputs. The Prediction struct merges user metrics with system metrics (predict_time) in terminal responses. Supports three accumulation modes (replace, increment, append) and dot-path key resolution (e.g. 'timing.preprocess' nests into {timing: {preprocess: v}}). * feat(coglet): wire metrics through supervisor and HTTP responses Metrics flow from Prediction → PredictionResult → supervisor PredictionState. Intermediate webhooks include accumulated metrics snapshot; terminal webhooks include all user metrics + predict_time. Sync HTTP responses merge user metrics into the metrics object alongside predict_time. * feat(coglet-python): add Scope and MetricRecorder PyO3 classes Scope is the per-prediction context obtained via current_scope(). MetricRecorder (scope.metrics) enforces type invariants — a key's type cannot change without delete first. Supports dict-style access, accumulation modes (replace/incr/append), and dot-path keys. All validation happens in Rust (PyO3, in-process). Validated metrics are sent to the coglet server via SlotSender IPC. ScopeGuard RAII manages per-prediction lifecycle alongside PredictionLogGuard. * feat(cog): export current_scope() from cog SDK Thin wrapper re-exporting current_scope() from coglet._sdk. Returns None when coglet is not installed (outside container). * chore: fix fmt and lint (clippy, ruff) * test(coglet): add unit tests for Prediction.set_metric Tests cover replace/overwrite/delete, increment (int/float/from-zero), append (create array/wrap non-array), dot-path nesting (deep, multiple leaves, delete leaf, increment), complex values, and build_metrics merging with predict_time override. * chore: regenerate Python type stubs for Scope, MetricRecorder, current_scope * fix(metrics): fix ScopeGuard sender leak, import path, and add integration tests - Fix ScopeGuard::drop to acquire GIL and release Arc<SlotSender>, preventing log-forwarder deadlock that blocked prediction completion - Fix current_scope() import in cog SDK to use attribute access (coglet._sdk.current_scope) since coglet._sdk is a PyO3 submodule - Add webhook harness support for capturing metrics in payloads - Add coglet_metrics.txtar: sync test for replace/incr/append modes, dot-path nesting, dict-style access, predict_time override - Add coglet_metrics_webhook.txtar: async webhook delivery test * docs: document metrics API in prediction and HTTP reference - Add Metrics section to python.md: recording, accumulation modes (replace/incr/append), dot-path keys, type safety, None handling - Add metrics field to HTTP response docs in http.md - Regenerate llms.txt * fix(coglet): export _sdk in __all__ so stubgen preserves it for pyright * fix(coglet): remove _sdk from __all__, auto-generate __init__.pyi, fix pyright - Remove `_sdk` from `__all__` in __init__.py and _impl (Rust). The `import _sdk as _sdk` pattern already signals re-export to type checkers. - Extend stub_gen.rs to also generate `coglet/__init__.pyi`, eliminating the hand-maintained stub and the mypy stubgen dependency. - Drop `uvx --from mypy stubgen` step from generate:stubs mise task. - Fix pyright error by using explicit `from coglet._sdk import current_scope` instead of attribute access through the re-export chain. - Fix typo in pyproject.toml: reportUnneesssaryContains → reportUnnecessaryContains * chore: mise fmt:python:fix * feat(sdk): add self.record_metric() and self.scope to BasePredictor Adds ergonomic metric recording directly on the predictor instance, eliminating the need for current_scope() imports and None checks. - Add record_metric(key, value, mode) method to BasePredictor - Add scope property that returns the current prediction scope - Simplify current_scope() since coglet is a hard dependency - Update docs to use self.record_metric() as the primary API - Fix noxfile wheel selection to pick platform-compatible wheel --------- Signed-off-by: Morgan Fainberg <code@tempusfrang.it>