feat: build-time schema generation in Go (tree-sitter) (#2782)
* feat: add static Python schema parser using tree-sitter
Implement pkg/schema/ package for build-time extraction of Cog predictor
signatures via tree-sitter (smacker/go-tree-sitter, CGo). Replaces runtime
Python-based schema generation with static analysis.
- pkg/schema/types.go: core type system (OrderedMap, PrimitiveType, FieldType,
InputField, OutputType, TypeAnnotation, etc.)
- pkg/schema/errors.go: typed SchemaError with error kinds
- pkg/schema/python/parser.go: tree-sitter AST walker that extracts imports,
module-scope constants, BaseModel subclasses, InputRegistry patterns,
function signatures, Input() metadata, and type annotations
- pkg/schema/python/parser_test.go: 50 tests covering basic predictors,
Input() constraints, choices (literal, module var, dict keys/values, concat),
optional/list/iterator/BaseModel outputs, train mode, InputRegistry
(attribute + method), default_factory hard error, and error cases
* feat: add OpenAPI 3.0.2 schema generation from PredictorInfo
Port schema.rs to Go: GenerateOpenAPISchema() produces a complete
OpenAPI spec from parsed predictor info. Includes input schema with
constraints, choices/enums, output types (single/list/iterator/concat/
object), fixed components (request/response/status/validation), and
post-processing (removeTitleNextToRef, fixNullableAnyOf).
Also adds OutputType.JSONType() to types.go, fixes pre-existing lint
issues in parser.go and errors.go.
* feat: add schema Generate() public API with COG_OPENAPI_SCHEMA support
Generate() reads a predict ref (e.g. predict.py:Predictor), loads the
source file, parses it, and produces OpenAPI JSON. Uses a Parser function
type to avoid import cycle between schema and schema/python.
COG_OPENAPI_SCHEMA env var allows bringing a pre-built schema file,
skipping all parsing and generation.
* feat: wire static schema gen into build pipeline and add cogEnvVars
- Pre-build static schema gen (Go tree-sitter) for SDK >= 0.17.0
- Post-build legacy schema gen (container) for SDK < 0.17.0
- Add cogEnvVars() to Dockerfile: COG_PREDICT_TYPE_STUB, COG_TRAIN_TYPE_STUB, COG_MAX_CONCURRENCY
- Add isLegacySDKVersion() and conditional coglet install (SDK dep handles it when unpinned)
- Add GenerateCombined() and MergeSchemas() for predict+train schemas
- Export BaseVersionRe and add DetectLocalSDKVersion() to wheels package
- Update test expectations for new ENV lines and coglet install behavior
* chore: fix clippy manual_str_repeat warning in protocol.rs
* feat: load bundled schema from disk, remove runtime Python schema gen
- Replace handler.schema() with load_bundled_schema() reading .cog/openapi_schema.json
- Remove schema() from PredictHandler trait (coglet-core)
- Remove schema() impl from PythonPredictHandler (worker_bridge.rs)
- Remove schema() method from PythonPredictor (predictor.rs)
- Read COG_MAX_CONCURRENCY from env instead of importing cog.config
- Update SDK detection to check cog.BasePredictor instead of cog._adt
* chore: enable CGo for go-tree-sitter with zig cross-compilation
- CGO_ENABLED=1 globally (goreleaser, mise.toml, CI env)
- goreleaser overrides: zig cc for linux/amd64 and linux/arm64
- darwin targets use native clang (zig lacks macOS SDK stubs)
- CI: mlugg/setup-zig@v2 in build-cog and release-dry-run jobs
* chore: update Cargo.lock for 0.17.0-rc.1
* test: add ITs for static/legacy schema gen and parser edge cases
- legacy_sdk_schema IT: verifies SDK < 0.17.0 falls back to runtime schema gen
- static_schema_gen IT: verifies Go tree-sitter schema in Docker label end-to-end
- Parser tests: falsy defaults (False, 0, 0.0, ""), no-input predictor,
async iterators, typing.List, typing.Union[X, None], list[Path], all-optional
- Support typing.Union[X, None] via comma-aware generic parsing
- Compact JSON for schema output (no wasted bytes in Docker labels)
* feat: add input validation at HTTP edge using bundled schema
Validates prediction and training inputs against the OpenAPI schema at the
Rust HTTP layer before dispatching to the Python worker. This catches missing
required fields and unknown fields early with pydantic-compatible error
responses (422 with detail array).
- Add InputValidator with jsonschema validation, inlining, and
additionalProperties enforcement
- Wire validators into PredictionService (separate Input/TrainingInput)
- Validate in create_prediction_with_id before slot acquisition
- Rewrite training routes with proper idempotency handling (ID mismatch
check, existing state return) instead of delegating to prediction routes
- Add 7 unit tests for InputValidator, 2 for training idempotency
* chore: allow MIT-0 license for borrow-or-share (jsonschema transitive dep)
* fix: resolve allOf/$ref wrappers in CLI input type coercion
The CLI sends all -i values as strings and relies on the schema to coerce
them to the correct types (integer, number, etc.). The static schema gen
emits allOf:[{$ref: ...}] for enum/choices fields, where the referenced
schema carries the concrete type but the wrapper does not. Without resolving
these wrappers, the CLI sends string '42' for integer fields and '3' for
integer enum choices, which the edge validator correctly rejects.
Also adds NewInputsForMode() to look up TrainingInput schema for train mode,
with fallback to Input for legacy schemas.
