cog

567b542b - feat: build artifacts API — unified pipeline for images and weights (#2695)

feat: build artifacts API — unified pipeline for images and weights (#2695) * fix: prevent panic from closing already-closed progress channel remote.WriteLayer from go-containerregistry closes the progress channel when done. The previous code had a race condition where it would also try to close the channel, causing a 'close of closed channel' panic. The fix removes the unnecessary select block and simply waits for the progress consumer goroutine to finish draining. * feat(config): add name field to weights configuration Add optional 'name' field to WeightSource struct and JSON schema, allowing users to specify a unique identifier for each weight entry: weights: - name: model-v1 source: ./weights/model-v1.zip target: /weights/model-v1 Includes tests for: - Schema validation with/without name field - YAML parsing with name field - JSON parsing with name field - Rejection of invalid properties (additionalProperties: false) * feat(push): wire up bundle format to use resolver.Push When pushing with COG_OCI_INDEX=1 (bundle format), use resolver.Push which routes to BundlePusher for building and pushing the OCI index with weights. This replaces the direct docker.Push call for bundle format. Adds resolveWeightFilePaths() helper to regenerate file paths from the weights config, since paths aren't stored in the lock file. Note: This wiring is in place but BundlePusher still has a bug with Docker Hub (400 Bad Request) that will be fixed in the artifact abstraction refactor. * feat(model): add artifact abstraction foundation — core interfaces and types Introduce unified artifact model for the build/push pipeline (Phase 1). This is purely additive — new types and interfaces, no behavior changes. - ArtifactType enum, ArtifactSpec and Artifact interfaces - ImageSpec/ImageArtifact and WeightSpec/WeightArtifact concrete types - WeightConfig with versioned schema and media type constants - Source.ArtifactSpecs() to derive specs from cog.yaml - Model.Artifacts field with GetImageArtifact/WeightArtifacts/ArtifactsByType helpers - Resolver.Build() populates Model.Artifacts from built image * feat(model): add Builder interface, WeightBuilder, and ImageBuilder Implement Phase 2 of the artifact abstraction: typed builders that produce artifacts from specs. - Builder interface: Build(ctx, ArtifactSpec) -> (Artifact, error) - WeightBuilder: hashes weight files, creates WeightConfig (v1.0 schema), manages lockfile as build cache (skip rehash on name+size match) - ImageBuilder: wraps Factory.Build() + docker.Inspect() -> ImageArtifact - Resolver.Build() now builds WeightArtifacts for configured weights - Fix WeightFile.Name bug: use config name when set, not filename derivation - Consolidate media types: remove plural MediaTypeWeights* constants, canonical singular MediaTypeWeight* used everywhere * refactor(model): merge Image into ImageArtifact, wire ImageBuilder into Resolver ImageArtifact is now the single type for OCI container images. The Image struct is deleted. Factory.Build() returns *ImageArtifact directly. Resolver.Build() delegates to ImageBuilder for the build+inspect flow. This completes the build-side artifact framework: all artifact construction (image and weight) now goes through the Builder interface. * feat(model): add WeightPusher — push weight artifacts as OCI 1.1 manifests Replace the broken fileBackedLayer approach (Docker Hub 400 Bad Request) with tarball.LayerFromFile from go-containerregistry. WeightPusher takes a WeightArtifact and pushes a proper OCI artifact manifest with config blob (WeightConfig JSON), tarball layer, and artifactType field. Returns a manifest descriptor for index assembly in Phase 4. Phase 3 of the artifact abstraction epic (cog-26r). * refactor(model): extract ImagePusher to own file, add PushArtifact method Move ImagePusher from pusher.go to image_pusher.go. Add PushArtifact(ctx, *ImageArtifact) method that takes an artifact directly instead of *Model, for Phase 4 BundlePusher composition. Existing Push(*Model) delegates to PushArtifact for backwards compatibility. * feat(model): rewrite BundlePusher to use typed pushers and descriptor-based index Replace the monolithic BundlePusher with an orchestrator that delegates to ImagePusher and WeightPusher, then assembles the OCI index from pushed manifest descriptors. This eliminates the fetch-back-from-registry pattern and the broken fileBackedLayer code path. - Add IndexBuilder.BuildFromDescriptors() for descriptor-based index assembly - Add registry.Client.GetDescriptor() (lightweight HEAD request) - Push weights concurrently with cancellation on first error - Remove resolveWeightFilePaths workaround from push.go CLI - BundlePusher now reads Model.Artifacts instead of WeightsManifest+FilePaths * refactor(model): always push as OCI index, remove ImageFormat branching Every model push now produces an OCI Image Index. Models without weights get a single-entry index; models with weights get image + weight entries. - Delete format.go (ModelImageFormat, FormatStandalone, FormatBundle, COG_OCI_INDEX) - Remove Model.ImageFormat and BuildOptions.ImageFormat fields - BundlePusher handles zero weights (image-only index) - Resolver.Push always uses BundlePusher, no pusherFor branching - push.go always calls resolver.Push, no standalone/bundle conditional - IsBundle() now checks WeightArtifacts() instead of ImageFormat * fix: remove unused httpClient after coglog removal cherry-pick * feat(model): add COG_OCI_INDEX gate for OCI index push path * refactor(model): remove deprecated IndexFactory, WeightsManifest, and dead code - Move hashFile() to dedicated hash.go - Remove IndexFactory, WeightsArtifactBuilder, fileBackedLayer (replaced by WeightBuilder + WeightPusher) - Remove WeightsManifest struct and Model.WeightsManifest field (use WeightArtifacts() instead) - Remove ToWeightsManifest() from WeightsLock - Remove backwards-compat WeightsManifest loading from Resolver.Build and modelFromIndex - Keep IndexBuilder.BuildFromDescriptors (used by BundlePusher) - Keep WeightFile struct (used by lockfile, WeightBuilder, CLI) -949 lines removed, 8 added. * refactor(model): merge weight pushers, migrate CLI, delete deprecated code - WeightPusher.Push() now uses WriteLayer for blob (multipart/progress/retry) followed by PushImage for the manifest. Adds WeightPushOptions with ProgressFn and RetryFn callbacks. - CLI 'cog weights build' migrated from WeightsLockGenerator to WeightBuilder. - CLI 'cog weights push' migrated from WeightsPusher to WeightPusher with concurrent push and progress tracking via WeightArtifact. - Deleted weights_pusher.go, weights_lock_generator.go and its test. - Net -407 lines. * feat(model): add weight annotations, OCI index gate, inspect commands - Add name/target annotations to OCI index weight entries (cog-p7p) - Wire COG_OCI_INDEX env var into buildOptionsFromFlags (cog-jft) - Add hidden `cog inspect` command with text/json/raw output (cog-1i1) - Add hidden `cog weights inspect` command for local/remote comparison (cog-kgb) * feat(weights): single combined tag, repo-only validation, ref/layer display, registry v3 fix - Replace two-tag scheme with single :weights-<name>-<12hex> tag per weight - Reject tags/digests in weights push/inspect (repo-only via name.NewRepository) - Show full ref per weight in push output and tag on name line in inspect - Inspect fetches full manifest to show layer digests and sizes - Fix uploadBlobSingle to return updated Location header (registry v3 compat) - Add integration test for weights build -> push -> inspect lifecycle * fix: integration tests for weights+bundle pipeline, inspect improvements, push reliability Production fixes: - PushIndex: use remote.Put instead of remote.WriteIndex (fixes empty index error) - WeightPusher: add WithCompressedCaching to fix DIGEST_INVALID on large uploads - Increase upload chunk size from 25MB to 256MB for fewer HTTP round-trips - Default inspect to PreferRemote so bundle indexes are visible Inspect improvements: - Show digest ref (repo@sha256:...), tag, and media type for indexes - Show layer total size alongside manifest size - Type label: 'Model Bundle (OCI Index)' instead of 'OCI Index (bundle)' - Propagate Digest/MediaType/Size from registry through to inspect output Integration test fixes: - Fix mock-weights media type (vnd.cog.weights -> vnd.cog.weight) and version (1 -> 1.0) - Rewrite oci_bundle_build.txtar with inline weights config - Rewrite oci_bundle_push.txtar to exercise BundlePusher via cog push - Fix weights_build.txtar to use file source with explicit name - Add oci_bundle_inspect.txtar for inspect --remote --json coverage - Update push_test.go for Put-based index push semantics * fix: clean up test mocks and stale fixtures - Replace panic() with errors.New() in resolver_test.go mock methods for consistent error behavior instead of test runner crashes - Remove stale 'source' field from weights_lock_test.go fixture (field was removed from WeightFile struct)