Support multi-source weights and HTTPS weight sources (#3008)
* Support multi-source weights and HTTPS weight sources
Weights can now declare multiple sources that merge into a single target
directory (last-in-wins for path conflicts). This covers the common case
of pulling .pth files from multiple GitHub release URLs into one weights
dir.
Config: source field accepts an object or array via custom unmarshalers.
Lockfile: Source -> Sources[] with ImportedAt moved to entry level.
WeightSpec: URI/Include/Exclude -> Sources[]SourceSpec.
New HTTPSource for https:// and http:// URIs with ETag fingerprinting.
Multi-source merge in resolveInventory with per-file owner tracking.
* Address review feedback on multi-source weights
- Reject HTTP URIs with embedded credentials in NewHTTPSource and
normalizeHTTPURI; the URI is recorded in weights.lock and would
leak via git. Extract parseHTTPURI helper so the rule lives in
one place.
- Treat weak ETags (W/"...") as no ETag โ they don't promise content
identity per RFC 7232 ยง2.3. Document ETag as a cache hint, not a
fingerprint.
- Log HEAD failures and weak-ETag fallbacks at debug level.
- Add MarshalYAML on WeightSourceList for symmetry with MarshalJSON;
without it a single-source weight round-trips through YAML as a
one-element sequence instead of a mapping.
- Add a "source is required" check in validateWeights so users see
an actionable error instead of the schema's "must be a mapping".
- Surface multi-source path collisions at info level โ silent
shadowing is easy to miss in cog.yaml diffs.
- Show "(+N more)" indicator in cog weights status for multi-source
weights so the truncation is visible.
- Render multi-source plan output on multiple lines instead of
joining URIs with commas (URLs can contain commas).
- Drop the unused CombineFingerprints helper and merged.Fingerprint
field; per-source fingerprints are tracked separately and used
wherever drift matters.
* Cover multi-source weight paths with tests
Every existing weight test instantiated a single-element source list,
leaving the multi-source merge, override, and per-source fingerprint
paths exercised only in production. Add tests for:
- merging disjoint files from two sources into one inventory and
resolving overlapping paths last-in-wins (table-driven, since the
two cases share setup shape)
- per-source fingerprint drift (mutating one source leaves the
other's fingerprint stable)
- PlanImport surfacing drift on a specific source index
- YAML and JSON parsing of the array form of source:
- MarshalYAML/MarshalJSON shape contract (mapping for single,
sequence/array for multi) with round-trip through unmarshal
- validateWeights accepting the multi-source form and attributing
pattern errors to the right source index
The new tests live alongside the existing single-source tests they
parallel โ multi-source isn't a separate concept, just a different
length of the source list.
* fix: add fuzz tests and fix config parsing panics (#3010)
Add Go fuzz tests for config parsing (FuzzParseBytes, FuzzFromYAML,
FuzzConfigComplete) and fix two categories of crashes found:
1. version.Matches/Equal/Greater panicked on invalid version strings
like "11.A". Replace MustVersion with graceful NewVersion handling.
2. yaml v4 LoadErrors created infinite recursion when errors.Is
traversed wrapped errors. Use %v instead of %w when wrapping yaml
parse errors to break the self-referential cycle.
Also validate CUDA minor version in validateCudaVersion().
* Test weak ETag fallback to SHA256 in HTTP source
---------
Co-authored-by: Mark Phelps <209477+markphelps@users.noreply.github.com>
