cog

596f1a47 - Overhaul development tooling to use mise as the single source of truth for tasks, with consistent naming conventions and intelligent caching. (#2685)

Overhaul development tooling to use mise as the single source of truth for tasks, with consistent naming conventions and intelligent caching. (#2685) * chore: parallel integration tests Integration tests were essentially serial resulting in slow runs both locally and in CI. This commit resolves this particular pain point by spinning each IT up as it's own invocation of go's `t.Run()`. Additionally, in support of future tests to ensure/enforce isolation any IT with the suffix `_serial` in the file name (e.g. test_serial.txtar) will be run without invoking `t.Parallel()`. * chore: Add RUSTUP_HOME and CARGO_HOME env To eliminate the `undefined` directory due to toolchain use, we now define CARGO_HOME and RUSTUP_HOME if they are not already set. Add ignores to .gitignore * chore: Add rust targets We support non-musl x86_64, aarch64, and darwin aarch64 * chore: restructure mise tasks with consistent naming - Language-based tasks for fmt/lint/test/typecheck: task:go, task:rust, task:python - Component-based tasks for build: build:cog, build:coglet, build:sdk - Aliases: fmt = fmt:check, lint = lint:check (non-destructive defaults) - Separate lint:rust:deny for cargo-deny checks - Remove cog-dataclass references (now merged into cog) * chore: restructure CI with path-based filtering and single gate job - Add changes detection job with native git diff (no external actions) - Add ci-complete gate job for branch protection (single required check) - Consolidate rust.yaml into ci.yaml - Add SUPPORTED_PYTHONS env at workflow level - Split into parallel fmt/lint/test jobs per language - Add Rust cache with save-on-main strategy - Add rust to CodeQL language matrix - Document CI architecture in .github/README.md * chore: simplify Makefile and remove redundant scripts - Rewrite Makefile as shim delegating to mise tasks - Remove redundant script/* files (format, lint, test-*, setup, build-wheels) - Keep script/generate-compat for standalone compatibility matrix generation - Add mise task generate:compat for same functionality - Update AGENTS.md with new mise task documentation * docs: update CONTRIBUTING.md for mise workflow, add Makefile warning - Add deprecation warning to Makefile recommending mise (suppressible with MAKE_NO_WARN=1) - Update CONTRIBUTING.md to use mise commands instead of script/* - Add mise trust to setup instructions - Update test, lint, format, and docs commands to use mise * chore: add source/output caching to mise build tasks Add sources/outputs for cache-eligible tasks: - build:sdk - rebuilds only when python/**/*.py changes - build:coglet:wheel* - rebuilds only when crates/**/*.rs changes - generate:go - rebuilds only when dist/cog-*.whl changes - generate:stubs - rebuilds only when coglet-python source changes - generate:compat - rebuilds only when tools/compatgen changes - docs - rebuilds only when docs/**/*.md changes Use 'mise run <task> --force' to bypass cache. * ci: replace llm-docs cron with CI check - Add docs:llm task to regenerate docs/llms.txt - Add docs:llm:check task to verify llms.txt is up to date (uses temp file diff) - Add check-llm-docs job to CI (runs when docs/ or README.md changes) - Delete llm-docs.yml cron workflow (no more auto-PRs) Developers now run 'mise run docs:llm' when updating docs. * ci: use gotestsum with github-actions format for integration tests - Add ci:test:integration task with --format github-actions for collapsed output - Fix docs:llm task to be portable (macOS compatible) - Regenerate docs/llms.txt * ci: use rust-toolchain and cargo-binstall actions in CI - Add MISE_DISABLE_TOOLS=rust,cargo-binstall to use CI actions instead - Use dtolnay/rust-toolchain for Rust jobs - Use taiki-e/install-action@cargo-binstall for jobs needing cargo tools - Skip mise for fmt:rust and lint:rust (only need rustfmt/clippy from toolchain) - Keep mise for jobs needing cargo:* tools (deny, nextest, maturin, zigbuild) This ensures toolchain is ready before mise installs cargo:* tools, avoiding race conditions when cargo-binstall compiles from source. * ci: run all jobs when workflow files change Changes to .github/workflows/ now trigger all jobs to ensure CI changes are fully tested. * ci: add rust toolchain setup to all jobs using mise All jobs that use mise need rust toolchain and cargo-binstall set up first, otherwise mise fails to install cargo:* tools. * fix: remove RUSTUP_HOME override from mise.toml RUSTUP_HOME being empty in CI caused mise to set it to a project-local path, which broke cargo-binstall since the toolchain was installed at the default ~/.rustup location. Keep CARGO_HOME override for local dev (avoids 'undefined' dir issue). * ci: lint-go needs build-sdk for wheel embedding lint:go depends on generate:go which copies dist/cog-*.whl into the binary. * feat: replace Python stub generator with pyo3-stub-gen - Replace custom scripts/generate_stubs.py with pyo3-stub-gen Rust crate - Add #[gen_stub_pyfunction], #[gen_stub_pyclass], #[gen_stub_pymethods] attributes - Create src/bin/stub_gen.rs for stub generation - Update mise tasks (generate:stubs, stub:check) to use cargo run --bin stub_gen - Replace make commands with mise run in CI (lint-python, test-go, integration) - Update bytes crate to 1.11.1 to fix RUSTSEC-2026-0007 - Add lint ignores for test files (S105, S106, S108, S110, B011, S603, B904, ANN) - Remove unused imports from test files (auto-fix) * feat: add mise.toml to CI triggers, build:all alias, install task, pin Python 3.12 - mise.toml changes now trigger all CI jobs (like workflow changes) - Add build:all alias for build task - Add install task (equivalent to make install) - Pin Python 3.12 in tools - fixes pyo3 abi3-py310 builds on systems with older Python * fix: add missing task dependencies - generate:go depends on build:sdk (needs wheel to embed) - lint:go:fix depends on generate:go (consistency with lint:go) - typecheck:go depends on generate:go (go vet needs generated code) - test:integration depends on build:cog, build:sdk, build:coglet:wheel * update mise.lock * fix: use github.ref for concurrency group on main branch Previously used github.run_id which made each run unique (no queueing). Now push to main uses refs/heads/main as group, so runs queue instead of running in parallel. * fix: force exit integration tests on SIGTERM/SIGINT Without this, go test ignores SIGTERM and keeps running when CI cancels the workflow, wasting runner time. * fix: remove unnecessary dependencies from lint tasks - lint:go no longer depends on generate:go (golangci-lint doesn't need wheel) - lint:go:fix no longer depends on generate:go - Remove typecheck:go (golangci-lint includes govet) - Update typecheck to only run rust and python * fix: remove deprecated govet.check-shadowing config * feat: skip slow integration tests by default for local runs Local: mise run test:integration (skips slow tests with -short) Local full: SHORT=0 mise run test:integration CI: runs full suite (no -short flag) * fix: use pre-built wheel for coglet-python tests, restore lint:go deps - lint:go depends on generate:go (golangci-lint needs wheel to compile) - test:coglet:python uses pre-built wheel from dist/ if available - Replace experimental_monorepo_root with [monorepo].config_roots * refactor: replace tox with nox for Python testing - Add noxfile.py with tests and typecheck sessions - Update mise tasks to use nox instead of tox - Remove tox.ini and tox/tox-uv from dev dependencies - CI uses astral-sh/setup-uv for Python version management - Local dev uses uv backend with auto-download Python versions * perf: use aqua backend for cargo-deny, cargo-insta, cargo-zigbuild Switch from cargo: to aqua: backend for tools that have aqua registry support. This provides: - Faster installs (pre-built binaries, no compilation) - Better security (cosign/SLSA verification enabled by default) - No dependency on Rust toolchain for these tools Tools switched: - cargo-deny -> aqua:EmbarkStudios/cargo-deny - cargo-insta -> aqua:mitsuhiko/insta - cargo-zigbuild -> aqua:rust-cross/cargo-zigbuild Tools remaining on cargo: backend (no aqua support): - cargo-nextest - maturin * fix: use uvx to run nox in CI mise's pipx:nox with uvx backend doesn't work when MISE_DISABLE_TOOLS includes python. Since uv is already available via astral-sh/setup-uv, run nox directly via uvx instead. * fix: use aqua gotestsum for CI integration tests - Add aqua:gotestyourself/gotestsum to tools (binary download, no compilation) - Use exec to run gotestsum directly so SIGTERM from CI cancellation propagates to the test process and terminates it cleanly - Removes go run overhead which didn't forward signals properly * fix: use aqua gotestsum for all test tasks Switch test:go and test:integration from 'go tool gotestsum' to use the aqua-installed gotestsum directly for consistency. * fix: always use venvs in nox sessions Remove CI-specific python=False logic that disabled venv creation. Nox's uv backend handles Python version management and venv creation consistently in both local dev and CI environments. * fix: run nox tests only for matrix python version Pass -p flag to nox so each matrix job only runs tests for its specific Python version instead of all 4 versions. * feat: add nox session for coglet-python tests - Add 'coglet' nox session for coglet-python binding tests - Update mise tasks: test:coglet:python (3.13), test:coglet:python:all - CI uses uvx nox with matrix python version for consistency