turborepo
eda81724 - feat: Add experimental `turbo trace` (#14992)

Commit
1 day ago
feat: Add experimental `turbo trace` (#14992) # feat: Add experimental turbo trace ## Description This PR adds `turbo trace`, an experimental command that selects the **test files** a change can affect. `turbo run --affected` and `turbo query affected` select packages and tasks; `turbo trace affected` goes one level further and names the test files. It follows each test's imports and other references through the repository: re-exports, jest, vitest and Next.js configs, tsconfig paths, lockfile changes, scripts, and files read by literal path. It reports a test when a changed file is among that test's transitive inputs. The command is **off unless `TURBO_EXPERIMENTAL_TRACE=1` is set**, and it is hidden from `turbo --help`. - Without the variable, `turbo trace …` parses exactly as it does on `main`, as `turbo run trace …`. That holds for the shim and managed setup too, so a repository task named `trace` keeps working. - Nothing else in `turbo` changes behaviour. The variable is not a task-hash input, and `turbo run` does not touch the new code. ```bash TURBO_EXPERIMENTAL_TRACE=1 turbo trace affected # base: merge base with main → working tree TURBO_EXPERIMENTAL_TRACE=1 turbo trace affected --base origin/main --json TURBO_EXPERIMENTAL_TRACE=1 turbo trace why packages/math/test/add.test.js packages/math/src/add.js TURBO_EXPERIMENTAL_TRACE=1 turbo trace lint ``` ### Design - **Soundness first.** A test is left out only when no changed file can reach it. Whatever the analysis can't resolve (a computed import, an unreadable config, a test it can't place) is reported, not silently dropped: - tests it can't place are listed with the reason `UnknownTest`. Without a jest or vitest config, the runners' default patterns decide what a test is; - tasks whose runner it can't map are listed under `uncoveredTasks`; - `--exit-code` counts both; - an SCM error, or a root turbo.json other than the default, selects every test; - `--strict` also counts the stated assumptions; - `--explain` prints the chain from a test to a changed file. - **Built on turbo's own model.** The command builds a `Run` the way `turbo boundaries` and `turbo query affected` do. It builds the task graph only when `--filter` needs it, as `turbo ls` does. It takes: - the workspace's JS packages, from `PackageGraph`; - the changed files, from `SCM::changed_files`. These are the same files `turbo run --affected` sees, with turbo's base resolution, ref validation and `TURBO_SCM_BASE`/`TURBO_SCM_HEAD`. They are prefetched while the package graph is built (a new `RunBuilder::prefetch_changed_files`, which does not narrow `filtered_pkgs`). A small layer on top adds the status (added, modified, deleted) that the diff rules need. Flags follow `turbo query affected`: `--base`, `--head`, `--exit-code`, plus `--filter`. The JSON uses the same list shape as `affectedTasks`: `{ affectedTests: { items: [{ path, package, task, reason: { __typename … } }], length }, uncoveredTasks }`. - **Storage and caching.** - The graph lives in `.turbo/trace/` and is refreshed against the tree on each run. - Caches keyed by this machine's file timestamps live in the user cache directory, never in the repository, so a `.turbo` restored by `actions/cache` into another checkout is safe. - Everything in `.turbo/trace`, and every cache entry, is treated as untrusted input. File names are checked against their canonical form before any read or delete. Every archive is structurally validated on load, in release builds too. - In CI the graph travels through Turborepo's own cache (`AsyncCache`), so local and remote caching, signing, `--cache-dir` and `--cache` all apply. - The key is the commit SHA, the `turbo` version, the OS and the architecture. - A run looks up HEAD, then the merge base, then up to 20 first-parent ancestors. - A restored graph is refreshed against the tree before use. - **Debug switches.** The analysis has internal kill switches for diagnosis. They are compiled out of release builds, behind the off-by-default cargo feature `debug-knobs` of `turborepo-trace-core`, which lists them. `turbo`'s own integration tests run the shipped configuration, without them. - **Errors, lints and tracing.** - One `thiserror` + `miette::Diagnostic` error type at the library boundary. - Every `unwrap`/`expect` outside tests has a narrow `#[allow]` with its invariant written next to it, and there are no blanket allows. - The entry points and the cache transport have `tracing` spans. ### Layout Ten library crates plus a dev-only test kit, all under `crates/turborepo-trace*`. Only `turborepo-trace` depends on other turbo crates. | Crate | What it does | |---|---| | `turborepo-trace` | command logic for `affected`/`why`/`lint`, and the adapters onto `Run`, `PackageGraph`, `SCM` and the cache | | `turborepo-trace-core` | data model, scheduler, file-system cache, path resolution, partial evaluation | | `turborepo-trace-lang-js` | JS/TS analysis (oxc parser, `oxc_resolver`) | | `turborepo-trace-lang-go`, `-lang-rust` | Go and Rust analysis (cargo features `go` and `rust`, on by default, about 1.1 MB) | | `turborepo-trace-frameworks` | jest, vitest, Next.js and tsconfig adapters | | `turborepo-trace-tasks` | recognizers for package scripts and other task sources | | `turborepo-trace-store` | the sharded graph and incremental refresh | | `turborepo-trace-graph` | the graph query engine | | `turborepo-trace-lockfile` | per-importer lockfile diffs (JS, Go and Cargo lockfiles) | | `turborepo-trace-testkit` | test-only: fixtures and a stub language (`publish = false`) | Outside those crates, the changes are small: - the `Trace` command and its gate (`turborepo-cli`: `args.rs`, `commands/trace.rs`, `shim.rs`); - `RunBuilder::prefetch_changed_files`; - `GitRepo::resolve_base` and `validate_git_ref` made public; - `batch_exists` on the cache multiplexer, and a `cache_dir()` helper in `turborepo-dirs`; - an oxc bump for `turbo-trace` and boundaries; - the docs page `reference/trace.mdx` and the env var row; - a nextest override that runs only the gate test on Windows. ### How to review - **Read line by line, about 4k lines.** The changes outside `crates/turborepo-trace*`, plus `crates/turborepo-trace/src/{api,adapter,transport}.rs` and `crates/turborepo-cli/src/commands/trace.rs`. - **Review by tests and behaviour.** The layer commits that add `turborepo-trace-core` through `-store` import an analysis engine that ran as a standalone tool before this PR. They are covered by about 2,200 tests, by integration tests through `turbo`, and by a corpus comparison: on a few hundred recorded CI changes from two large monorepos, `turbo trace affected` selected the same tests as the standalone tool, plus the test files that jest's default patterns match. In both monorepos, every test that failed in CI for a reason other than flakiness was selected. The commits build and pass in order: 1. `chore: Update oxc crates to 0.156.0`: turbo's existing oxc users (4 files). 2. The five layer commits `feat: Add turborepo-trace-*`: the imported analysis crates. 3. `feat: Add the turborepo-trace library and its turbo adapters`. 4. `feat: Add experimental turbo trace`: the command, gate, storage under `.turbo/trace`, transport through turbo's cache, docs and integration tests. 5. `test:` and `build:` commits: the Windows nextest override, and `opt-level = "s"` for some trace crates in `release-turborepo`. 6. `fix:`, `refactor:`, `perf:` and `chore:` commits that harden the command and fit it to turbo: - untrusted store contents; - managed setup for gate-off `turbo trace`; - jest/vitest default patterns; - changed files from turbo's SCM; - no lint baseline file; - unreachable code removed; - the root turbo.json; - no task graph unless needed; - tracing spans. ### Deliberately duplicated, with follow-ups - **Resolver:** `oxc_resolver` 11 sits beside turbo's `unrs_resolver`, and the selection's resolution semantics and tests depend on it. Follow-up: one resolver, for example by moving `turbo-trace` and boundaries to `oxc_resolver`. - **Lockfile parser:** `turborepo-trace-lockfile` sits beside `turborepo-lockfiles`; ours computes the per-importer dependency diffs that selection depends on. Follow-up: add those diffs to `turborepo-lockfiles`. - **turbo.json reading:** `turbo.json` is read again for the graph's task model. A parity test checks `dependsOn` and `outputs` against `turbo run --dry=json` over 43 fixtures and 134 tasks. Follow-up: take the definitions from `TurboJsonLoader`. - **Import tracers:** `crates/turbo-trace`, the import tracer behind `boundaries` and `query file`, is untouched; its README now says so. Follow-up: move them onto `turborepo-trace-lang-js`. ### Known gaps - An untracked test file hidden by a `.ignore` file counts as a change but is not discovered as a test, because the graph's walk honours `.ignore`. - The task definitions of turbo's native Cargo and uv packages are not read for the graph's task model. - Go and Rust analysis is validated on fixtures and on one monorepo's history, less than JS/TS. A TS-only change can still select some Go tests that read files by computed paths: this over-selects, and never drops a test. - Windows: it compiles, and at run time it exits with "not supported on Windows yet". ### Not in this PR - Test-level selection inside `turbo run test --affected`. The plan: an env var for the skip list, plus an optional digest in `TaskHashable` written only when set, so full runs keep today's hash. - An `affectedTests` GraphQL field and `turbo query affected --tests`. - A `trace` key in `turbo.json`. - Python and shell analysis. - Further follow-ups: - `tracing` spans in place of the engine's own timing; - a context object in place of two process-wide switches; - one glob implementation for task globs; - subcommand telemetry; - structured errors inside the engine; - splitting the largest source files. ### Questions for maintainers 1. The graph is stored in the task cache under its own key. Is a non-task artifact there acceptable (key namespace, analytics, eviction), or should it get a prefix or its own endpoint? 2. Should the JSON use the `{"data": …}` envelope of `turbo query` now, to ease graduation? 3. Is `turbo trace` the right name, next to the global `--trace` flag and `crates/turbo-trace`? 4. Would you take a status-carrying `changed_files`, and caller-supplied untracked files, in `turborepo-scm`? 5. Is the `turbo trace: …` stderr style acceptable, or should it go through `turborepo_ui`? 6. `Run` already builds an untracked-file index in the background. May trace read it? ### Cost - **Size:** the stripped `turbo` binary (`release-turborepo`, Linux x86_64) grows from 50.9 MB to **68.6 MB (+17.7 MB, +35%)**. Compressed with xz it is 17.8 MB. `opt-level = "s"` is set for the trace crates that are off the hot path. - **Build time:** a cold release build of `turbo` is unchanged within noise. - **Lines:** about 245k lines of Rust in `src`, about 70k lines of tests, and about 12 MB of test fixtures. - **`turbo run`:** unchanged with the gate off. `--dry=json` timings on two large monorepos and on a generated 1,500-package fixture are within noise of `main`, and the dry-run JSON is identical. - **`turbo trace affected`, warm:** a no-op, an edit or a `--changed` run takes 0.43–1.0× the time of the same analysis as a standalone binary, not counting `RunBuilder`. Building the `Run` (package graph, lockfile) is a fixed cost on top, the same one `turbo query` pays: about a second on a very large workspace, a few hundred milliseconds on a mid-sized one. - **Cold, and restore through the cache:** cold runs take 0.90–1.04× the time of the standalone binary. A graph restored through turbo's cache takes 0.80–0.95×. - **Peak memory:** about 20–35% above the standalone, roughly `RunBuilder`'s own peak. ## Testing Instructions - `cargo test -p turbo --test trace_test`: 52 integration tests over the `trace_jest`, `trace_vitest`, `trace_polyglot` and `trace_go_climb` fixtures, in the shipped configuration. They cover: - **the gate:** off, unset, `0` and `false`, a byte-identical tree, the `turbo run trace` fallback, and hidden help; - **selection:** changes, adds, deletes, renames and directory changes, a base that has moved on, and jest default test files without a config; - **other languages:** a workspace with vitest, a Go module and a Rust crate that share one data file. A Go, Rust or TS change selects only that language's tests, and a change to the shared file selects the tests of all three that read it; - **output and commands:** `--json` stability, `--exit-code`, `why`, and `lint`, which reads no baseline file; - **the graph's storage:** - restored through a local cache directory into a fresh clone; - `.turbo` copied into a second checkout; - a pull-request run with the GitHub Actions base. - `cargo test -p turbo --test trace_task_definitions_test`: the trace's task model against `turbo run --dry=json`. - `cargo test -p turbo --test managed_run_pin_test`: gate-off `turbo trace` gets the same managed-setup check as `turbo run`, and gate-on `turbo trace` is routed like `turbo query`. - `cargo test -p turborepo-cli`: help snapshots and argument parsing. - `cargo test -p 'turborepo-trace*'`: the analysis crates' tests (2,215). They include crafted `graph.json` names and forged shards with valid checksums. - Manual: ```bash cd turborepo-tests/integration/fixtures/trace_jest git init -q && git add -A && git commit -qm base echo '// edit' >> packages/math/src/add.js TURBO_EXPERIMENTAL_TRACE=1 turbo trace affected --explain ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Anthony Shew <anthony.shew@vercel.com>
Author
Parents
Loading