turbo

00e9f66d - perf: Replace regex captures with hand-written parsers in berry lockfile identifiers (#13776)

perf: Replace regex captures with hand-written parsers in berry lockfile identifiers (#13776) ## Summary Replaces the two hot regexes in `crates/turborepo-lockfiles/src/berry/identifiers.rs` (`^(?:@([^/]+?)/)?([^@/]+)$` for `Ident` and `^(?:@([^/]+?)/)?([^@/]+?)(?:@(.+))$` for `Descriptor`/`Locator`) with exact hand-written parsers. On a yarn-berry monorepo these regexes run ~40–50k times per `turbo run` invocation: once per descriptor in every lockfile entry key during `BerryLockfile` construction, once per `resolve_package` call from the transitive-closure DP, and once per dependency inside every `all_dependencies` call. Cold-path regexes (`from_patch_reference`, `patch_file`, `is_patch_builtin`) are unchanged. Public API, error messages, and `Cow::Borrowed` lifetimes are unchanged. - **Author model:** anthropic/claude-fable-5 - **Reviewer model:** openai/gpt-5.6-sol (`gpt_performance_reviewer`) ## Hypothesis Berry identifier parsing is a materially hot path (per-call regex capture extraction and its allocation) on large yarn-berry monorepos; replacing it with allocation-free manual parsing reduces end-to-end `turbo run --dry=json` wall time with zero behavior change. Diagnostics that motivated it: instrumented span profile on cal.com showed `resolve_package` 45.5 ms/11,520 calls, `all_dependencies` 19.3 ms/3,812 calls, `parse_lockfile` 40.4 ms of a ~210 ms run; `perf` on a symbolized build showed `regex_automata::dfa::onepass::DFA::try_search_slots_imp` (8.0%) and `Regex::captures_at` (2.4%) as top leaves of a lockfile-only workload. ## Methodology - Environment: Linux 6.18.40 x86_64, Intel Xeon @ 2.90 GHz, 2 vCPU, 4.2 GB RAM, THP=madvise, rustc 1.97.0-nightly (pinned `nightly-2026-05-22`), hyperfine 1.20.0 (official release, SHA-256 verified). Shared/noisy VM disclosed; mitigated with paired balanced blocks, medians, and bootstrap CIs. - Binaries: both built with `cargo build --profile release-turborepo -p turbo`. Baseline from clean checkout `37be819` (SHA-256 `992fbd03…`, 50,632,544 B); candidate from this diff (SHA-256 `77d54a01…`, 50,630,752 B; −1,792 B). Both preserved outside the checkout; all evidence fingerprints these exact binaries. - Workload: cal.com @ `176037d0afbe572f870a3c702985e7cd83fe6c0c` (yarn berry, 1.5 MB yarn.lock, 3,935 entries), detached and immutable. Command: `turbo run build --dry=json --no-daemon` (daemon disabled; warm page cache; identical preparation both sides). - Recorded phases: clean-checkout `baseline` and same-command `after` (20 runs each, stable binary path), 8 paired AB/BA `comparison` hyperfine blocks (`--warmup 3 --runs 8`, order alternating, 64 samples per binary), and correctness `validation`, all via `run_performance_command` with binary and corpus fingerprints. ## Results (recorded evidence) End-to-end, cal.com dry run (8 balanced blocks, block speedup = 1 − candidate_median/baseline_median): | block | order | baseline median | candidate median | speedup | |---|---|---|---|---| | 1 | AB | 151.9 ms | 138.0 ms | +9.19% | | 2 | BA | 145.3 ms | 149.4 ms | −2.85% | | 3 | AB | 148.1 ms | 143.9 ms | +2.83% | | 4 | BA | 145.7 ms | 145.3 ms | +0.31% | | 5 | AB | 157.3 ms | 149.1 ms | +5.20% | | 6 | BA | 151.4 ms | 144.6 ms | +4.53% | | 7 | AB | 153.4 ms | 144.4 ms | +5.90% | | 8 | BA | 163.3 ms | 142.4 ms | +12.85% | - Mean block speedup **+4.74%** (median +4.87%); both orders improve (AB +5.78%, BA +3.71%); bootstrap (20k resamples) 95% CI of mean block speedup **[+1.59%, +7.91%]**; pooled medians 150.8 → 144.4 ms (**−6.5 ms, −4.28%**, n=64/64). - Recorded baseline vs after phases (same command text, 20 runs): 153.0 ± 8.9 ms → 142.0 ± 6.0 ms. - Earlier independent 12-block set against a bit-different (pre-rustfmt, semantically identical) candidate build agreed: mean +3.30%, both orders positive, 95% CI [+2.34%, +4.23%]. Mechanism corroboration (targeted benchmark; full parse + `all_dependencies` + `resolve_package` sweep over all 3,935 entries ×20 iterations per process; baseline binary built from a pristine worktree of `37be819`, candidate from this diff; identical bench source, outputs identical `keys=3935 deps=4247 resolved=4107`): 6 balanced AB/BA hyperfine blocks, 60 samples per binary, mean block speedup **+11.62%**, both orders positive (AB +12.76%, BA +10.47%), bootstrap 95% CI **[+9.27%, +13.92%]**, pooled medians 676.9 → 594.2 ms (**−12.2%**). Non-regression sanity (recorded): payload (pnpm; berry path not executed) 90.2 ± 3.7 ms vs 88.1 ± 4.6 ms — unchanged within noise. ## Correctness - `cargo test -p turborepo-lockfiles`: **314/314 pass**, including a new exhaustive differential test (all 55,987 strings up to length 6 over `{@, /, a, b, :, \n}` compared against the original regexes, kept in-test as oracles) and realistic/edge cases (Unicode, patch locators, `@a@b/c@1.0.0`, trailing `@`, newline-in-range, empty string). - `cargo clippy -p turborepo-lockfiles --all-targets` and `cargo fmt --check`: clean. - End-to-end equality with both preserved binaries on cal.com: `--dry=json` output byte-equal modulo the random run `id` (115 tasks; all task hashes and global hash identical); `turbo prune @calcom/web` outputs recursively identical, pruned `yarn.lock` byte-identical. ## Review gpt_performance_reviewer (openai/gpt-5.6-sol) **approved** the exact final diff after one round of blocking findings (the initial mechanism benchmark lacked reproducible source, paired blocks, and CIs; it was replaced with the fully provenance-tracked paired benchmark above). Reviewer summary: parsers preserve the original regex semantics with exhaustive differential coverage and unchanged API/error behavior; targeted benchmark and end-to-end paired evidence are consistent with the regex-elimination mechanism; approval limited to the stated cal.com workload claims. ## Limitations - The change only affects yarn-berry repositories; the measured claim is specific to the cal.com workload. Other berry repos are expected to benefit in proportion to lockfile size, but no cross-repository number is claimed. - Measurements were taken on a shared 2-vCPU VM with documented noise; mitigations: balanced AB/BA blocks, medians, bootstrap CIs, and an independent earlier block set that agrees. Co-authored-by: vercel-gh-bot-2[bot] <282331341+vercel-gh-bot-2[bot]@users.noreply.github.com> Co-authored-by: Anthony Shew <anthony.shew@vercel.com>