next.js
12f20769 - Add CI for upgrade evals with shared preparation (#99908)

Commit
33 days ago
Add CI for upgrade evals with shared preparation (#99908) Stacked on #99880. ### Why? Upgrade evals need CI feedback with reusable preparation and failure evidence that reviewers can read directly in the job log. ### How? - Run all twelve Codex/Claude trials in parallel through `pnpm eval:upgrade --all` on relevant same-repository PR changes and manual dispatch. Check out the exact PR head to evaluate the reviewed candidate. - Prepare candidate packages once per batch. Reuse pnpm/Turbo caches and expiring credential-free Sandbox tooling snapshots, then share app preparation between agent forks. Save replacement snapshot IDs even after a cache hit or eval failure. - Use repository credentials to pull the existing Vercel project's OIDC token. Run a pinned Vercel CLI only for authentication, without a root CLI dependency or GitHub deployment record. - Fail CI for failed or invalid trials and retain transcripts, delivered apps and grading evidence for 14 days. Print brief case results, then collect failed/invalid verdicts and redacted native case logs at the end with workflow-command parsing disabled. <!-- NEXT_JS_LLM -->
Author
Parents
Loading