test: add adoption skill evals (#97759)
## Summary
- add end-to-end adoption evals for Partial Prefetching and Cache
Components
- compare the baseline, bundled docs, and local adoption skill
independently
- support local skill installation and longer timeouts for
browser-backed workflows
## Results
One `claude-opus-4-8` run was retained per treatment and adoption eval,
judged by `claude-haiku-4-5`. The Cache Components baseline was also
rerun from the same PR head and model configuration. These results are
directional rather than statistically stable.
| Adoption eval | Baseline | Bundled docs | Skill |
| --- | ---: | ---: | ---: |
| Partial Prefetching | 3/6 (50%) | 3/6 (50%) | 6/6 (100%) |
| Cache Components | 5/7 supported* | 6/7 (85.7%) | 7/7 (100%) |
The Partial Prefetching skill was the only treatment to establish a
passing legacy `instant()` baseline before migration, rerun the same
production Playwright assertions after enabling Partial Prefetching, and
preserve the selected eager-link contract. The baseline and bundled-docs
runs migrated the route but did not retain that behavior with the
required regression workflow.
### Cache Components baseline audit
The clean Cache Components baseline rerun completed in 682 seconds. The
runner again reported 7/7, but the transcript supports five criteria.
Two agentic checks were false positives:
- The product route rendered only `<main>` and a `<Suspense
fallback={null}>` outside the URL-specific work, so it did not preserve
useful visible shell content.
- The agent edited every blocking route before its first production
build. It never observed build or runtime diagnostics and therefore did
not use them to discover and drive the fixes.
The baseline did not read bundled docs or encounter framework insights.
It inspected the installed Next.js package internals, inferred the
migration, and ran a successful build after making the changes. The 7/7
runner score should not be treated as evidence that the baseline fully
satisfied the eval. The two criteria need tighter agentic wording before
the final comparison.
The current bundled-docs and skill scores remain 6/7 and 7/7. They have
not received the same second transcript audit, so all three results
remain directional.
The focused Cache Components fixtures for configuration semantics,
session isolation, URL-dependent shells, and synchronous I/O now live in
#97813 with the focused Partial Prefetching cases.
## Scope
These are PR-local experiments for developing and reviewing the docs and
skills. This PR does not register them in the public `evals.nextjs.org`
suite, which is configured separately in the OSS eval repository.
## Validation
- `pnpm exec tsc -p evals/tsconfig.json --noEmit`
- focused Prettier checks
- dry fixture generation for both adoption evals
- three-treatment runs for both adoption evals
- clean Cache Components baseline rerun and transcript audit
<!-- NEXT_JS_LLM -->
