next.js

d365ff80 - refactor static workers to help with parallelization & cache sharing (#68546)

refactor static workers to help with parallelization & cache sharing (#68546) ### What During SSG, the fetch cache isn't shared between renders of different pages within a single export worker. This means that if **Page A** fetches `/foo` and **Page B** also fetches `/foo`, they will not be deduped and instead will be fetched from origin within each worker. Additionally, we currently only prerender 1 path per worker, which doesn't allow us to take advantage of concurrency or smarter groupings. ### Why Currently the export worker is set up to handle indivdual pages. We fire off an exportPage request to all of the known paths and jest-worker handles concurrency & parallelization. ### How This refactors the export worker to accept an array of paths. For this iteration, it doesn't attempt to do any smart grouping of paths (ie ones that share a layout, etc), it just tries to create reasonably sized groups. It will chunk a minimum of 25 pages per worker, and each worker will process `experimental.staticGenerationMaxConcurrency` (defaults to 2, tbd on final numbers here) exports at a time. This lets us set up the `IncrementalCache` inside of the worker before we process pages. As we process pages, the incremental cache will be populated with entries, and subsequent requests will leverage the cache rules therein. This also disables the auto no-cache behavior that we introduced with Next 15 during SSG. Unless a fetch is explicitly opting out of caching, we'll attempt to cache it during SSG so another worker can re-use it. Closes NDX-78 Closes NDX-19