Allow sitemap.ts and sitemap/page.tsx to coexist in Turbopack (#89789)
Solves #78609
Today, Turbopack maps `app/sitemap.ts` to an AppPage of `/sitemap/route`, and `app/sitemap/page.tsx` maps to `/sitemap/page`, which leads to a conflict.
This treats sitemap.xml the same way as robots.txt and site.webmanifest, and puts the xml extension in the AppPage, and not later in the path resolution. This creates an AppPage of `/sitemap.xml/route` which no longer conflicts. This is pretty much what webpack was doing.
Because the page segment is now sitemap.xml, I had to update downstream logic for routes that export sitemap / generateSitemap.
The turbopack-utils is needed for dynamic sitemap routes like the example here: https://nextjs.org/docs/app/api-reference/file-conventions/metadata/sitemap#generating-multiple-sitemaps
In that setup, when a request for /product/sitemap/1.xml comes in, the "multiple" matcher matches and definition.page is "/product/sitemap/[__metadata_id__]/route". This becomes the page in the hot reloader. After that, normalizedPageToTurbopackStructureRoute strips [__metadata_id__] and it becomes /products/sitemap, but our updated Rust key is now /products/sitemap.xml/route... so add back the .xml!
My demo app tests four things in parallel:
1) app/sitemap.ts renders at /sitemap.xml
2) app/sitemap/page.tsx renders at /sitemap and the two do not conflict 3) app/product/sitemap.ts works when generating a static sitemap (/product/sitemap.xml) 4) app/product/sitemap.ts works when generating a dynamic sitemapo (/product/sitemap/1.xml) as in the linked docs above
Also I asked Claude if I've fixed all the relevant codepaths, and here's its output:
```
Case 1: Single dynamic sitemap — app/products/sitemap.ts (no generateSitemaps)
- Input page: /products/sitemap.xml/route, ext: .ts
- Strip /route → /products/sitemap.xml
- No [__metadata_id__] → skip
- Doesn't end with /sitemap (ends with /sitemap.xml) → skip
- Add /route → /products/sitemap.xml/route
- Entry key: /products/sitemap.xml/route ✓
Case 2: Multi-dynamic sitemap — app/products/sitemap.ts with generateSitemaps
- Input page: /products/sitemap/[__metadata_id__]/route, ext: .ts
- Strip /route → /products/sitemap/[__metadata_id__]
- Strip [__metadata_id__] → /products/sitemap
- Ends with /sitemap and ext .ts ≠ .xml → add .xml → /products/sitemap.xml
- Add /route → /products/sitemap.xml/route
- Entry key: /products/sitemap.xml/route ✓
Case 3: Static sitemap — app/sitemap.xml
- Input page: /sitemap.xml/route, ext: .xml
- Strip /route → /sitemap.xml
- No [__metadata_id__] → skip
- ext IS .xml → skip .xml addition
- Add /route → /sitemap.xml/route
- Entry key: /sitemap.xml/route ✓
Case 4: Non-sitemap metadata — app/opengraph-image.tsx
- Input page: /opengraph-image/route, ext: .tsx
- Strip /route → /opengraph-image
- No [__metadata_id__] → skip
- Doesn't end with /sitemap → skip
- Add /route → /opengraph-image/route ✓
Case 5: Non-sitemap multi-dynamic — app/opengraph-image.tsx with generateImageMetadata
- Input page: /opengraph-image/[__metadata_id__]/route, ext: .tsx
- Strip /route → /opengraph-image/[__metadata_id__]
- Strip [__metadata_id__] → /opengraph-image
- Doesn't end with /sitemap → skip
- Add /route → /opengraph-image/route ✓
All cases produce the correct entry key.
```
Seems legit. Would still appreciate someone with more Next.js knowledge validating my assumptions