perf: lazy-load more modules in the snapshot (#34061)
## Summary
Lazifies a large fraction of the JS code currently baked into the CLI
startup snapshot. End result:
| | Snapshot blob | Delta |
| --- | ---: | ---: |
| Before this stack (main) | ~11.4 MB | — |
| After | **7,331,556 bytes** | **−~3.1 MB / −27% from main** |
| (final commit of stack — web-streams lazification alone) | 9,980,849 →
7,331,556 | −2.65 MB |
Verified with `DENO_LOG_LAZY_LOAD=1 deno run hello.js`: **0 lazy loads
at startup**, in both TTY and pipe stdout modes. A
non-`fetch`/`stream`/`fs.promises`/`node:repl` program no longer pays
parse cost for any of those subtrees.
## What's now lazy
### Web platform (final commit)
The 208 KB `06_streams.js` polyfill and every ext module that pulls it:
- `ReadableStream` / `WritableStream` / `TransformStream` and all their
inner controllers/readers (13 stream classes)
- `Request` / `Response` / `fetch` / `EventSource` (chain through
`22_body.js` → `06_streams.js`)
- `caches` / `CacheStorage` / `Cache`
- `CompressionStream` / `DecompressionStream`
- `node:stream/web`
- `Deno.serve` / `Deno.serveHttp` / `Deno.upgradeWebSocket` /
`Deno.Command` / `Deno.run` / `Deno.spawn*` / `Deno.kill` /
`Deno.openKv`
### Node polyfills (earlier in stack)
- HTTP cluster: `node:http` / `node:http2` / `node:https` /
`node:_http_*` / `node:internal/http*`
- Crypto cluster: `node:crypto` /
`node:internal/crypto/{cipher,hash,...}`
- Streams cluster: `node:zlib`, `node:repl`, `node:internal/repl`,
`node:readline`, `node:readline/promises`
- Process cluster: `node:child_process`, `node:internal/child_process`,
`node:dgram`, `node:cluster`
- TLS cluster: `node:tls`, `node:_tls_common`, `node:_tls_wrap`
- Misc: `node:fs/promises`, `node:assert/strict`,
`node:internal/event_target`, `node:internal/fs/utils`
Kept eager (loading them is on the hot path of every program):
`node:stream`, `node:stream/promises`, `node:net`, `node:tty`,
`node:module`, `node:process`.
## Overview of changes
### Infrastructure (`build(snapshot)` + `refactor(core)`)
- **`DENO_SNAPSHOT_IMPORT_GRAPH=<file>`** env var: dump JSONL of every
esm/lazy-script edge during snapshot build. Used to identify exactly
which scripts are dragging which polyfills into the snapshot.
- **`DENO_LOG_LAZY_LOAD=1`** runtime env var: prints a stderr line each
time a lazy_loaded_esm / lazy_loaded_js entry actually parses at
runtime. Cache hits suppressed.
- **Captured `__bootstrap`** in `01_core.js` so deferred `loadExtScript`
calls still find `core`/`primordials`/`internals` after `99_main.js`
deletes `globalThis.__bootstrap`.
- **Residual `.ts` transpile in `build.rs`**: pre-transpile any
`lazy_loaded_js` / `lazy_loaded_esm` file that wasn't consumed at
snapshot time so the runtime loader receives parseable JS rather than
TypeScript.
- **Lazy-ESM resolve fallback**: in `module_map`, if static-import
resolve fails, fall back to the lazy ESM source list before erroring
(lets `node:_http_*` re-export work without eager registration).
### Bug fixes pulled out of the lazification work
- `fix(ext/node)`: defer `lazyLoadProcess()` to `deprecated()` wrapper
to break the `assert.ts ↔ process.ts` cycle exposed by lazification.
- `fix(core)`: drop the module-map borrow before recursively
re-evaluating a lazy ESM module — the prior code held it across
`module.evaluate(scope)` and panicked on `RefCell::borrow_mut` during
recursive lazy_load_esm.
### Final commit — web-streams chain
- `runtime/js/98_global_scope_shared.js`: converts every streams-pulling
global to `propNonEnumerableLazyLoaded` / wrapper-function form.
- `runtime/js/99_main.js`: stops spreading `denoNs` with `{...denoNs}`
(which invokes every getter); uses `ObjectDefineProperties +
getOwnPropertyDescriptors` instead. Same fix for the unstable-feature
merge loop. Wraps the wasm-streaming callback and defers
`registerDeclarativeServer` to the `addMainModuleHandler` callback.
- `ext/web/13_message_port.js`: drops top-level streams import;
`markNotSerializable` registration moved into `06_streams.js` itself
(inverts the dep so message_port no longer drags streams).
- `ext/node/polyfills/01_require.js`: lazifies `internal/child_process`
(which pulled `40_process.js → 22_body.js`) and `stream/web` (which
pulled `14_compression.js`).
- `ext/node/polyfills/internal/streams/fast-utf8-stream.js`: replaces
`import * as fs from "node:fs"` with `createLazyLoader("node:fs")`. The
static import was re-entering `node:fs`'s evaluating body and
TDZ-trapping on `lazyUtf8Stream().default`.
- `ext/node/polyfills/internal/fs/{handle,promises}.ts`: defers every
top-level `promisify(lazyFs().X)` to first-call wrappers. Same TDZ
cycle: `node:fs`'s `export const promises = mod.promises` line
re-triggers `get promises` while `lazyInternalPromises().default` is
still in TDZ.
## Outcome
| Surface | Improvement |
| --- | --- |
| Snapshot size | 11.4 MB → **7.33 MB** (−3.1 MB / −27%) |
| `deno run empty.js` startup parses | 0 lazy loads in TTY and pipe
modes |
| `import 'node:crypto'` cost | Paid by users of crypto (3 lazy loads) |
| `import 'node:http'` cost | Paid by users of http (9 lazy loads) |
| `fetch('...')` first-call cost | Loads `26_fetch.js` + `22_body.js` +
`06_streams.js` on demand |
Programs that don't touch streams/fetch/http/repl/Deno.serve no longer
pay the parse cost.
## Test plan
- [ ] `cargo test` passes
- [ ] `cargo test --test node_compat` passes (down from 43 → ~38 fails,
the remainder are pre-existing on main: `IO Safety violation` in `fork`
and the v8 weak-handle GC flake in `test-repl-tab-complete-buffer`)
- [ ] `DENO_LOG_LAZY_LOAD=1 deno run empty.js` prints 0 lazy loads (TTY
and pipe)
- [ ] Smokes: `Deno.serve`, `fetch`, `new
ReadableStream/Request/Response`, `structuredClone(new
ReadableStream())` rejection, `fs.promises.readdir/readFile`,
`node:child_process.spawn`, `node:stream/web`
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
