Build the inline Flight script from React's strings instead of re-encoding them (#100142)
React's Flight server on Node writes two kinds of chunks:
- Short rows are UTF-8-encoded into a 4 KB buffer and written as bytes.
- Strings longer than about 1365 characters are [written as JS
strings](https://github.com/facebook/react/blob/b618bbb4/packages/react-server/src/ReactServerStreamConfigNode.js#L61-L72),
leaving the encoding to the consumer.
A row is one line, `id:<json>`, and React [builds it as a single
string](https://github.com/facebook/react/blob/b618bbb4/packages/react-server/src/ReactFlightServer.js#L4793-L4795).
So a row with 1365+ characters of JSON is written as a string too. Pages
with a lot of markup and few Suspense boundaries are mostly such rows.
## Before
`renderToNodeFlightStream` piped React into `new PassThrough()`, a
byte-mode stream. Node encodes every string written to a byte-mode
stream, so each of those strings became a Buffer on the way in. Both
consumers then decoded it: `createFromNodeStream` for SSR, and the
inline-data writer, which did this for each chunk to build the
`__next_f` script:
```js
const decodedString = chunk.toString('utf-8')
htmlInlinedData = htmlEscapeJsonString(JSON.stringify([1, decodedString]))
output.push(Buffer.from(`<script>self.__next_f.push(${htmlInlinedData})</script>`))
```
## After
The stream is object mode, so the strings reach the consumers as
strings. The inline writer escapes them straight into the script. SSR
reads them directly, which React's reader
[supports](https://github.com/facebook/react/blob/b618bbb4/packages/react-client/src/ReactFlightClient.js#L5500-L5508).
Prerender capture, byte counting, Web stream conversion and the HTTP
response still need bytes and encode for themselves.
## Numbers
`sandbox-bench`: 16 fresh VMs per experiment, each running both arms
twice, interleaved. Cells are the 95% range across VMs; bold is p <
0.01.
Each commit against the one before it. `/streaming/heavy` has 240
boundaries, each with a 3 KB string prop and a 1.5 KB text node.
`/streaming/heavy-unicode` (added in this PR) is the same with one
non-Latin-1 character in each string, which makes V8 store them at two
bytes per character.
| Commit | `heavy` median | `heavy-unicode` median | `heavy-unicode`
throughput, 8 clients |
| --- | --- | --- | --- |
| Switch the stream to object mode | no change | −4% to 0% | no change |
| Escape inline strings without decoding | −7% to 0% | **−15% to −11%**
| **+17% to +26%** |
| Encode fragments into one Buffer | −15% to 0% | **−20% to −6%** |
**+8% to +38%** |
| All three | **−16% to −6%** | **−28% to −19%** | **+21% to +42%** |
The inline writer can only skip the decode once it receives strings,
which is what the first commit provides. On its own that commit also
removes the SSR decode and adds the `toWellFormed()` scan, and those
roughly cancel. The next two commits remove the decode and re-encode in
the inline writer.
The whole PR by route. The app routes were measured twice, on new VMs.
| Route | Median latency, 1 client | Throughput, 8 clients |
| --- | --- | --- |
| `/streaming/heavy` | **−16% to −6%** | **+19% to +38%** |
| `/streaming/heavy-unicode` | **−28% to −19%** | **+21% to +42%** |
| `/blog` | **−14% to −7%** | **+11% to +24%** |
| `/blog`, repeated | **−15% to −5%** | **+8% to +23%** |
| `/docs` | **−15% to −9%** | **+8% to +44%** |
| `/docs`, repeated | −14% to −2% | +1% to +20% |
| `/dashboard` | −11% to −1% | **+6% to +22%** |
| `/dashboard`, repeated | no change | **+9% to +32%** |
TTFB moves with the median. `/streaming/chunkstorm` (1 KB strings) and
`/streaming/bulk` (384-byte rows) have no strings over the threshold and
did not move in any commit. `heavy-unicode` checks whether
`toWellFormed()`, which scans two-byte strings, has a measurable cost.
The two-byte route improves more than the one-byte route, because
encoding and decoding two-byte strings is more expensive to begin with,
so whatever the scan costs is small compared to what the other two
commits remove. In CPU profiles, `utf8Write` (the encode) is down 48%,
`Buffer#toString` (the writer's decode) down 94%, `TextDecoder` (the SSR
decode) about 65%. Memory is not recorded by this harness.
<details>
<summary>All cells</summary>
Change with 95% range across the 16 VMs of each experiment. Bold is p <
0.01; "no change" means the range includes zero.
| Route | Median latency, 1 client | TTFB, 1 client | Median latency, 8
clients | Throughput, 8 clients |
| --- | --- | --- | --- | --- |
| `/streaming/heavy` | **−11.0% (−16.3 to −5.7)** | **−15.4% (−19.7 to
−11.1)** | **−17.0% (−25.7 to −8.3)** | **+28.4% (+18.8 to +38.0)** |
| `/streaming/heavy-unicode` | **−23.3% (−27.7 to −18.9)** | **−27.8%
(−32.4 to −23.2)** | **−20.6% (−27.5 to −13.7)** | **+31.4% (+20.6 to
+42.2)** |
| `/blog` | **−10.3% (−13.5 to −7.1)** | **−12.7% (−15.8 to −9.6)** |
**−13.3% (−18.5 to −8.1)** | **+17.8% (+11.2 to +24.4)** |
| `/blog`, repeated | **−10.0% (−14.8 to −5.2)** | **−11.6% (−17.0 to
−6.2)** | −10.7% (−19.2 to −2.2) | **+15.5% (+7.7 to +23.3)** |
| `/docs` | **−11.9% (−15.2 to −8.6)** | **−13.9% (−17.2 to −10.6)** |
no change (−25.2 to +0.4) | **+26.0% (+8.2 to +43.8)** |
| `/docs`, repeated | −8.2% (−14.4 to −2.0) | **−10.3% (−16.7 to −3.9)**
| no change (−14.4 to +7.2) | +10.6% (+1.3 to +19.9) |
| `/dashboard` | −5.5% (−10.5 to −0.5) | −6.9% (−13.5 to −0.3) | −9.3%
(−16.5 to −2.1) | **+13.9% (+6.1 to +21.7)** |
| `/dashboard`, repeated | no change (−8.4 to +2.8) | no change (−12.0
to +3.0) | **−12.2% (−19.7 to −4.7)** | **+20.3% (+8.8 to +31.8)** |
| `/streaming/chunkstorm` | no change (−7.4 to +1.4) | no change (−8.2
to +0.8) | no change (−13.1 to +7.5) | no change (−2.8 to +20.8) |
| `/streaming/bulk` | no change (−6.2 to +2.2) | no change (−6.3 to
+2.3) | no change (−8.0 to +10.2) | no change (−5.9 to +13.5) |
</details>
## Output is unchanged
- **HTML bytes.** Identical between arms on every bench route. A unit
test checks the inline script bytes for mixed string and byte input,
including UTF-8 split across chunks and `</script` split across strings.
- **Lone surrogates.** A JS string can contain half of a surrogate pair.
UTF-8 cannot represent it, so encoding replaces it with U+FFFD. Before,
everything was encoded, so SSR, the inline script and the RSC response
all showed U+FFFD. Now SSR would see the original character while the
client still sees U+FFFD, a hydration mismatch. So the stream runs each
string through `toWellFormed()`, which replaces lone surrogates with
U+FFFD the same way. The e2e test checks all three agree and that
hydration logs no mismatch.
- **`res.write`.** Custom servers still receive Buffers; the response
pipe converts strings before writing. An e2e with a custom server checks
this in dev and production, with and without cache components.
<details>
<summary>Verification</summary>
- Unit: `test/unit/inline-flight-data.test.ts`. Exact inline script
bytes for mixed string and byte input at three sizes; UTF-8 split across
adjacent byte chunks; byte views with a nonzero offset; `</script` split
across string chunks.
- E2E `test/e2e/app-dir/flight-native-chunks`: `test-start-turbo`,
`test-dev-turbo`, `test-start-webpack`.
- E2E `test/e2e/app-dir/custom-server-response-chunks`: same three, plus
start and dev with the cache-components env. Rebuilding without the
response-boundary conversion makes its RSC case fail with `string` and
`uint8array` chunks, so the test does see Next's writes. Compression is
off in the fixture; with it on, the server only sees gzip's output.
- `test/e2e/app-dir/rsc-basic` under `test-start-turbo`.
- `pnpm --filter=next types`, prettier, eslint.
Bench cells: `pr100142-stress` (streaming routes; base is the
bench-route commit, cand is head), `pr100142-realistic` and
`pr100142-realistic-confirm` (`/blog,/dashboard,/docs`, same arms),
`pr100142-buffer` (last commit against its parent, streaming routes).
</details>
next.js
a831acd9 - Build the inline Flight script from React's strings instead of re-encoding them (#100142)
