deno
e0523a56 - perf(ext/web): implement base64url encode/decode as simdutf ops (#36398)

Commit
82 days ago
perf(ext/web): implement base64url encode/decode as simdutf ops (#36398) This replaces the JS base64url shim in `ext/web/00_infra.js` (vendored from std/encoding long ago, since rewritten there) with three Rust ops backed by the same simdutf bindings the standard base64 ops use. The `node:buffer` base64url paths get 2.7-24x faster and land at parity with Node 26.5. Same-tree A/B, release-lite builds, macOS aarch64, medians of 5, ns/op: | benchmark | main | this PR | speedup | Node 26.5 | |---|--:|--:|--:|--:| | `buf.toString("base64url")`, 32 B | 106 | 39 | 2.7x | 38 | | `buf.toString("base64url")`, 256 B | 282 | 53 | 5.3x | 56 | | `buf.toString("base64url")`, 1 KiB | 898 | 100 | 9.0x | 130 | | `buf.toString("base64url")`, 64 KiB | 49,541 | 3,353 | 14.8x | 4,768 | | `buf.toString("base64url", 1, 257)` sub-range | 421 | 54 | 7.8x | 52 | | `Buffer.from(s, "base64url")`, 43 ch | 571 | 61 | 9.3x | 57 | | `Buffer.from(s, "base64url")`, 342 ch | 1,113 | 78 | 14.3x | 74 | | `Buffer.from(s, "base64url")`, 1,366 ch | 3,023 | 153 | 19.7x | 137 | | `Buffer.from(s, "base64url")`, 87,382 ch | 162,314 | 7,376 | 22.0x | 6,124 | | `buf.write(s, "base64url")`, 342 ch | 1,078 | 44 | 24.3x | 56 | | dirty input (mixed alphabet), 342 ch | 1,306 | 905 | 1.4x | 135 | | dirty input (junk chars), 342 ch | 1,417 | 972 | 1.5x | 146 | | `hash.digest("base64url")`, sha256 | 501 | 430 | 1.16x | 295 | Untouched paths (std base64, `atob`/`btoa`) stay within noise (±1%). ## Why it was slow Encode ran the native base64 op and then three full-string `replaceAll` passes, plus a `TypedArrayPrototypeSlice` copy for sub-range encodes. Decode ran a `SafeRegExp` validation, a padding fixup, and two more `replaceAll` passes before the native op. Worst of all, `Buffer.from(str, "base64url")` decoded into a fresh `Uint8Array` and then copied the result into the pooled buffer via `blitBuffer`: an extra allocation and a full copy on every call. ## The fix Three ops mirroring the standard base64 op surface, using simdutf's `Url` alphabet (unpadded output, decode accepts padded and unpadded input): - `op_base64url_encode_from_buffer`, used by `base64urlSlice` and `hash.digest("base64url")` - `op_base64url_decode`, used by the `base64UrlToBytes` cleaning path - `op_base64url_decode_into`, used by `base64urlWrite` to decode straight into the target buffer The first commit adds the ops and their tests, nothing calls them yet. The second commit rewires the four call sites and deletes the shim. Three places where the new ops deliberately differ from their standard base64 counterparts, all measured: - `op_base64url_decode_into` has no strict pre-pass (`op_base64_decode_into` tries `Strict` first). simdutf's `Url + Strict` rejects unpadded final chunks, and real-world base64url (JWT parts, `toString` output) is unpadded, so a strict fast path would miss the common case. A unit test pins this. - It reports invalid input as a `-1` sentinel instead of throwing. The caller falls back to the JS cleaning path on invalid input, and materializing an exception across the op boundary costs ~4.4 µs per call. With the throw/catch structure the dirty-input rows above were 3-4x *slower* than main; with the sentinel they beat it. - There is no `#[string]`-returning encode op and no ≤4096 size split like `base64Slice`: `op_base64url_encode_from_buffer` (builds the one-byte V8 string directly) measured faster than the `#[string]` variant at every size (39 vs 51 ns at 32 B), so it is the only encode op and is used unconditionally. ## Behavior None intended. Verified two ways: - A 254-case corpus (padded/unpadded, embedded and oversized padding, whitespace in every position, mixed alphabet, junk characters, `len % 4 == 1` residues, high-Unicode strings) run through `Buffer.from`, `buf.write` at offsets, and `buf.toString` sub-ranges is byte-identical before and after this change, and also byte-identical to Node 26.5.0. - A property test over 100k random buffers (0-4096 bytes): encode output identical to the old shim, every encode/decode round-trip holds. Two notes: - The internal `forgivingBase64UrlEncode`/`forgivingBase64UrlDecode` exports of `00_infra.js` are gone (no in-tree consumers remain). - Dirty base64url input now attempts a native decode before the JS cleaning fallback, same structure as the standard base64 path. Node's cleaning semantics (`base64clean`, std-alphabet acceptance) are unchanged. Why delete instead of re-syncing with std: std rewrote its base64url entirely, so the copy was orphaned drift. #35881 and #36188 handled the previous instances of this. ## Testing - New Rust unit tests: RFC 4648 vectors across all `len % 3` / `len % 4` classes, error inputs, 64 KiB round-trip, the strict-rejects-unpadded pin. - New TS tests in `buffer_test.ts` and `crypto_hash_test.ts`: round-trips past the 8 KiB stack-buffer boundary, sub-range `toString`, dirty-input fallback, truncating writes into small targets, offset validation, SHA-256 digest vectors. - `unit_node::buffer_test`, `unit_node::crypto::crypto_hash_test`, `unit::text_encoding_test` green. `tools/lint.js` and `tools/format.js` clean. - WPT: not applicable, base64url is not web-exposed through these ops. ## Left for follow-ups - The JS cleaning fallback (`base64clean`) stays in JS. It only runs on invalid input now. - The new helpers are not unified with the existing `simdutf_base64_*` helpers, so the base64/atob/btoa hot paths stay byte-for-byte untouched. Trivial to unify if preferred. - The sentinel and no-size-split findings likely apply to the standard base64 ops too. Separate PR if there is interest. I used Claude Code to help investigate and write this change.
Parents
Loading