llvm-project
f6663fef - [Bitstream] Shrink BitCodeAbbrev's inline operand storage

Commit
5 hours ago
[Bitstream] Shrink BitCodeAbbrev's inline operand storage `BitCodeAbbrev` stored its operands in a `SmallVector<BitCodeAbbrevOp, 32>`: 528 bytes of inline storage, a 640-byte malloc block once allocated via `std::make_shared`. Most abbreviations have just a few operands, and a reader keeps every abbreviation of every open block (and the BLOCKINFO ones) alive. That adds up for Clang module files: the dependency scanner keeps many `ASTReader`s with hundreds of PCMs open at once, and each `ModuleFile` holds several cursors with their own abbreviation lists. Counted with temporary instrumentation of `ReadAbbrevRecord` over one scan of a 40-context/400-TU workload against the macOS SDK (`clang-scan-deps -j 14`, clean module cache): 3.88M abbreviations read, up to 39.8k alive at once. Operand counts: | operands | 2 | 3 | 4 | 5 | 6 | 7 | 9 | 10 | 11 | 17 | |-----------|------:|------:|-----:|-----:|----:|----:|----:|----:|----:|----:| | share | 43.6% | 27.0% | 4.9% | 7.4% | 2.5%| 2.5%| 2.5%| 4.9%| 2.5%| 2.5%| | cumulative| 43.6% | 70.6% |75.5% |82.8% |85.3%|87.7%|90.2%|95.1%|97.5%| 100%| LLVM IR bitcode keeps few abbreviations alive (39 at most when running `llvm-dis` on the -O2 -g bitcode of SemaExpr.cpp), so this mostly matters for module files. Average bytes per abbreviation, including the spilled operand array (which `SmallVector` sizes to max(2 * capacity + 1, N)), with macOS malloc size classes (measured with `malloc_size`) and a glibc-like model (16-byte chunks with 8 bytes of overhead): | inline capacity | 32 | 8 | 5 | 4 | 3 | 2 | 0 | |-----------------|----:|----:|----:|----:|----:|----:|----:| | macOS | 640 | 231 | 164 | 156 | 140 | 141 | 113 | | glibc model | 560 | 211 | 163 | 156 | 142 | 148 | 127 | This patch picks an inline capacity of 3, the best option that keeps inline storage on both allocators, and fits 70% of the abbreviations without a second allocation. `ReadAbbrevRecord` now collects the operands in a local buffer and constructs the abbreviation from it (via a new `BitCodeAbbrev(ArrayRef<BitCodeAbbrevOp>)` constructor), so that larger lists are allocated once from their final size instead of growing 3 -> 7 -> 15 -> 31. This doesn't trust the operand count read from the stream for reserving memory. Each operand info always decodes to exactly one `BitCodeAbbrevOp`. A new unit test round-trips abbreviations with 1 to 40 operands. Measured on top of 21197c9e146c with `clang-scan-deps -j 14` on the 40-context/400-TU workload, on two otherwise idle M1 Ultra machines (20 interleaved rounds each, pooled), mean and 95% confidence interval of the round-paired difference: | module cache | peak footprint | scan time | |---|--:|--:| | empty | 312.9 -> 301.5 MB, -11.4 +- 5.0 MB (-3.6%) | -0.4% +- 0.9% (n.s.) | | populated | 226.3 -> 213.3 MB, -12.9 +- 4.1 MB (-5.7%) | median 1.48 -> 1.46 s | (Populated-cache scans occasionally stalled for several seconds with every binary alike, so their scan time is given as the median.) The instrumentation above predicts a direct saving of 39.8k x ~500 B = ~20 MB for clean scans and 20.5k x ~500 B = ~10 MB for incremental ones. The scan output is identical. Bitcode reading throughput is unchanged (8 rounds, paired, all n.s.): `llvm-dis` of a 39 MB -O2 -g module -0.1% +- 2.2%, of a 26 MB one +0.5% +- 3.7%, `llvm-bcanalyzer -dump` of the 39 MB module +0.9% +- 1.3%, of a PCM +0.2% +- 4.5%. Rejected alternatives: - An inline capacity of 8: ~90 more bytes per abbreviation, ~3.6 MB more at the measured peak. - No inline storage: ~1 MB less at the peak on macOS, but two allocations for every abbreviation. - Exact-size trailing storage (one allocation, ~97 B average on macOS): requires replacing the incremental `Add()` interface every writer uses. - Shrinking `BitCodeAbbrevOp` (16 bytes): literals can be 64-bit values.
Author
Committer
Parents
Loading