[Bitstream] Shrink BitCodeAbbrev's inline operand storage
`BitCodeAbbrev` stored its operands in a `SmallVector<BitCodeAbbrevOp, 32>`:
528 bytes of inline storage, a 640-byte malloc block once allocated via
`std::make_shared`. Most abbreviations have just a few operands, and a reader
keeps every abbreviation of every open block (and the BLOCKINFO ones) alive.
That adds up for Clang module files: the dependency scanner keeps many
`ASTReader`s with hundreds of PCMs open at once, and each `ModuleFile` holds
several cursors with their own abbreviation lists.
Counted with temporary instrumentation of `ReadAbbrevRecord` over one scan of
a 40-context/400-TU workload against the macOS SDK (`clang-scan-deps -j 14`,
clean module cache): 3.88M abbreviations read, up to 39.8k alive at once.
Operand counts:
| operands | 2 | 3 | 4 | 5 | 6 | 7 | 9 | 10 | 11 | 17 |
|-----------|------:|------:|-----:|-----:|----:|----:|----:|----:|----:|----:|
| share | 43.6% | 27.0% | 4.9% | 7.4% | 2.5%| 2.5%| 2.5%| 4.9%| 2.5%| 2.5%|
| cumulative| 43.6% | 70.6% |75.5% |82.8% |85.3%|87.7%|90.2%|95.1%|97.5%| 100%|
LLVM IR bitcode keeps few abbreviations alive (39 at most when running
`llvm-dis` on the -O2 -g bitcode of SemaExpr.cpp), so this mostly matters for
module files.
Average bytes per abbreviation, including the spilled operand array (which
`SmallVector` sizes to max(2 * capacity + 1, N)), with macOS malloc size classes
(measured with `malloc_size`) and a glibc-like model (16-byte chunks with 8
bytes of overhead):
| inline capacity | 32 | 8 | 5 | 4 | 3 | 2 | 0 |
|-----------------|----:|----:|----:|----:|----:|----:|----:|
| macOS | 640 | 231 | 164 | 156 | 140 | 141 | 113 |
| glibc model | 560 | 211 | 163 | 156 | 142 | 148 | 127 |
This patch picks an inline capacity of 3, the best option that keeps inline
storage on both allocators, and fits 70% of the abbreviations without a
second allocation. `ReadAbbrevRecord` now collects the operands in a local
buffer and constructs the abbreviation from it (via a new
`BitCodeAbbrev(ArrayRef<BitCodeAbbrevOp>)` constructor), so that larger lists
are allocated once from their final size instead of growing 3 -> 7 -> 15 -> 31.
This doesn't trust the operand count read from the stream for reserving
memory. Each operand info always decodes to exactly one `BitCodeAbbrevOp`.
A new unit test round-trips abbreviations with 1 to 40 operands.
Measured on top of 21197c9e146c with `clang-scan-deps -j 14` on the
40-context/400-TU workload, on two otherwise idle M1 Ultra machines (20
interleaved rounds each, pooled), mean and 95% confidence interval of the
round-paired difference:
| module cache | peak footprint | scan time |
|---|--:|--:|
| empty | 312.9 -> 301.5 MB, -11.4 +- 5.0 MB (-3.6%) | -0.4% +- 0.9% (n.s.) |
| populated | 226.3 -> 213.3 MB, -12.9 +- 4.1 MB (-5.7%) | median 1.48 -> 1.46 s |
(Populated-cache scans occasionally stalled for several seconds with every
binary alike, so their scan time is given as the median.) The instrumentation
above predicts a direct saving of 39.8k x ~500 B = ~20 MB for clean scans and
20.5k x ~500 B = ~10 MB for incremental ones. The scan output is identical.
Bitcode reading throughput is unchanged (8 rounds, paired, all n.s.):
`llvm-dis` of a 39 MB -O2 -g module -0.1% +- 2.2%, of a 26 MB one +0.5% +-
3.7%, `llvm-bcanalyzer -dump` of the 39 MB module +0.9% +- 1.3%, of a PCM
+0.2% +- 4.5%.
Rejected alternatives:
- An inline capacity of 8: ~90 more bytes per abbreviation, ~3.6 MB more at
the measured peak.
- No inline storage: ~1 MB less at the peak on macOS, but two allocations for
every abbreviation.
- Exact-size trailing storage (one allocation, ~97 B average on macOS):
requires replacing the incremental `Add()` interface every writer uses.
- Shrinking `BitCodeAbbrevOp` (16 bytes): literals can be 64-bit values.