llvm-project
762498c4 - [SampleProfile] Hint imports for in-module indirect call targets (#243573)

Commit
22 hours ago
[SampleProfile] Hint imports for in-module indirect call targets (#243573) ThinLTO can import a function without bringing along the indirect target that its profile expects to inline. The imported code can then retain an indirect call even though the profile identifies the hot target. Consider a [Godbolt example](https://godbolt.org/z/fYzv9cTeb) that shows the missing import. The `worker` in `driver.cpp` calling `run_batch` in `core.cpp` . `run_batch` indirectly calls `hot_target`, also defined in `core.cpp` . In all three cases below, the profile says to inline `hot_target` into `run_batch`: | Case | Behavior before this patch | |---|---| | The profile also says to inline `run_batch` into `worker`. | While processing `worker`, the sample loader records import hints for both functions, since both are external to `driver.cpp`. Both bodies can be imported. | | The profile keeps `run_batch` separate, and ThinLTO does not otherwise import it. | Neither body needs importing into `driver.cpp`. The `core.cpp` backend already has both bodies and can promote and inline `hot_target` into `run_batch`. | | The profile keeps `run_batch` separate, but ThinLTO selects it for importing through the summary. | `driver.cpp` can receive `run_batch` without `hot_target`. Pre-link deferred indirect-call promotion, and the sample loader skipped the target's import hint because its definition was already in `core.cpp` . | This patch addresses the third case. During pre-link, it adds `hot_target` to `run_batch`'s InlinedGUIDs. Summary generation turns that hint into a critical edge from `run_batch` to `hot_target`, so importing `run_batch` can also bring in its profiled target. Thanks @apolloww for [suggesting the summary](https://github.com/llvm/llvm-project/pull/237445#issuecomment-6939482210). After `run_batch` is imported and inlined into `worker`, the loop still contains an indirect call: ```asm .Lbatch_loop: blr x21 // Indirect call remains here subs w22, w22, #1 add x0, x0, #1 // dispatch adds 1 to the result b.ne .Lbatch_loop ``` With the target imported alongside the caller, the backend has its body available for promotion and inlining. Indirect-call promotion remains deferred to the backend. The change is enabled by default; `-sample-profile-preserve-deferred-inlines=false` restores the previous behavior. This is carved out from #237445 as discussed in https://github.com/llvm/llvm-project/pull/237445#issuecomment-6955648663. Assisted-by: GPT-6 Astra
Author
Parents
Loading