llvm-project
3e18e029 - [LoongArch] Avoid store forwarding blocks in small copies

Commit
37 days ago
[LoongArch] Avoid store forwarding blocks in small copies Store-to-load forwarding fails when a load is wider than the store that wrote its bytes, for GPR, FPR, LSX and LASX accesses alike. On LA664 such a load takes 14-17 cycles instead of 7-10. Small memcpy/memmove are lowered to ld.d/st.d, fld.d/fst.d, vld/vst or xvld/xvst copies. Add a pass that breaks such a copy into smaller copies when its load is likely to read data from in-flight narrower stores: - A narrower store shortly before the load writes inside the copied bytes through the same base register or frame index: break the copy around the store. - In a loop, a store of the previous iteration of at least 8 bytes may write the same buffer, as in LZ77 back-reference copies: break the copy into 8-byte pieces. The pass runs at -O1 and above and can be disabled with -loongarch-disable-avoid-sfb. rustc-perf bufreader_snappy on a Loongson-3A6000 with rustc's default loongarch64 target features (+f,+d,+lsx,+relax): 282.8M -> 241.0M cycles (-14.8%). This patch refers to X86AvoidStoreForwardingBlocks.
Author
Committer
Parents
Loading