llvm-project

abbf1f18 - [Support] Fix Process::PreventCoreFiles() when coredumps are piped

[Support] Fix Process::PreventCoreFiles() when coredumps are piped On many current Linux systems, coredumps are no longer dumped in the CWD but instead piped to a utility such as systemd-coredumpd that stores them in a deterministic location. This can be done by setting the kernel.core_pattern sysctl to start with a '|'. However, when using such a setup the kernel ignores a coredump limit of 0 (since there is no file being written) and we can end up piping many gigabytes of data to systemd-coredumpd which causes the test suite to freeze for a long time. While most piped coredump handlers do respect the crashing processes' RLIMIT_CORE, this is notable not the case for Debian's systemd-coredump due to a local patch that changes sysctl.d/50-coredump.conf to ignore the specified limit and instead use RLIM_INFINITY (https://salsa.debian.org/systemd-team/systemd/-/commit/64599ffe44f0d). Fortunately there is a workaround: the kernel recognizes the magic value of 1 for RLIMIT_CORE to disable coredumps when piping. One byte is also too small to generate any coredump, so it effectively behaves as if we had set the value to zero. The alternative to using RLIMIT_CORE=1 would be to use prctl() with the PR_SET_DUMPABLE flag, however that also prevents ptrace(), so makes it impossible to attach a debugger. See https://github.com/llvm/llvm-project/pull/83701 and https://github.com/llvm/llvm-project/issues/45797 Reviewed By: MaskRay Pull Request: https://github.com/llvm/llvm-project/pull/83703