transformers

0f562ff2 - add richer diagnostics to TP/EP distributed test crash handler

add richer diagnostics to TP/EP distributed test crash handler Enhance the SIGABRT / ProcessExitedException debug output so the next CI failure gives enough information to identify exactly which C++ thread called abort() and why: _snapshot_threads (both pre- and post-cleanup): - Add SigBlk/SigCgt signal masks per thread (detects masked SIGABRT) - Add kernel-space call stack via /proc/self/task/TID/stack (much more detail than wchan alone — shows full kernel call chain for gloo runloops) - Add Python thread pthread_t values (CPython ident == pthread_t, lets you directly match faulthandler "Thread 0x..." to a Python thread) - Add current syscall number per thread - Dump /proc/self/maps to allow offline pthread_t → stack-range resolution for C++ threads whose pthread_t won't appear in the Python thread list New _snapshot_process_extras (both pre- and post-cleanup): - faulthandler.dump_traceback(all_threads=True) frozen at snapshot time (separate from the crash-time faulthandler output) - Native C-level call stack via backtrace_symbols_fd() for current thread - PyTorch thread-pool config (get_num_threads / get_num_interop_threads) - Python GC counts and gc.garbage (cyclic refs through C-ext objects can prevent destructor ordering and trigger the abort) - Relevant env vars (GLOO_*, TORCH_DISTRIBUTED_DEBUG, OMP_NUM_THREADS…) - Open file descriptors: surviving socket FDs after destroy_process_group mean gloo did not fully tear down — the most likely root cause here Add a second snapshot call after destroy_process_group since both ranks show destroy_process_group:done in their breadcrumbs before the SIGABRT, meaning the crash occurs during interpreter shutdown / C++ destructor teardown — the post-cleanup snapshot is closest to the actual crash point. All new files are included in the RuntimeError trace block so they appear in the CI raw log without any manual artifact download. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>