feat(comm): one-shot P2P custom all-reduce for TP=2 inference (#8997)
## Summary
Adds a latency-optimized one-shot all-reduce for P2P-capable CUDA pairs
(TP=2), filling the existing `deepspeed.comm.inference_all_reduce` hook
— the same slot the CPU backend fills with shm+AVX. **Split out of
#8602**: this is a generic comm-layer optimization with zero dependency
on segment-KI; any AutoTP TP=2 workload benefits, including plain `HF
generate` whose decode steps all-reduce small hidden-state tensors
through the TP-wrapped linears.
### How it works
- `csrc/inference/custom_ar.cu`: one-shot P2P kernel over CUDA IPC
buffers (2x4MB, sequence-numbered flags)
- `deepspeed/comm/custom_ar.py`: topology gating — world==2, P2P
capable, bounded-spin handshake probe; **symmetric verdict** via
`all_reduce(MIN)` so a unidirectional fabric fault disables the fast
path on both ranks (prevents the peer-spin hang)
- `op_builder/custom_ar.py` (`DS_BUILD_CUSTOM_AR`): built through the
standard op-builder path like every other op (wheel-prebuildable),
initialized next to the shm op in `TorchBackend`, mirroring
`build_shm_op`
- `layers.py`: `RowParallel` routes no-grad small-message decode through
`inference_all_reduce` (falls back to NCCL unchanged); inference mode
uses reduce-scatter+all-gather
- Fallbacks are safe and logged: non-CUDA, world≠2, no P2P, oversized
(>4MB), or build failure → NCCL
## Not a duplicate
Split from #8602 (segment-KI), which remains the umbrella PR. No other
open PR implements a CUDA fast path for the `inference_all_reduce` hook
(CPU shm path is `ShareMemCommBuilder`, untouched here).
## Tests
`tests/unit/comm/test_custom_ar.py` — routing contract (4 tests,
CPU-runnable):
- not-ready before initialize; initialize() no-ops without a process
group
- payload gate is zero when not ready (no message can ever qualify)
Run: `pytest tests/unit/comm/test_custom_ar.py tests/unit/comm/ -q` → 38
passed, 28 skipped on CPU (local, Python 3.11, CPU only).
**On-device validation**: the kernel + handshake path was exercised on
2x GPU (AutoDL, see #8602 commit `4bcc86b82` verification) through the
ZeRO-3→arena→TP2 rollout e2e. A dedicated 2-GPU parity test (custom_ar
vs NCCL all-reduce) is planned as follow-up; reviewer may request
hardware specifics from #8602's logs.
## Compatibility matrix
Validated: CPU (comm unit suite), CUDA TP=2 (via #8602 e2e). Not
validated: XPU/HPU/NPU (code routes through `get_accelerator()` and
falls back; only `can_device_access_peer` is CUDA-specific, marked
`#ignore-cuda`). ZeRO/offload unaffected (inference-path only).
## AI assistance
This PR was prepared with AI assistance (code assembly, builder wiring,
contract tests, review-hardening fixes — symmetric handshake verdict,
`get_accelerator()` routing). The human submitter reviewed every line;
the handshake-hang analysis and the split decision originated from human
review discussion.
---------
Signed-off-by: Ma, Guokai <guokai.ma@intel.com>
Signed-off-by: Guokai Ma <guokai.ma@intel.com>
Signed-off-by: Ma, Guokai <guokai.ma@gmail.com>
Co-authored-by: pengdurice <pengduhit@gmail.com>