DeepSpeed
0fe80cb9 - [AffineIR] Check strided shard destinations without pairwise comparisons (#9039)

Commit
1 day ago
[AffineIR] Check strided shard destinations without pairwise comparisons (#9039) **Rebased on master** (PRs #9010 and #9011 have merged; this branch is rebased on master at `5dc44e8`, so the diff contains only this PR's changes). `ParamAffineMap._validate_destinations` (added in #8922) checks the box-shaped pieces of a column split by comparing every box of a rank with every other one. `validate()` runs whenever a ZeRO, AutoEP or block-gather map is built and before every conversion, and this pass has no budget, so validation grows quadratically with the number of strided pieces per rank. This PR sweeps the boxes with `boxes_disjoint` from #9011 first. Only a rank that fails the sweep takes the pairwise walk, which names the shared shard address, so error messages are unchanged. The second commit moves the remaining hand-written geometry onto `affine_geometry`: `_dest_box` returns a `LogicalBox` built with `coordinate()`, the shared address comes from `intersection()` and `dot()`, and `address_range()` / `overlaps_itself()` move there from `affine.py`. The transfer executor reuses `address_range()` for its buffer-reach check and `dot()` for chunk offsets. No behavior or message changes. `validate()` of 8 ranks with interleaved columns (64 rows; ms, single CPU process): | pieces per rank | before | after | |---|---|---| | 128 | 29 | 4 | | 512 | 441 | 17 | | 2048 | 6654 | 68 | Tests: - Randomized check over 200 seeds: column blocks of row bands tiling a shard, half with one block shifted onto another. `validate()` refuses exactly the maps whose pieces write some shard address twice (enumerated independently in the test), and every refusal names an address that is written twice. The test checks this contract only, so it does not depend on which helper does the sweep. - Mutation checks: making the sweep always report "disjoint" fails 48 of the 200 cases. Forcing every rank through the pairwise walk still passes all 200. - All `tests/unit/checkpoint/test_*affine*` files: 839 passed, 1 skipped on 4× NVIDIA L20 at f6e1f6c. At ac3a329 on CPU (`DS_ACCELERATOR=cpu`, 4 ranks): 842 passed, 5 skipped. ## AI assistance Implementation, tests and this description were drafted with GitHub Copilot CLI and reviewed/validated by the author. --------- Signed-off-by: Jin, Youzhi <youzhi.jin@intel.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Ma, Guokai <guokai.ma@gmail.com>
Author
Parents
Loading