xla
70d2e9ea - [SPMD] Support reduce-scatter in manual sharding (#7231)

Commit
1 year ago
[SPMD] Support reduce-scatter in manual sharding (#7231) Summary: This PR is to add experimental support of cc ops in manual sharding zones. This one adds reduce-scatter as the initial step. The key here is to add channel_id, replica_groups, and use_global_device_ids in the lowering. Test Plan: PJRT_DEVICE=TPU XLA_USE_SPMD=1 python test/spmd/test_xla_sharding.py -v -k test_spmd_reduce_scatter
Author
Parents
Loading