jax
a10c5dd3 - [partitioning] Support no-collectives local-write scatter along sharded inserted_window_dims.

Commit
3 days ago
[partitioning] Support no-collectives local-write scatter along sharded inserted_window_dims. Support a 0-collective scatter mode when sharded `inserted_window_dims` are indexed via `scatter_dims_to_operand_dims` (with replicated/compatible indices and updates, and non-`CLIP` mode): each shard shifts `scatter_indices` by `-axis_index * shard_size` in `in_collective` and writes only updates whose shifted indices fall within its local `[0, shard_size)` range, relying on `hlo.ScatterOp` dropping out-of-bounds updates on all other shards. This unblocks sharded-axis point/slice updates (e.g. `arr.at[0, ...].add(..., out_sharding=...)`) as well as `jax.grad` of `top_k`, `approx_top_k`, and `gather` along sharded axes with 0 communication collectives. PiperOrigin-RevId: 1024202392
Author
Parents
Loading