jax
a0f8a4aa - [Pallas:MGPU] Add an all-reduce mode to the reduce-scatter kernel

Commit

99 days ago

[Pallas:MGPU] Add an all-reduce mode to the reduce-scatter kernel The same strategy we used before (ld_reduce) works, we just need to issue more instructions. PiperOrigin-RevId: 819267594

References

#31381 - Remove old ROCm build code

#31768 - [ROCm] Support lowering through PJRT_Triton_Extension

#32571 - [Pallas:MGPU] Add an all-reduce mode to the reduce-scatter kernel

#33157 - Resolve undefined behavior in bitshift unit test

#579 - Create rocm-test-requirements.txt

#581 - Fix/pallas tests shared memory

#584 - Use plain bazel to test jax, use hermetic rocm dependency

#585 - update a test for checking zero ROCm GPU event

#34135 - [ROCm] update to test if there are GPU events when doing profiling on…

#591 - Adding run_pytest_rocm.sh

#592 - Upstream ci download jax rocm wheels

#34304 - [ROCm] Add ROCm platform support for psend/precv collective operations

#599 - Use xla from rocm-jaxlib-v0.8.0 release branch

#34450 - [ROCm] Upstream ROCm CI Nightly Wheel Testing

#611 - Enable test_comparison_scalar tests for ROCm GPUs

#34470 - Enabled testTridiagonal for ROCm devices.

#34474 - Added support for GESVDJ on ROCm devices

#34494 - Enabled ROCm devices to default to Jacobi SVD on smaller matrices.

#632 - Add script for running bazel tests on ROCm

#634 - Skip tridiagonal solve tests on ROCm

Author

apaszke

Committer

Google-ML-Automation

Parents

4b5d4143

jax a0f8a4aa - [Pallas:MGPU] Add an all-reduce mode to the reduce-scatter kernel

jax
a0f8a4aa - [Pallas:MGPU] Add an all-reduce mode to the reduce-scatter kernel