feat: Distributed data parallel training support (#2464)
* first experiment distributed
* feat: add DistributedUtils (MPI&NCCL working)
* feat: add DistributedUtils (MPI&NCCL working)
* fix: no need for amdgpu now
* chore: cleanup&propose how to use amdgpu
* chore: add preferences for CUDA-awareness
* feat: fix devices for CUDA-awareness
* chore: add tests
* chore: get rid of unnecessary deps
* chore: update NEWS.md
* chore: cleanup env
* chore: update docs
* chore: update docs & cleanup
* chore: update docs & cleanup
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
* Update docs/src/guide/gpu.md
* Update docs/src/guide/gpu.md
* chore: add PR review suggestions
* chore: fix docs
* fix: add runtests.jl
* chore: small docs update
* chore: remove pkgs from deps
---------
Co-authored-by: CarloLucibello <carlo.lucibello@gmail.com>
Co-authored-by: Carlo Lucibello <carlo.lucibello@unibocconi.it>
