DeepSpeed
1aaed1a0 - Merge branch 'tohtana/fix_zero_init_patch' of github.com:microsoft/DeepSpeed into tohtana/fix_zero_init_patch

Commit

298 days ago

Merge branch 'tohtana/fix_zero_init_patch' of github.com:microsoft/DeepSpeed into tohtana/fix_zero_init_patch

References

tohtana/fix_zero_init_patch

#6388 - Fix patch for parameter partitioning in zero.Init()

Author

tohtana

tohtana

Parents

Files9

deepspeed
- __init__.py
- comm
  - comm.py
  - torch.py
- runtime
  - config.py
  - engine.py
  - swap_tensor
    - partitioned_param_swapper.py
- sequence
  - cross_entropy.py
- utils
  - groups.py
tests/unit/sequence_parallelism
- test_ulysses.py