DeepSpeed
a855405e - DeepSpeed Ulysses release (#4198)

Commit

1 year ago

DeepSpeed Ulysses release (#4198) Co-authored-by: Masahiro Tanaka <mtanaka@microsoft.com>

References

#4198 - DeepSpeed Ulysses release

Author

samadejacobs

samadejacobs

Parents

Files7

deepspeed
- comm
  - comm.py
  - torch.py
- runtime
  - engine.py
  - zero
    - partition_parameters.py
- sequence
  - __init__.py
  - layer.py
- utils
  - groups.py