microsoft/DeepSpeed

Pull Requests Commits

stop tracking backward chain of broadcast (z3)

Masahiro Tanaka committed 1 year ago

f6f32023

Remove optimizer step on initialization (#5104)

tohtana committed 1 year ago

Verified 18179807

Add HIP conversion file outputs to .gitignore (#5111)

lekurile committed 1 year ago

Verified 25a02047

Skip Triton import for AMD (#5110)

lekurile committed 1 year ago

Verified d04a8386

Fix alignment of optimizer states when loading (#5105)

tohtana committed 1 year ago

Verified 0a10bd42

Support `exclude_frozen_parameters` for `zero_to_fp32.py` script (#4979)

andstor committed 2 years ago

Verified 2518cc42

47% FastGen speedup for low workload - refactor allocator (#5090)

HeyangQin committed 2 years ago

Verified 3c811c96

Split is_synchronized_device api to multiple apis (#5026)

BacharL committed 2 years ago

Verified 697f945a

Switch hasattr check from compile to compiler (#5096)

loadams committed 2 years ago

Verified 3255569b

[xs] fix ZEROPP convergence test (#5061)

yundai424 committed 2 years ago

Verified 688239e3

optimize clip_grad_norm_ function (#4915)

mmhab committed 2 years ago

Verified 961bc856

[NPU] replace 'cuda' with get_accelerator().device_name() (#5095)

minchao-sun committed 2 years ago

Verified 4f477328

HPU Accelerator: fix supported_dtypes API (#5094)

nelyahu committed 2 years ago

Verified b42a4706

Update nv-accelerate to latest torch (#5040)

loadams committed 2 years ago

Verified ec49222c

Enable torch.compile with ZeRO (Experimental) (#4878)

tohtana committed 2 years ago

Verified c3cfe96b

Add backwards compatibility w/ older versions of diffusers (<0.25.0) (#5083)

lekurile committed 2 years ago

Verified e212845e

Update torch version for nv-torch-latest-cpu (#5086)

loadams committed 2 years ago

Verified e469e7d9

Revert "Update nv-torch-latest-version"

loadams committed 2 years ago

55eb78ee

Update nv-torch-latest-version

loadams committed 2 years ago

889620b0

Stop tracking backward chain of broadcast in initialization (#5075)

tohtana committed 2 years ago

Verified 5a721de3

Fix verification for ZeRO3 leaf module (#5074)

tohtana committed 2 years ago

Verified f02d7bda

Further refactor deepspeed.moe.utils + deepspeed.moe.layer type hints (#5060)

Matthew Hoffman committed 2 years ago

Verified 9922270f

[doc/1-line change] default stage3_param_persistence_threshold is wrong in the doc (#5073)

ByronHsu committed 2 years ago

Verified 3e6d6069

Make batch size documentation clearer (#5072)

segyges committed 2 years ago

Verified dde64b00

[Zero++ qgZ] Fall back to reduce_scatter if `tensor.numel() % (2 * global_world_size) != 0` (#5056)

ByronHsu committed 2 years ago

Verified 592325ab

adding hccl to init_distributed function description (#5034)

nelyahu committed 2 years ago

Verified 2eafe41b

Update import for changes to latest diffusers (#5065)

mrwyattii committed 2 years ago

Verified a049370c

load linear layer weight with given dtype (#4044)

polisettyvarma committed 2 years ago

Verified 567f97b2

Optimize grad_norm calculations by reducing device/host dependency (#4974)

nelyahu committed 2 years ago

Verified 61daaa1e

Delay reduce-scatter for ZeRO3 leaf modules (#5008)

tohtana committed 2 years ago

Verified 19e0dc39

Older