microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into tohtana/compile-zero

tohtana committed 1 year ago

Verified cb540d35

Add backwards compatibility w/ older versions of diffusers (<0.25.0) (#5083)

lekurile committed 1 year ago

Verified e212845e

Update torch version for nv-torch-latest-cpu (#5086)

loadams committed 1 year ago

Verified e469e7d9

Revert "Update nv-torch-latest-version"

loadams committed 1 year ago

55eb78ee

Update nv-torch-latest-version

loadams committed 1 year ago

889620b0

remove unused global

mrwyattii committed 1 year ago

82f80d12

update DistributedTest to work with torch.compile tests

mrwyattii committed 1 year ago

c2ba829d

Stop tracking backward chain of broadcast in initialization (#5075)

tohtana committed 1 year ago

Verified 5a721de3

Fix verification for ZeRO3 leaf module (#5074)

tohtana committed 1 year ago

Verified f02d7bda

Further refactor deepspeed.moe.utils + deepspeed.moe.layer type hints (#5060)

ringohoffman committed 1 year ago

Verified 9922270f

[doc/1-line change] default stage3_param_persistence_threshold is wrong in the doc (#5073)

ByronHsu committed 1 year ago

Verified 3e6d6069

Make batch size documentation clearer (#5072)

segyges committed 1 year ago

Verified dde64b00

[Zero++ qgZ] Fall back to reduce_scatter if `tensor.numel() % (2 * global_world_size) != 0` (#5056)

ByronHsu committed 1 year ago

Verified 592325ab

Merge branch 'master' into tohtana/compile-zero

tohtana committed 1 year ago

Verified c1e10e5e

enable accelerator check for bf16

tohtana committed 1 year ago

eb27b9d3

add check for bf16

tohtana committed 1 year ago

94cc97a5

increse tolerance in tests

tohtana committed 1 year ago

0b3dae9b

fix exception used in test

tohtana committed 1 year ago

96c8647f

adding hccl to init_distributed function description (#5034)

nelyahu committed 1 year ago

Verified 2eafe41b

Update import for changes to latest diffusers (#5065)

mrwyattii committed 1 year ago

Verified a049370c

load linear layer weight with given dtype (#4044)

polisettyvarma committed 1 year ago

Verified 567f97b2

Optimize grad_norm calculations by reducing device/host dependency (#4974)

nelyahu committed 1 year ago

Verified 61daaa1e

Merge branch 'master' into tohtana/compile-zero

tohtana committed 1 year ago

7131d6e5

Merge branch 'tohtana/z3_moe_bwd' into tohtana/compile-zero

tohtana committed 1 year ago

06379688

Delay reduce-scatter for ZeRO3 leaf modules (#5008)

tohtana committed 1 year ago

Verified 19e0dc39

pass only tensors to z3 hooks to prevent dynamo from displaying errors

tohtana committed 1 year ago

5e3a0707

disable memory_efficient_linear when torch.compile is enabled

tohtana committed 1 year ago

ca419b4d

use no zero + no compile as baseline for tests

tohtana committed 1 year ago

d8c0a14c

[NPU] Change log level to debug (#5051)

CurryRice233 committed 1 year ago

Verified 6de31de7

Fix broken model names in inference CI (#5053)

mrwyattii committed 1 year ago

Verified 449f9ad0

Older