microsoft/DeepSpeed

Pull Requests Commits

some debug changes.

awan-10 committed 1 year ago

06c7c334

100 percent match between DS (base_mlp + base_attn) and HF.

awan-10 committed 1 year ago

f169851c

Revert "Comment out bias add in attn"

awan-10 committed 1 year ago

1794a144

Comment out bias add in attn

molly-smith committed 1 year ago

25870306

Set attention past_layer to presents every pass

molly-smith committed 1 year ago

1fa46ee1

typo and uncomment mlp bias add

molly-smith committed 1 year ago

dedb3d85

Print more tensors. Add torch syncs to attn.

molly-smith committed 1 year ago

1aed330c

complete attn baseline

molly-smith committed 1 year ago

29415d08

Merge branch 'debug-ds-inf' of github.com:microsoft/DeepSpeed into debug-ds-inf

awan-10 committed 1 year ago

f746e9df

reorder code to cleanup/merge base_mlp in one place.

awan-10 committed 1 year ago

e245219f

Add attention baseline

molly-smith committed 1 year ago

548a3966

attn layer_past print

molly-smith committed 1 year ago

085e469a

add attention prints

molly-smith committed 1 year ago

e82e81cf

add flags to run base mlp and ds mlp.

awan-10 committed 2 years ago

81d9c1aa

add enable/disable debug flag.

awan-10 committed 2 years ago

535342bb

baseline mlp done. matches HF e2e.

awan-10 committed 2 years ago

10540aa0

add several debug prints/instrumentation to do side by side comparison.

awan-10 committed 2 years ago

d0f765b0

awan-10 committed 2 years ago

f66b6a80

rraminen committed 2 years ago

Verified 2fc56841

Clone tensors to avoid torch.save bloat (#3348)

tjruwase committed 2 years ago

Verified 5c3ebd7e

[CPU] Support Intel CPU inference (#3041)

delock committed 2 years ago

Verified 1f72082f

[ROCm] Hip headers fix (#3532)

rraminen committed 2 years ago

Verified 5147b90a

fix typo in comments with deepspeed/ (#3537)

digger-yu committed 2 years ago

Verified c8d3f5eb

Syncing FusedAdam with new Apex features (#3434)

jomayeri committed 2 years ago

Verified 9685eb92

share inflight registry between PartitionedParameterCoordinators (#3462)

HeyangQin committed 2 years ago

Verified 4716b0f7

Fix broadcast error on multi-node training with ZeroStage3 and TensorParallel=2 (#2999)

YizhouZ committed 2 years ago

Verified 9f4a8763

Fix passing offload_optimizer_config instead of offload_param_config to (#3420)

mmhab committed 2 years ago

Verified 5c6da1f0

fix if statement for self.mpu (#3497)

mrwyattii committed 2 years ago

Verified 1824cb55

Doc updates (#3520)

goodship1 committed 2 years ago

Verified 82991df0

fix typo with docs/ (#3523)

digger-yu committed 2 years ago

Verified d73f99d8

Older