microsoft/DeepSpeed

Pull Requests Commits

Create optimizer state for only trainable parameters

tjruwase committed 3 years ago

ba39d7d0

Several fixes for our read-the-docs build (#1579)

jeffra committed 3 years ago

Verified a8a17f23

Enables ZeRO-3 inference (#1514)

jeffra committed 3 years ago

Verified 2332cb31

[CI] transformers@master has been fixed (#1573)

stas00 committed 3 years ago

Verified 74baf5bb

switch bin files to use python3 instead of python (#1185)

jeffra committed 3 years ago

Verified 236890d6

Render docs for pipe.ProcessTopology (#1505)

James Reed committed 3 years ago

Verified fafc827d

Remove hard tensorboardX requirement (#1571)

jeffra committed 3 years ago

Verified a90497ec

[launcher/runner] respect CUDA_VISIBLE_DEVICES for a single node (#960)

stas00 committed 3 years ago

Verified e3c2d7b1

[autotuning] guard tabulate package import (#1569)

jeffra committed 3 years ago

Verified 938449e3

Enforce nccl/rccl alignment of start location of each shard (#1564)

amathews-amd committed 3 years ago

Verified 4a0b1032

bump DSE commit

jeffra committed 3 years ago

4625add6

set hf hash (#1568)

jeffra committed 3 years ago

Verified da7bff40

Fix partial recovery of sparse_tensor_module_names and dynamically check if gradient data is sparse (#1562)

Dipet committed 3 years ago

Verified 4bf4ab7a

Add autotuning news post (#1565)

cli99 committed 3 years ago

Verified bda3d0e6

[build] support cuda-11.5 (#1558)

stas00 committed 3 years ago

Verified fa8d6c0a

Update offload parameter names (#1536)

tjruwase committed 3 years ago

Verified 7567c76c

Autotuning (#1554)

cli99 committed 3 years ago

Verified 9caa74e5

Add documentation for bfloat16 (git commit 648f7bfa5009484b822064d0c28d377da6dd71a0) (#1516)

manuelciosici committed 3 years ago

Verified b7cc7c8e

Fix zinf none swapper (#1550)

tjruwase committed 3 years ago

Verified 488105eb

Add warmup_type arguments in WarmupLR and WarmupDecayLR (#1530)

skpig committed 3 years ago

Verified 76847f42

Fix sparse attention for small block-sizes (#1545)

RezaYazdaniAminabadi committed 3 years ago

Verified 3ed77304

Tensor-Parallelism general support (#1512)

RezaYazdaniAminabadi committed 3 years ago

Verified 9ce00a21

backward compatibility (#1549)

conglongli committed 3 years ago

Verified b16dd943

jeffra committed 3 years ago

fa9d3e84

Fix 1bit extra issue (#1542)

jeffra committed 3 years ago

Verified 2665c8b1

Use cuda tensors for allgather (#1548)

tjruwase committed 3 years ago

Verified bd3ebddf

CPU-Adam: Fix compile Issue (#1537)

RezaYazdaniAminabadi committed 3 years ago

Verified af443f63

Modify inference engine (#1520)

chunyang-wen committed 3 years ago

Verified f0122007

[unit tests] allow unique port for tests

jeffra committed 3 years ago

0af15b98

fstr for multnode_runner (#1532)

chunyang-wen committed 3 years ago

Verified 93c71831

Older