deepspeedai/DeepSpeed

Pull Requests Commits

remove fwd stack only once

hwchen2017 committed 1 day ago

687b7f57

Fix memory leak in zero optimizer

hwchen2017 committed 1 day ago

8bb204ab

Fix routed-expert gradient clipping for AutoEP + AutoTP folding (#8823)

0z5a committed 1 day ago

Verified a7ebdf96

ZeRO-3 NVMe offload: fix "param already assigned swap buffer" in the parameter prefetch (#9014)

xylian86 committed 1 day ago

Verified 6c7a6e0e

fix test_multiple_request_boundaries_before_step (#9028)

stas00 committed 1 day ago

Verified af519898

Keep fp32 CPU optimizer states in fp32 when loading a state_dict (#8974)

xylian86 committed 1 day ago

Verified b43c383f

DeepNVMe: fix silent I/O failures on misaligned buffers (#8985)

xylian86 committed 1 day ago

Verified 1b022dcd

CI: stop pip check from failing the AWS fallback on deepspeed-kernels (#9020)

pengdurice committed 2 days ago

Verified 21e7ba48

AutoEP: read and write the transformers output-capture registry as a dict (#9029)

pengdurice committed 2 days ago

Verified e550bedb

Restore fp32 masters for ZeRO-3 and the ZeRO-1 BF16 optimizer (#8892)

sfc-gh-truwase committed 2 days ago

Verified 8af59b2e

AutoEP activation tests: skip the Triton grouped-GEMM path when Triton cannot run (#8996)

GuCarpenter committed 3 days ago

Verified 6d64e1b9

fix(rollout): pass a pre-built 4-D mask to the static graph decode forward (#9004)

anyadontfly committed 3 days ago

Verified 8b7422e2

Run the v1 multi-rank CPU suite on pull requests (#8944)

delock committed 3 days ago

Verified 29c3652b

AutoEP + AutoTP folding: allow CPU optimizer offload under ZeRO-3 (#8829)

0z5a committed 3 days ago

Verified f47dc3e3

ZeRO-3: reuse the weight partition as the master copy when it already has the master dtype (#8842)

pengdurice committed 3 days ago

Verified ed827399

[Hybrid Engine] Require an explicit accelerator opt-in for graph capture (#8980)

VenusTZZ committed 4 days ago

Verified b62afa82

Check that converting through an affine map writes every element once (#8975)

Achyuthan-S committed 4 days ago

Verified 7e58feb6

Clear stale offload scratch before restoring low-precision gradients (#8728)

adenzhou1350 committed 4 days ago

Verified 0d0aa30b

Fix AutoEP storage-map transfer test for CPU tensor views (#9012)

jinyouzhi committed 4 days ago

Verified 67c83616

Support Transformers 5.20 cache query offsets in DeepSpeedStaticCache (#8852)

nathon-lee committed 4 days ago

Verified a23268d0

Fix bug in Adagrad optimizer initialization (#8976)

hwchen2017 committed 4 days ago

Verified 735c4754

Count each expert once in the fp32 gradient-clipping norm (#8478)

alanhuangyoo committed 4 days ago

Verified 31e6abca

Fix torch-latest workflow unit failures (#8950)

sfc-gh-truwase committed 4 days ago

Verified a38957c1

[AutoTP] Support frozen AutoTP models and manage AutoTP mode per model (#8851)

jinyouzhi committed 5 days ago

Verified d8241a9b

XPU: register pinned host memory via Level Zero ext_memmap (#8887)

delock committed 5 days ago

Verified 36b2ab3b

[checkpoint] Execute affine transfer plans with bounded peer staging (#8809)

0z5a committed 5 days ago

Verified f3c6cbb2

ZeRO-3: build the partial-offload device string from the public accelerator API (#8994)

GuCarpenter committed 5 days ago

Verified d9225000

Keep float32 intermediates in the clamped SwiGLU eager fallback (#8992)

GuCarpenter committed 5 days ago

Verified 961df84b

Add an opt-in deterministic execution policy (#8917)

ziming-zh committed 5 days ago

Verified dfc63adf

Update version.txt post-v0.19.9 release (#8999)

loadams committed 5 days ago

Verified 2cd4984a

Older