microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into zhipwang_dev

sfc-gh-truwase committed 351 days ago

Verified 5b41be38

Update version after 0.18.1 release (#7647)

loadams committed 352 days ago

Verified 9a012d21

Update version.txt in advance of next release

loadams committed 352 days ago

3631712b

Add DataStates-LLM: Asynchronous Checkpointing Engine Support (#7166)

mauryaavinash95 committed 352 days ago

Verified d1e62ff2

Ulysses HF Accelerate integration (#7638)

stas00 committed 353 days ago

Verified 64c0052f

z2: don't pass `dtype` to `report_ipg_memory_usage` (#7636)

stas00 committed 353 days ago

Verified 9c86cd98

z2: report param name and not zero id in assert (#7637)

stas00 committed 353 days ago

Verified fc854363

No Muon optimizer for embeding and lm_head layer (#7641)

delock committed 353 days ago

Verified 67b365af

Fix illegal memory access with multi_tensor_apply size above INT_MAX (#7639)

wangyan-mms committed 353 days ago

Verified 7af561c2

add support for tensor learning rate (vs scalar) (#7633)

NirSonnenschein committed 355 days ago

Verified 407708cd

Take **kwargs in __init__ of DeepSpeedZeroOptimizer subclasses (#7634)

eternalNight committed 355 days ago

Verified 2734a6a1

[XPU][CI] recover xpu-max1100 workflow (#7630)

Liangliang-Ma committed 362 days ago

Verified 69e03e52

Add ZenFlow code for Stage 3 (#7516)

JoshWoo2003 committed 362 days ago

Verified 7cb1b88e

sfc-gh-truwase committed 1 year ago

b7cd78f0

Update email address (#7624)

sfc-gh-truwase committed 1 year ago

Verified 79caae1c

[TiledMLP] moe support (#7622)

stas00 committed 1 year ago

Verified 1b08325d

Clarify document of leaf module config (#7623)

tohtana committed 1 year ago

Verified 1ae1cdd8

Blog of zenflow binding study (#7614)

delock committed 1 year ago

Verified 2b68bbc5

Enable grad scaler for ZeRO-0 + torch.autocast path (#7619)

tohtana committed 1 year ago

Verified 71d077da

Super offload blog Chinese version (#7620)

delock committed 1 year ago

Verified 65322e10

add print_dist util (#7621)

stas00 committed 1 year ago

Verified 4eb37729

Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) (#7604)

tohtana committed 1 year ago

Verified 7d9a2f2b

Show mismatching values when DeepCompile test fails (#7618)

tohtana committed 1 year ago

Verified 82a9db7e

DeepCompile: Use min_cut_rematerialization for partitioning joint graphs (#7609)

eternalNight committed 1 year ago

Verified 2a769889

[wall_clock_breakdown] always log stats when enabled (#7617)

stas00 committed 1 year ago

Verified 9cbd3edd

Fixed save_checkpoint race when consolidating NVMe offloaded tensors (#7613)

H1manshu21 committed 1 year ago

Verified e37c37ac

Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. (#7599)

zhengchenyu committed 1 year ago

Verified 07e76bd4

Minor fix in the SuperOffload blog (#7612)

xylian86 committed 1 year ago

Verified 330f738c

DeepCompile: Fix IPG bucket clearing (#7610)

eternalNight committed 1 year ago

Verified aa90f544

Handle the case of DeepCompile's enabled but not activated (#7603)

tohtana committed 1 year ago

Verified e32e8173

Older