microsoft/DeepSpeed

Pull Requests Commits

allow seperate learning rate "muon_lr" and "adam_lr" for muon optimizer (#7658)

delock committed 71 days ago

Verified df59f203

disable nv-lightning-v100.yml cI (#7681)

stas00 committed 74 days ago

Verified 283f6f5f

leaf modules: explain better (#7674)

stas00 committed 75 days ago

Verified d25fd486

[modal ci] fixes (#7676)

stas00 committed 76 days ago

Verified b073a557

Update version.txt after release (#7675)

loadams committed 76 days ago

Verified c4b8c3f3

README refresh (#7668)

sfc-gh-truwase committed 78 days ago

Verified e993fea3

Fix typo in pytorch-profiler.md documentation (#7652)

kunheek committed 79 days ago

Verified 4691bebd

resolved a 0-dim tensor slicing bug from _get_state_without_padding (#7659)

therealnaveenkamal committed 79 days ago

Verified 6fe3c7ab

UlyssesSP: TiledMLP doc - recomputes forward twice (#7664)

stas00 committed 79 days ago

Verified 76a40757

[bug]: fixed comm_dtype in extra_large_param_to_reduce (#7660)

therealnaveenkamal committed 82 days ago

Verified 3e64f495

Fix misplaced overflow handling return in fused_optimizer.py (#7645)

rraminen committed 82 days ago

Verified d56e847b

ALST/UlyssesSP: more intuitive API wrt variable seqlen (#7656)

stas00 committed 85 days ago

Verified 02da3732

ulysses mpu: additional api (#7649)

stas00 committed 85 days ago

Verified 433e3c72

Deduplicate fp32 weights under torch autocast and ZeRO3 (#7651)

eternalNight committed 86 days ago

Verified 706f6e82

Update version after 0.18.1 release (#7647)

loadams committed 90 days ago

Verified 9a012d21

Update version.txt in advance of next release

loadams committed 90 days ago

3631712b

Add DataStates-LLM: Asynchronous Checkpointing Engine Support (#7166)

mauryaavinash95 committed 90 days ago

Verified d1e62ff2

Ulysses HF Accelerate integration (#7638)

stas00 committed 91 days ago

Verified 64c0052f

z2: don't pass `dtype` to `report_ipg_memory_usage` (#7636)

stas00 committed 91 days ago

Verified 9c86cd98

z2: report param name and not zero id in assert (#7637)

stas00 committed 91 days ago

Verified fc854363

No Muon optimizer for embeding and lm_head layer (#7641)

delock committed 91 days ago

Verified 67b365af

Fix illegal memory access with multi_tensor_apply size above INT_MAX (#7639)

wangyan-mms committed 92 days ago

Verified 7af561c2

add support for tensor learning rate (vs scalar) (#7633)

NirSonnenschein committed 93 days ago

Verified 407708cd

Take **kwargs in __init__ of DeepSpeedZeroOptimizer subclasses (#7634)

eternalNight committed 93 days ago

Verified 2734a6a1

[XPU][CI] recover xpu-max1100 workflow (#7630)

Liangliang-Ma committed 100 days ago

Verified 69e03e52

Add ZenFlow code for Stage 3 (#7516)

JoshWoo2003 committed 100 days ago

Verified 7cb1b88e

sfc-gh-truwase committed 106 days ago

b7cd78f0

Update email address (#7624)

sfc-gh-truwase committed 106 days ago

Verified 79caae1c

[TiledMLP] moe support (#7622)

stas00 committed 106 days ago

Verified 1b08325d

Clarify document of leaf module config (#7623)

tohtana committed 106 days ago

Verified 1ae1cdd8

Newer Older