microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into cpu-adam-mode-fix

delock committed 2 days ago

Verified 9d209cc8

Route isend/irecv to nonblocking backend methods and stage them as async on MPS (#8303)

PKUWZP committed 2 days ago

Verified 9311fd55

Honor adam_w_mode in the CPU multi_tensor_adam binding

PKUWZP committed 2 days ago

1e17e84f

perf(rollout): add HybridEngine rollout profiling (#8295)

nathon-lee committed 2 days ago

Verified da3ca683

Skip zero-sized parameters in HP fragment mapping (#8298)

fwerkor committed 3 days ago

Verified 573ce488

Drop the documented grad_hooks ZeRO option, which does not exist (#8242)

vineethsaivs committed 3 days ago

Verified f76ab880

Register native pinned host memory with CUDA for GPU DMA (#8283)

sfc-gh-truwase committed 3 days ago

Verified cb260806

Fix zero reduce bucket size validation (#8266)

fwerkor committed 3 days ago

Verified e70b9035

[AutoTP] Convert HF embedding_rowwise tp_plan entries to SKIP specs (#8294)

ebarkhordar committed 3 days ago

Verified 20ea4543

fix(engine): resolve inference workspace attribute lookup (#8288)

nathon-lee committed 3 days ago

Verified fb57b819

Enable DeepSpeed support on Apple Silicon (MPS) with ZeRO Stage 1-3 (#8293)

PKUWZP committed 3 days ago

Verified 64fcec6b

Fix repeated FlopsProfiler metric accumulation (#8246)

baremetaldevx86 committed 3 days ago

Verified ebb75d7e

fix: stop DeepSpeedConfig writing max_grad_norm back into the caller's config dict (#8289)

ebarkhordar committed 3 days ago

Verified f5671753

Fix ZeroDivisionError in compute_elastic_config return_microbatch on non-0.2 elasticity (#8286)

ebarkhordar committed 3 days ago

Verified ad1c516e

docs: note async cpu_checkpointing perf and expandable_segments (#8287)

sfc-gh-truwase committed 3 days ago

Verified 96ffef2c

Fix ZeRO-1/2 with zero-sized parameters (#8280)

fwerkor committed 4 days ago

Verified edaa7221

[typo] Add raise for RuntimeError (#8281)

jinyouzhi committed 5 days ago

Verified 11b518a0

Enable activation offloading (#8255)

pengdurice committed 5 days ago

Verified 9a297cfa

Non-reentrant activation checkpoint CPU offload (#8282)

sfc-gh-truwase committed 5 days ago

Verified 858e91ee

Make pass contracts cover the passes DeepCompile actually schedules; Renaming files (#8251)

pengdurice committed 5 days ago

Verified 35c8b03c

Add fused triton kernel for swiglu (#8244)

hwchen2017 committed 5 days ago

Verified 24a41afa

Route FPDT and checkpoint writer pins through accelerator pin_memory (#8257)

sfc-gh-truwase committed 6 days ago

Verified 310eb8cb

[AutoTP] Complete uneven sharding and universal checkpoint support (#8185)

jinyouzhi committed 6 days ago

Verified aa3914df

Keep parameter dtype through ZeRO-3 weight quantization (#8215)

adityasingh2400 committed 7 days ago

Verified 6098f799

Fix docstring Args entries that name a parameter the function does not take (#8223)

darkdi committed 7 days ago

Verified 32d51a18

Add CUDA graph support for HybridEngine generation (#8271)

PKUWZP committed 7 days ago

Verified 80c8e5ba

Fix ZeRO-3 crash in AutoTP universal-checkpoint metadata (#8270)

PKUWZP committed 7 days ago

Verified 313ce47b

NCCL and other backend PG timeout 30min -> 10min (#8253)

stas00 committed 9 days ago

Verified 24402c3b

Fix int32 overflow in Triton grouped-GEMM expert offset (#8261)

PKUWZP committed 9 days ago

Verified c5331bc1

Add compile.offload_activation_pin_memory for DeepCompile activation offload (#8258)

sfc-gh-truwase committed 10 days ago

Verified c952d92a

Older