microsoft/DeepSpeed

Pull Requests Commits

docs: note async cpu_checkpointing perf and expandable_segments

sfc-gh-truwase committed 22 hours ago

9eb6e1d4

Tersify activation offload comments

sfc-gh-truwase committed 1 day ago

b8f76a98

Wire async CPU offload into native cpu_checkpointing

sfc-gh-truwase committed 1 day ago

50d0609a

Drop Microsoft copyright from activation offload files

sfc-gh-truwase committed 2 days ago

687d5f6f

Tersify activation offload comments

sfc-gh-truwase committed 2 days ago

c9c579cd

Correct stride-0 skip comment

sfc-gh-truwase committed 2 days ago

6bb8f177

Pin activation offload host buffers via accelerator API

sfc-gh-truwase committed 2 days ago

59abc558

Honor max_fwd_stash_count=0 in activation offload

sfc-gh-truwase committed 2 days ago

08c8ce31

Finish C1 review follow-ups for activation offload

sfc-gh-truwase committed 2 days ago

7efaa98d

Keep ATen pin for eager activation offload DMA

sfc-gh-truwase committed 2 days ago

59f4b0dd

Route contiguous activation offload pins through accelerator pin_memory

sfc-gh-truwase committed 2 days ago

b36d1b6e

Address review of non-reentrant activation checkpoint CPU offload

sfc-gh-truwase committed 2 days ago

c0d616ba

Resolve compute stream per use; fix sliced-view restore

winglian committed 2 days ago

0986b391

Non-reentrant activation checkpoint CPU offload

winglian committed 2 days ago

d4a02b49

[AutoTP] Complete uneven sharding and universal checkpoint support (#8185)

jinyouzhi committed 2 days ago

Verified aa3914df

Keep parameter dtype through ZeRO-3 weight quantization (#8215)

adityasingh2400 committed 2 days ago

Verified 6098f799

Fix docstring Args entries that name a parameter the function does not take (#8223)

darkdi committed 2 days ago

Verified 32d51a18

Add CUDA graph support for HybridEngine generation (#8271)

PKUWZP committed 3 days ago

Verified 80c8e5ba

Fix ZeRO-3 crash in AutoTP universal-checkpoint metadata (#8270)

PKUWZP committed 3 days ago

Verified 313ce47b

NCCL and other backend PG timeout 30min -> 10min (#8253)

stas00 committed 4 days ago

Verified 24402c3b

Fix int32 overflow in Triton grouped-GEMM expert offset (#8261)

PKUWZP committed 5 days ago

Verified c5331bc1

Add compile.offload_activation_pin_memory for DeepCompile activation offload (#8258)

sfc-gh-truwase committed 6 days ago

Verified c952d92a

Add native (DeepNVMe) host-memory pinning backend for accelerators (#8211)

sfc-gh-truwase committed 7 days ago

Verified aa0e91b9

Add configurable sum gradient reduction (#8232)

zupengwang committed 7 days ago

Verified 1cae4920

Fix DeepCompile last use for unconsumed waits (#8254)

tohtana committed 7 days ago

Verified 9bd89f9d

(1/2) Implementing Compiler Pass for AutoTP (#8204)

therealnaveenkamal committed 8 days ago

Verified 79046032

Upgrade pip in Python install-smoke jobs (#8247)

tohtana committed 9 days ago

Verified 22969fa7

[DeepCompile] Fix staticmethod handling on Python 3.9 (#8240)

baremetaldevx86 committed 10 days ago

Verified 1d580d65

[DeepCompile] Fix KeyError on frozen parameters in ZeRO-3 (#8214)

roycho96 committed 10 days ago

Verified 04b4b0cb

Keep the elastic batch size within max_train_batch_size (#8237)

vineethsaivs committed 10 days ago

Verified 2cfebbda

Older