microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into jomayeri/bf16-zero-check

tjruwase committed 2 years ago

Verified fafd05a1

adding attribute checks for bf opt with zero

jomayeri committed 2 years ago

cc043010

Update torch_checkpoint_engine.py (#3019)

stas00 committed 2 years ago

Verified e355863b

[docs] add new paper to readme/docs (#3018)

jeffra committed 2 years ago

Verified 4292e8c5

Fix buffer size for pipeline parallel and communication schedule (#2862)

tohtana committed 2 years ago

Verified b528f50e

ckpt: create directories in checkpoint_engine (#2988)

adammoody committed 2 years ago

Verified 43d58d99

Fix Meta Tensor checkpoint load for OPT models (#2990)

lekurile committed 2 years ago

Verified 3798e605

[zero] prevent poor configs from running w. zero-offload (#2971)

jeffra committed 2 years ago

Verified 457850dc

Fix issue between our abstract accelerator and colossalai's version of op_builder (#2963)

jeffra committed 2 years ago

Verified 58a4a4d4

bug fix for skipping mbs (#2171)

rahilbathwal5 committed 2 years ago

Verified 6379defa

jeffra committed 2 years ago

d58b4df3

deepspeed.init_distributed() support for TCP protocols (#2905)

noabauma committed 2 years ago

Verified db15ef57

[RFC] add device abstraction to allow other device than CUDA be used (#2221)

delock committed 2 years ago

Verified 0acf7e9c

Improve overflow handling (#2944)

tjruwase committed 2 years ago

Verified 80d8fcbd

Check for local CUDA graphs when enable_cuda_graph=True (#2941)

lekurile committed 2 years ago

Verified 87eaf8f9

AutoTP Assert Kernel Injection Support (#2939)

molly-smith committed 2 years ago

Verified 2ede0d94

TP unsupported models and assertions (#2810)

molly-smith committed 2 years ago

Verified 4ae3a3da

Add MPICH Multinode Runner (#2839)

inkcherry committed 2 years ago

Verified 8d53ac0c

Fixes `AttributeError` in #2853 (#2854)

saforem2 committed 2 years ago

Verified 91d7090e

Always convert input mask to half (#2851)

molly-smith committed 2 years ago

Verified 17fa0876

Fix CPUAdam for when `vendor_id_raw` is not provided (#2836)

FarzanT committed 2 years ago

Verified 9886d6d9

using container when loading inference checkpoints (#2875)

HeyangQin committed 2 years ago

Verified dc01cee5

better eval sampler (#2907)

mayank31398 committed 2 years ago

Verified f1d2a15b

Enable tensor fragments for zero 2 & 3 (#2727)

tjruwase committed 2 years ago

Verified 541e423a

add missing license info to top of all source code (#2889)

jeffra committed 2 years ago

Verified da84e60d

Reduce I/O size (#2814)

tjruwase committed 2 years ago

Verified 8710f051

Remove deprecated `torch._six` imports (#2863)

yasyf committed 2 years ago

Verified d3de7375

AutoTP tutorial web formatting and news (#2883)

molly-smith committed 2 years ago

Verified b47c592a

Check device count before running dist tests (#2799)

HeyangQin committed 2 years ago

Verified 7e77cf71

Automatic Tensor Parallelism Blog Links (#2877)

molly-smith committed 2 years ago

Verified 859d7c92

Older