microsoft/DeepSpeed

Pull Requests Commits

add sharded checkpoint loading for AutoTP path to reduce the peak mem… (#3102)

sywangyi committed 2 years ago

Verified d10b8ca0

Hybrid Engine Refactor and Llama Inference Support (#3425)

cmikeh2 committed 2 years ago

Verified 0a61d5d6

add skip for mismatched cuda versions (#3436)

mrwyattii committed 2 years ago

Verified d2e6110e

Enable auto TP policy for llama model (#3170)

jianan-gu committed 2 years ago

Verified 2c63e349

jeffra committed 2 years ago

5314c9a1

Update 2020-09-09-sparse-attention.md (#3432)

goodship1 committed 2 years ago

Verified e0e80850

Adagrad support in ZeRO (#3401)

jomayeri committed 2 years ago

Verified d3550dc8

[zero_to_fp32] fix shared param recovery (#3407)

stas00 committed 2 years ago

Verified 77ebf760

Add ZeRO 1 support to PP for BF16. (#3399)

jomayeri committed 2 years ago

Verified dcb4a7d6

OPT Activation Function Hotfix (#3400)

cmikeh2 committed 2 years ago

Verified 52d7e80a

Fix `PipelineEngine.eval_batch` result (#3316)

nrailg committed 2 years ago

Verified b0d9c4d0

Update links (#3424)

loadams committed 2 years ago

Verified b2ac7f63

Add HE support for the rest of model containers (#3191)

RezaYazdaniAminabadi committed 2 years ago

Verified 3e856464

save tensors in context of memory_efficient_linear (#3413)

tohtana committed 2 years ago

Verified 42858a98

doc updates (#3415)

goodship1 committed 2 years ago

Verified b4b63f52

Fix pipeline module evaluation when contiguous activation checkpointing is enabled (#3005)

BacharL committed 2 years ago

Verified 7ddc3b01

remove megatron-lm, no longer pip installable (#3389)

jeffra committed 2 years ago

Verified a094c976

[docs] add twitter badge to readme

jeffra committed 2 years ago

Verified f7d71ec1

Fix memory leak in zero2 contiguous gradients (#3306)

BacharL committed 2 years ago

Verified 01d17492

stage_1_and_2.py: do gradient scale only for fp16 (#3166)

guoyejun committed 2 years ago

Verified 0e357666

[DRAFT] Tentative implementation of MiCS (#2964)

zarzen committed 2 years ago

Verified 2e99f6ed

Auto TP Tutorial with T5 Example (#2962)

molly-smith committed 2 years ago

Verified d9253950

fixing default communication_data_type for bfloat16_enabled and docs (#3370)

Alexander Jipa committed 2 years ago

Verified d56268f3

Fix redundant shared_params in zero_to_fp32.py (#3149)

ShijieZZZZ committed 2 years ago

Verified 39825a90

add deepspeed chat blog links, add tags (#3369)

conglongli committed 2 years ago

Verified 085981bf

Update DS-Chat issue template (#3368)

mrwyattii committed 2 years ago

Verified 3031eec4

fix mpich launcher issue in multi-node (#3078)

sywangyi committed 2 years ago

Verified a748bfc6

hybrid_engine: check tuple size when fusing lora params (#3311)

adammoody committed 2 years ago

Verified c9196b61

README.md: Update MosaicML docs link (#3344)

kobindra committed 2 years ago

Verified 39b429d5

add bf16 cuda kernel support (#3092)

dc3671 committed 2 years ago

Verified 297cd9ed

Newer Older