deepspeedai/DeepSpeed

Pull Requests Commits

Enable auto TP policy for llama model (#3170)

jianan-gu committed 2 years ago

Verified 2c63e349

Merge branch 'master' into jomayeri/issue-3367

jomayeri committed 2 years ago

Verified ca3e1717

jeffra committed 2 years ago

5314c9a1

Merge branch 'master' into jomayeri/issue-3367

jomayeri committed 2 years ago

Verified a342686f

Adding self.losses to engine constructor.

jomayeri committed 2 years ago

9c9070d2

Update 2020-09-09-sparse-attention.md (#3432)

goodship1 committed 2 years ago

Verified e0e80850

Adagrad support in ZeRO (#3401)

jomayeri committed 2 years ago

Verified d3550dc8

[zero_to_fp32] fix shared param recovery (#3407)

stas00 committed 2 years ago

Verified 77ebf760

Changing monitor loss to aggregate loss over gas.

jomayeri committed 2 years ago

33bd1da3

Add ZeRO 1 support to PP for BF16. (#3399)

jomayeri committed 2 years ago

Verified dcb4a7d6

OPT Activation Function Hotfix (#3400)

cmikeh2 committed 2 years ago

Verified 52d7e80a

Fix `PipelineEngine.eval_batch` result (#3316)

nrailg committed 2 years ago

Verified b0d9c4d0

Update links (#3424)

loadams committed 2 years ago

Verified b2ac7f63

Add HE support for the rest of model containers (#3191)

RezaYazdaniAminabadi committed 2 years ago

Verified 3e856464

save tensors in context of memory_efficient_linear (#3413)

tohtana committed 2 years ago

Verified 42858a98

doc updates (#3415)

goodship1 committed 2 years ago

Verified b4b63f52

Fix pipeline module evaluation when contiguous activation checkpointing is enabled (#3005)

BacharL committed 2 years ago

Verified 7ddc3b01

remove megatron-lm, no longer pip installable (#3389)

jeffra committed 2 years ago

Verified a094c976

[docs] add twitter badge to readme

jeffra committed 2 years ago

Verified f7d71ec1

Fix memory leak in zero2 contiguous gradients (#3306)

BacharL committed 2 years ago

Verified 01d17492

stage_1_and_2.py: do gradient scale only for fp16 (#3166)

guoyejun committed 2 years ago

Verified 0e357666

[DRAFT] Tentative implementation of MiCS (#2964)

zarzen committed 2 years ago

Verified 2e99f6ed

Auto TP Tutorial with T5 Example (#2962)

molly-smith committed 2 years ago

Verified d9253950

fixing default communication_data_type for bfloat16_enabled and docs (#3370)

Alexander Jipa committed 2 years ago

Verified d56268f3

Fix redundant shared_params in zero_to_fp32.py (#3149)

ShijieZZZZ committed 2 years ago

Verified 39825a90

add deepspeed chat blog links, add tags (#3369)

conglongli committed 2 years ago

Verified 085981bf

Update DS-Chat issue template (#3368)

mrwyattii committed 2 years ago

Verified 3031eec4

fix mpich launcher issue in multi-node (#3078)

sywangyi committed 2 years ago

Verified a748bfc6

hybrid_engine: check tuple size when fusing lora params (#3311)

adammoody committed 2 years ago

Verified c9196b61

README.md: Update MosaicML docs link (#3344)

kobindra committed 2 years ago

Verified 39b429d5

Newer Older