microsoft/DeepSpeed

Pull Requests Commits

faster allreduce with omp parallel for reduce kernel (#4049)

delock committed 2 years ago

Verified 7f26bb6a

autoTP for fused qkv weight (#3844)

inkcherry committed 2 years ago

Verified 6b877d2d

enable autoTP for MPT (#3861)

sywangyi committed 2 years ago

Verified 0bafeac4

fix opt-350m shard loading issue in AutoTP (#3600)

sywangyi committed 2 years ago

Verified 76953a37

fix comm logging for inference (#4043)

delock committed 2 years ago

Verified 0b507253

fix: Remove duplicate word the (#4051)

digger-yu committed 2 years ago

Verified 389bf693

Engine side fix for loading llama checkpoint fine-tuned with zero3 (#3981)

minjiaz committed 2 years ago

Verified 15f94ae7

polishing timers and log_dist (#3996)

Alexander Jipa committed 2 years ago

Verified b354c28b

fix(pipe): `strict` parameter of method `PipelineModule.load_state_dir` is not work (#4020)

hughpu committed 2 years ago

Verified 09601bb8

Monitored Loss Calculations (#4030)

jomayeri committed 2 years ago

Verified eeab613a

Remove print of weight parameter in RMS norm (#4031)

Puneesh Khanna committed 2 years ago

Verified ad661b8e

Pass missing positional arguments in `DeepSpeedHybridEngine.generate()` (#4026)

XuehaiPan committed 2 years ago

Verified 43188ff0

remove duplicate check for pp and zero stage (#4033)

inkcherry committed 2 years ago

Verified e8600b7f

[CPU] FusedAdam and CPU training support (#3991)

delock committed 2 years ago

Verified 0f540632

[zero_to_fp32] 3x less cpu memory requirements (#4025)

stas00 committed 2 years ago

Verified 1cc9caa9

add llama2 autoTP support in replace_module (#4022)

Dino Chen committed 2 years ago

Verified f3943cf9

fix typo in SECURITY.md (#4019)

jstan327 committed 2 years ago

Verified 777ae39a

fix gates size retrieval logic in _rnn_flops (#3921)

pinstripe-potoroo committed 2 years ago

Verified f4d18fa2

Make Ascend NPU available (#3831)

hipudding committed 2 years ago

Verified 23a11a39

add /dev/shm size to ds_report (#4015)

jeffra committed 2 years ago

Verified 19d5c03d

Re-enable elastic training for torch 2+ (#4010)

loadams committed 2 years ago

Verified 6b2365e4

Specify triton 2.0.0 requirement (#4008)

mrwyattii committed 2 years ago

Verified 8e808392

Fix user arg parsing in single node deployment (#4007)

mrwyattii committed 2 years ago

Verified 0cc2d6ff

silence warning (#4009)

mrwyattii committed 2 years ago

Verified 11f8e4a5

Allow user to select name of .deepspeed_env (#4006)

loadams committed 2 years ago

Verified 6580a2db

Option to exclude frozen weights for checkpoint save (#3953)

tjruwase committed 2 years ago

Verified 0a0819b7

Make AMD/ROCm apex install to /blob to save test/compile time. (#3997)

loadams committed 2 years ago

Verified ceccfa3e

Re-enable skipped unit tests (#3939)

mrwyattii committed 2 years ago

Verified 7b850d3d

[CPU] Use allreduce_low_latency for AutoTP and implement low latency allreduce for CPU backend (single node) (#3919)

delock committed 2 years ago

Verified 1bc3b784

ZeRO Gradient Accumulation Dtype. (#2847)

jomayeri committed 2 years ago

Verified 8afcda2a

Newer Older