microsoft/DeepSpeed

Pull Requests Commits

Add FALCON Auto-TP Support (#3640)

RezaYazdaniAminabadi committed 2 years ago

Verified f3c93b05

update MMEngine support in README.md (#3879)

HAOCHENYE committed 2 years ago

Verified 385e89d4

update lightning version in CI (#3882)

mrwyattii committed 2 years ago

Verified 7c126f43

Fix LoRA Fuse/Unfuse in Hybrid Engine (#3563)

sxjscience committed 2 years ago

Verified d81dfdab

fix retrieval of out_channels in _conv_trans_flops_compute (#3834)

pinstripe-potoroo committed 2 years ago

Verified c1c1d249

[zero] revert PR #3166, it disabled grad clip for bf16 (#3790)

jeffra committed 2 years ago

Verified 691d246e

Zero3 Fix allreduce optimization for extra large tensor (#3832)

BacharL committed 2 years ago

Verified d229ff17

scripts/check-torchcuda.py: add checking for tensor.is_cuda (#3843)

guoyejun committed 2 years ago

Verified 807d1b5d

checking process_group before merging bucket ranges (#3521) (#3577)

Alexander Jipa committed 2 years ago

Verified 2ded2ff0

[profiling]add show_straggler argument to log_summary() (#3579)

delock committed 2 years ago

Verified 5d1124f2

Reduce Unit Test Time (Part 2) (#3838)

mrwyattii committed 2 years ago

Verified fd1d2c64

Disable AMD test flows in YML (#3847)

loadams committed 2 years ago

Verified c973e157

zero/mics.py: use on_accelerator instead of cuda only (#3806)

guoyejun committed 2 years ago

Verified b4626194

Fix racing condition in GatheredParameters (#3819)

HeyangQin committed 2 years ago

Verified f8551b43

Re-enable GPT-J unit tests and refactor inference tests (#3618)

mrwyattii committed 2 years ago

Verified 78b76935

Reduce Unit Test Times (Part 1) (#3829)

mrwyattii committed 2 years ago

Verified 7726fc8d

Update zeropp.md (#3821)

samadejacobs committed 2 years ago

Verified 25e500e8

support model declaration in zero.Init context (#3592)

tohtana committed 2 years ago

Verified 203ac9d7

July 2022 update: chrome://tracing is deprecated, and by default will redirect to https://ui.perfetto.dev. (see https://chromium.googlesource.com/catapult/+/refs/heads/main/tracing/docs/perfetto.md) (#3805)

keyboardAnt committed 2 years ago

Verified a63f9152

Revert "Prevent hangs in CI during parallel run compilation (#2844)" (#3817)

jeffra committed 2 years ago

Verified 6102d128

Prevent hangs in CI during parallel run compilation (#2844)

mrwyattii committed 2 years ago

Verified 2b2be85f

Add an api in deepspeed engine for adjusting micro batch size during training (#3773)

kisseternity committed 2 years ago

Verified 1b888399

bug fix: triton importing error (#3799)

stephen-youn committed 2 years ago

bafaf3c0

Fix Bug in transform.cu (#3534)

rraminen committed 2 years ago

aebdfb3b

jeffra committed 2 years ago

d33f1f85

Requires grad checking. (#3789)

jomayeri committed 2 years ago

5eb25986

Missing strided copy for gated MLP (#3788)

cmikeh2 committed 2 years ago

c86e4e31

Bug Fixes for autotuner and flops profiler (#1880)

cli99 committed 2 years ago

c80855b5

Add ZeRO++ Japanese blog (#3797)

tohtana committed 2 years ago

b0752b2e

adding zero++ to navigation panel of deepspeed.ai (#3796)

HeyangQin committed 2 years ago

94479e2b

Older