huggingface/accelerate

Pull Requests Commits

muellerzr committed 1 year ago

e5f6d570

Add msamp init args to deal with weight qtype defaults

muellerzr committed 1 year ago

854e44a0

muellerzr committed 1 year ago

4a37e5f0

muellerzr committed 1 year ago

21569352

muellerzr committed 1 year ago

e7979c30

muellerzr committed 1 year ago

019d14da

muellerzr committed 1 year ago

7db3368b

Snapshot: Works

muellerzr committed 1 year ago

41ce97ca

Freeze: working version

muellerzr committed 1 year ago

f3fe0d07

Working base fsdp version with only MSAMP

muellerzr committed 1 year ago

8da59ac9

Working MS-AMP with non-distributed and DDP

muellerzr committed 1 year ago

bb4061c7

Fixed MS-AMP implementation

muellerzr committed 1 year ago

64552d49

Off, but we're training now at least

muellerzr committed 1 year ago

8ed5816d

Working MS-AMP implementation + some fixes for accelerate

muellerzr committed 1 year ago

569c23da

destroy process group in `end_training` (#3012)

SunMarc committed 1 year ago

Verified 589fddd3

Wrong import check for TE (#3016)

muellerzr committed 1 year ago

Verified 99c69aaf

fix default value for rank size in cpu threads_per_process assignment logic (#3009)

rbrugaro committed 1 year ago

Verified 00785cd9

Enable FSDP & Deepspeed + FP8 (#2983)

muellerzr committed 1 year ago

Verified a452327e

Fix `find_tied_params` for models with shared layers (#2986)

qubvel committed 1 year ago

Verified 851cf343

update version to 0.34.dev0 (#3007)

SunMarc committed 1 year ago

Verified cd5698bb

Add small util to enable FSDP offloading quickly (#3006)

muellerzr committed 1 year ago

Verified 90d50239

Make env variables optional for FSDP (#2998)

muellerzr committed 1 year ago

Verified 3bde6156

Fix deepspeed tests (#3003)

muellerzr committed 1 year ago

Verified dc3b5ad8

clear memory after offload (#2994)

SunMarc committed 1 year ago

Verified 12a5befd

Support skip_first_batches for XLA (#2966)

yitongh committed 1 year ago

Verified 79ca85c2

Fix typo on warning str: "meta device device" -> "meta device" (#2997)

HeAndres committed 1 year ago

Verified 13d93c4f

Explicit check for `step` when loading the state (#2992)

muellerzr committed 1 year ago

Verified d982751a

Fix gated test (#2993)

muellerzr committed 1 year ago

Verified 95edc68c

Fix bug of clip_grad_norm_ for xla fsdp (#2941)

append-only committed 1 year ago

Verified 288accc0

remove .md to allow proper linking (#2977)

nbroad1881 committed 1 year ago

Verified 83b06101

Older