accelerate
Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed.
#2674

Merged

Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed. #2674

muellerzr merged 15 commits into huggingface:main from fabianlim:fsdp_deepspeed

draft fsdp vs ds

3c3cf2e7

reframe to migration doc

35af870e

updated functionality section

c67249fb

cast to float32

fd81eb13

improvements to float32 casting

c650673e

some cleanup

2bf8b9cd

fabianlim force pushed from 93c9e76d to 2bf8b9cd 1 year ago

pacman100 commented on 2024-04-16

addressed @pacman100's comments

d68c5d36

muellerzr commented on 2024-04-18

Apply some of @muellerz suggestions

c9794e78

change to subsections

af71e9f7

changed the manner upcasting warnings are surfaced

0b8e97c9

fabianlim force pushed from ffc2299a to 0b8e97c9 1 year ago

update document to discuss fsdp and ds plugins. minor fixes.

681c697d

fabianlim force pushed from 2e88c7bd to 681c697d 1 year ago

muellerzr approved these changes on 2024-04-24

muellerzr requested a review from

pacman100 1 year ago

@muellerzr's new suggestions

3b74c268

explain all-or-nothing

c366b787

pacman100 approved these changes on 2024-04-29

add @pacman100's comments

59ed2c1a

minor fix

92125b89

muellerzr merged 9557598c into main 1 year ago

stas00 commented on 2024-04-29

stas00 commented on 2024-06-03

Reviewers

pacman100

muellerzr

stas00

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

accelerate Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed. #2674 Merged

Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed. #2674

accelerate
Add Upcasting for FSDP in Mixed Precision. Add Concept Guide for FSPD and DeepSpeed.
#2674

Merged