accelerate
Enable FSDP & Deepspeed + FP8
#2983
Merged
Overview
Commits
22
Changes
View On GitHub

Enable FSDP & Deepspeed + FP8 #2983

muellerzr merged 22 commits into main from muellerzr-fp8-deepspeed-support
muellerzr
muellerzr muellerzr requested a review from SunMarc SunMarc 1 year ago
muellerzr
HuggingFaceDocBuilderDev
SunMarc
SunMarc approved these changes on 2024-08-01
muellerzr muellerzr changed the title Enable FSDP + FP8 Enable FSDP & Deepspeed + FP8 1 year ago
muellerzr
muellerzr Working version rebased from main
dd174117
muellerzr muellerzr force pushed from ab32228f to dd174117 1 year ago
muellerzr kwargs
58877a42
muellerzr Clean
463da1a1
muellerzr Fix more nits
71c42207
muellerzr Fin
b5becfb0
muellerzr Delay autocast flag
1d837b52
muellerzr muellerzr requested a review from SunMarc SunMarc 1 year ago
muellerzr Enable FP8 autocast during eval only if specified
72a25fa3
muellerzr Fin
59af9994
muellerzr Rm comment
9f8f3069
muellerzr All done
3fbec3bd
muellerzr muellerzr requested a review from BenjaminBossan BenjaminBossan 1 year ago
muellerzr
BenjaminBossan
BenjaminBossan commented on 2024-08-14
muellerzr Zero3 works!
ad3d6693
muellerzr
muellerzr Let the wrapper come off during unwrap_model
cece1e9e
muellerzr Add import check
eaf018dc
muellerzr Migrate all to benchmarks folder and make TE import check work
8cbe9614
muellerzr Add readme
e6a5f4f4
muellerzr Add README to benchmarks folder
3d807827
muellerzr Update CLI to now include fp8 args
80791ad7
muellerzr Add test config for 0_34
2f71e28a
muellerzr Finish adding to config yaml
a10864e0
muellerzr Write docs
9f86ae37
muellerzr Expound docs w/ FP8
6dca1484
muellerzr Add to toctree
521b0986
muellerzr muellerzr merged a452327e into main 1 year ago
muellerzr muellerzr deleted the muellerzr-fp8-deepspeed-support branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum