accelerate
Fixup dataloader state dict bugs + incorporate load/save_state API
#3034
Merged
Overview
Commits
19
Changes
View On GitHub

Fixup dataloader state dict bugs + incorporate load/save_state API #3034

muellerzr merged 19 commits into main from muellerzr-save-load-state-dl
muellerzr
muellerzr v1
f9d914ee
muellerzr More testing, need to try on H100
079eba45
muellerzr Bigger batch for h100 test
5515b70c
muellerzr test tweak
aecf440d
muellerzr Fixup all tests!
baa041d8
muellerzr Bookmark
1e5032e7
muellerzr Fix issues, working now
463a045a
muellerzr rm num samples
4ec77b5f
muellerzr Uncomment
58669510
muellerzr muellerzr requested a review from BenjaminBossan BenjaminBossan 1 year ago
muellerzr muellerzr requested a review from SunMarc SunMarc 1 year ago
HuggingFaceDocBuilderDev
muellerzr Give stateful dl end of dl
36a735c6
muellerzr Make skip DL stateful
e0ea50dd
muellerzr
muellerzr commented on 2024-08-22
muellerzr
muellerzr commented on 2024-08-22
BenjaminBossan
BenjaminBossan commented on 2024-08-23
byi8220
muellerzr
muellerzr Migrate to update_state_dict
b32f37ae
muellerzr try/finally
285ab46a
muellerzr Add comments to test
a2de1495
muellerzr rm comment
160d0ee7
muellerzr Document
ff8aefa3
muellerzr muellerzr requested a review from BenjaminBossan BenjaminBossan 1 year ago
byi8220
muellerzr
byi8220
BenjaminBossan
BenjaminBossan approved these changes on 2024-08-23
muellerzr refactor out for eventual override
3ca6272a
muellerzr Doc nit
5ad3091c
muellerzr
BenjaminBossan
BenjaminBossan approved these changes on 2024-08-23
byi8220
muellerzr Brute force it
2a83d541
muellerzr muellerzr merged 726140ca into main 1 year ago
muellerzr muellerzr deleted the muellerzr-save-load-state-dl branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum