accelerate
Add early support for `torchdata.stateful_dataloader.StatefulDataLoader` within the `Accelerator`
#2895
Merged
Merged
Add early support for `torchdata.stateful_dataloader.StatefulDataLoader` within the `Accelerator` #2895
muellerzr
merged
74 commits into
huggingface:main
from
byi8220:stateful-dataloader
temporary commit
79a8fa2d
checkout?
efa1e7dd
dataloader wrapper
8dc107dc
tmp
f342f4c5
weird failing test
065849a8
trying multiple inheritance
1e3fad19
DataLoaderAdapter
a41cf384
make style
88314882
Some dark magic dynamic reflection (for backwards compat)
140f1e65
typo
727afeb9
some tests
73683b42
more mixin stuff
32c318ee
maybe found broken test?
57c6f575
this is a very invasive feature
ed612d14
i think the feature is done?
511050e3
Merge branch 'huggingface:main' into stateful-dataloader
8dbc1a30
add xpu support (#2864)
df439606
Merge branch 'stateful-dataloader' of https://github.com/byi8220/acce…
c778e328
better tests
4e000559
discovered a bug
0471fe31
maybe fixed bug?
3036b7fe
make style
9ade2e9d
hopefully this is PR ready
ba0f5c60
properly skip tests
b774291d
parameterize
fde597d2
Merge branch 'huggingface:main' into stateful-dataloader
f273abcf
Merge branch 'huggingface:main' into stateful-dataloader
8a46eb6c
Merge branch 'huggingface:main' into stateful-dataloader
e4e1cacf
temporary commit
8bf2fe24
checkout?
ca4338d1
dataloader wrapper
c38f317f
tmp
17a2a195
weird failing test
b39a6060
trying multiple inheritance
d1e82e0a
DataLoaderAdapter
d99d734c
make style
39b2866b
Some dark magic dynamic reflection (for backwards compat)
f58f609c
typo
f2119cfd
some tests
7adec944
more mixin stuff
8850af32
maybe found broken test?
6ff0f680
this is a very invasive feature
4f28d2e2
i think the feature is done?
a9b637dc
better tests
03845432
discovered a bug
0e0515d2
maybe fixed bug?
809aca04
make style
5145c2d4
hopefully this is PR ready
ca74ff22
properly skip tests
a8f8bf31
parameterize
59738f4c
Merge branch 'stateful-dataloader' of https://github.com/byi8220/acce…
d264939d
Update src/accelerate/utils/dataclasses.py
8f04c1e8
Update src/accelerate/data_loader.py
45db4b98
merge conflicts
0ffc64b4
Merge branch 'stateful-dataloader' of https://github.com/byi8220/acce…
03a77740
move imports
8d2c6c3a
make style
6bfe871b
merge conflicts?
7a344e45
merges are breaking tests
6ff997e2
fix test name
4739524d
Merge branch 'huggingface:main' into stateful-dataloader
4de91594
Merge branch 'huggingface:main' into stateful-dataloader
abf815aa
Require safetensors>=0.4.3
06597d48
undo last commit
4142c7ff
muellerzr
requested changes
on 2024-07-29
byi8220
commented
on 2024-07-29
Merge branch 'huggingface:main' into stateful-dataloader
f02f18cf
minor style
35977ca6
Merge branch 'huggingface:main' into stateful-dataloader
597e9103
muellerzr
approved these changes
on 2024-08-20
Merge branch 'main' into stateful-dataloader
419f6073
address pr comments
4188d4c5
Torchdata version 0.8.0 is stable now
51377a43
added docs and require torchdata>=0.8.0 for testing
f4b6bb5d
BenjaminBossan
approved these changes
on 2024-08-21
test base_dataloader attr doesn't cause infinite recursion
d02dfcc0
address pr
21bc4204
replace super().__iter__ with self.base_dataloader.__iter__
74e2f53d
muellerzr
approved these changes
on 2024-08-21
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub