accelerate
8067abae - Fix StatefulDataLoader KeyError with num_workers > 0 (#3931)

Commit
152 days ago
Fix StatefulDataLoader KeyError with num_workers > 0 (#3931) When num_workers > 0, StatefulDataLoader uses _MultiProcessingDataLoaderIter which produces a different state_dict structure that may not contain _sampler_iter_yielded or _num_yielded keys. The adjust_state_dict_for_prefetch method was accessing these keys directly, causing a KeyError. Added key existence checks before accessing these state_dict entries. Fixes #3110
Author
Parents
Loading