accelerate

3e27a7c6 - Fix iterable dataset sharding condition when n_shards == num_processes (#3958)

Fix iterable dataset sharding condition when n_shards == num_processes (#3958) * Fix iterable dataset sharding condition when n_shards == num_processes Use >= instead of > so native HF dataset sharding is used when the shard count exactly matches the process count, instead of falling back to the less efficient IterableDatasetShard wrapper. * Fix iterable dataset sharding condition when n_shards == num_processes Use >= instead of > so native HF dataset sharding is used when the shard count exactly matches the process count, instead of falling back to the less efficient IterableDatasetShard wrapper. * nice