pytorch

bda6ff09 - [1/4][DataPipe] Properly cleanup unclosed files within generator function (#89973)

[1/4][DataPipe] Properly cleanup unclosed files within generator function (#89973) There is a case that `file.close` is never called because when generator function has never reached to the end. A simple example would be `zip` two datepipes with different length. The longer DataPipe would never reach the end of generator and then it will be cleaned up by `gc`. So, the line of `file.close` is not executed. (This is the reason that Vitaly has to create this [hack](https://github.com/pytorch/pytorch/blob/4451eb24e6287dff62ff8a7ec0eda6a6998807b0/torch/utils/data/datapipes/iter/combining.py#L573-L583) to retrieve all remaining data to make sure generator function is fully executed) However, this hack introduces another problem where an infinite datapipe would make `zip` never end as it would try to deplete the infinite iterator. See: https://github.com/pytorch/data/issues/865 So, in this PR, I am adding a `try-finally` clause to make sure the `file.close` is always executed during the destruction of `generator` object. Then, we don't need the hack within `zip` any more. Differential Revision: [D41699470](https://our.internmc.facebook.com/intern/diff/D41699470) Pull Request resolved: https://github.com/pytorch/pytorch/pull/89973 Approved by: https://github.com/NivekT