pytorch
5844f00b - [FSDP] Add `low_prec` prefix to param and reduce dtype varnames (#86512)

Commit
2 years ago
[FSDP] Add `low_prec` prefix to param and reduce dtype varnames (#86512) This PR renames `param_dtype` and `reduce_dtype` in `HandleConfig` to `low_prec_param_dtype` and `low_prec_reduce_dtype` to emphasize that they are meant to be of the low precision (if not `None`). (In my mind, mixed precision refers to the paradigm of using both full and low precision together during training. "Reduced" and "low precision" mean the same thing, but I prefer the term "low precision" in the code since it is shorter. A particular dtype can be a low precision dtype or a full precision dtype.) Pull Request resolved: https://github.com/pytorch/pytorch/pull/86512 Approved by: https://github.com/zhaojuanmao
Author
Committer
Parents
Loading