pytorch
c1e2fa81 - [dtensor] add StrategyType and TupleStrategy (#99435)

Commit
1 year ago
[dtensor] add StrategyType and TupleStrategy (#99435) This PR refactors the current StrategyList. It introduces a StrategyType, which is the base class of Strategy, and it have two sub strategies: 1. Refactor the previous StrategyList to OpStrategy 2. Add TupleStrategy, the new strategy added to deal with tuple cases where it could return multiple different OpStrategy for an op. This would help support a more complicated op and unblocks compile mode FSDP Pull Request resolved: https://github.com/pytorch/pytorch/pull/99435 Approved by: https://github.com/mrshenli
Author
Committer
Parents
Loading