expanded weights without fast rules (#70140)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/70140
[Design Doc for Expanded Weights](https://gist.github.com/samdow/fa0a164fec7963f93ff45284989cfc55) <-- gives an overview of the design for Expanded Weights
Introduces the ExpandedWeights mechanism and user-facing API without any custom implemented, faster rules.
- User facing API is in `_stateless.py` (with documentation)
- Testing is in test_expanded_weights
- The rest is the implementation of the erroring fallback + the mechanism for being able to register faster per sample grad rules. Only linear is implemented here, but they are all implemented in #70141
Test Plan: Imported from OSS
Reviewed By: mikaylagawarecki
Differential Revision: D34350950
Pulled By: samdow
fbshipit-source-id: 69c664b0bc3dff6951358d79d7e5d94882f7aef2
(cherry picked from commit ae1620d3b6507b27c3bc08ecfb2b1418aa8ce7d7)
