[ao][sparsity] Composability of fusion and sparsity (#74847)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74847
Similar to the other PRs in this stack, the main problem was
that fusion needed to detect the original module type of parametrized
module when sparse prepare was called before fusion. In addition, there
was a potential issue with fusion before sparse_prepare but after the
sparse_config is created. However, in practice fusion moves the
references to the original modules into the fused module without issue.
Thus the original sparse_config that pointed to the original modules
gets automatically updated. If the fusion method changes this may cause
an issue since no explicit handling or updating of these pointers was
needed.
Test Plan:
python test/test_ao_sparsity.py TestComposability
Imported from OSS
Reviewed By: vkuzo, andrewor14, jerryzh168
Differential Revision: D35240273
fbshipit-source-id: 62ed66689b285c3fa68f1e149266ab877f1cdd8e
(cherry picked from commit 2adb002c43f702fa1f18637157264fcbc545002a)
