Revert "Add activation functions (ReLU and SiLU for now) for structured sparse linear operator (#101339)"
This reverts commit bfb3941ad8aaf0af159c2bec3cf1cbec1488f335.
Reverted https://github.com/pytorch/pytorch/pull/101339 on behalf of https://github.com/izaitsevfb due to Depends on #100881, which has to be reverted due to internal build breakage. ([comment](https://github.com/pytorch/pytorch/pull/101339#issuecomment-1553618216))
