Revert #3610 & #3684 (#3688)
* Revert "linalg_test: define test matrices lazily (#3684)"
This reverts commit 2be1baa41a170192c209c94b060d0d034d1de2c2.
* Revert "Make LU gradient work for low-rank matrices (#3610)"
This reverts commit 23deefa71838ceeab41977ac0ab781164c914a8c.
