[quant] Remove assert for weight since it could be non-Tensor (#74365)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74365
att
Test Plan:
Meta-Internal tests with fx2trt
Imported from OSS
Reviewed By: andrewor14
Differential Revision: D34952754
fbshipit-source-id: 11d392a520c9ab7c9484c96841f2b39fbbbc3f80
(cherry picked from commit e8a2348d5c6f9717b010972819723affba37a0e4)
