Remove _all_weight_values again (#38504)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/38504
Test Plan: Imported from OSS
Differential Revision: D21579530
Pulled By: jamesr66a
fbshipit-source-id: 4449c92142200eaadc68b59d6f5f964ba60b1c80
