[FX] Fix handling of attributes (#47030)
Summary:
Probably works :)
Fixes https://github.com/pytorch/pytorch/issues/46872
Pull Request resolved: https://github.com/pytorch/pytorch/pull/47030
Reviewed By: ngimel
Differential Revision: D24652600
Pulled By: Chillee
fbshipit-source-id: 3fe7099ad02d1b5c23a7335b855d36d373603d18
