Add glue code to collect debug info from all components
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/30888
Test Plan: Imported from OSS
Differential Revision: D18857139
Pulled By: mrshenli
fbshipit-source-id: 5c1bfb83a21a4a57c4297bb94f14baa09520b791
