[package] make extern a dict (#57336)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/57336
Avoid a small n^2
Differential Revision: D28114189
Test Plan: Imported from OSS
Reviewed By: astaff
Pulled By: suo
fbshipit-source-id: 2672669ad0e23169d70c92f9d5ed61f66081f248
