use default_factory and not a mutable dictionary (#1788)
Summary:
Sorry, I broke this yesterday
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1788
Reviewed By: xuzhao9
Differential Revision: D47810975
Pulled By: janeyx99
fbshipit-source-id: 9109d2446e4d41d917f957eb9925f757126f058b
