Improve error message and print stack backtrace when failing to load models and canary_models (#1615)
Summary:
With the fix, the error message will correctly show missing modules or indicate that a certain model does not exist.
Pull Request resolved: https://github.com/pytorch/benchmark/pull/1615
Test Plan:
Test case 1: run a model with dependencies installed
```
(pytorch) PS D:\my git repo\benchmark> python run.py resnet50
Running eval method from resnet50 on cpu in eager mode with input batch size 32 and precision fp32.
CPU Total Wall Time: 2833.809 milliseconds
CPU Peak Memory: 0.6416 GB
```
Test case 2: run a model with missing dependencies
```
(pytorch) PS D:\my git repo\benchmark> python run.py yolov3
Traceback (most recent call last):
File "D:\my git repo\benchmark\run.py", line 303, in <module>
Model = load_model_by_name(args.model)
File "D:\my git repo\benchmark\torchbenchmark\__init__.py", line 612, in load_model_by_name
module = importlib.import_module(f'.models.{model_pkg}', package=__name__)
File "C:\Users\temp\anaconda3\envs\pytorch\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "D:\my git repo\benchmark\torchbenchmark\models\yolov3\__init__.py", line 15, in <module>
from .yolo_train import prepare_training_loop
File "D:\my git repo\benchmark\torchbenchmark\models\yolov3\yolo_train.py", line 6, in <module>
from torch.utils.tensorboard import SummaryWriter
File "C:\Users\temp\anaconda3\envs\pytorch\lib\site-packages\torch\utils\tensorboard\__init__.py", line 1, in <module>
import tensorboard
ModuleNotFoundError: No module named 'tensorboard'
```
Test case 3: run a canary_model with missing dependencies:
```
(pytorch) PS D:\my git repo\benchmark> python run.py gat
Warning: The model gat cannot be found at core set.
Traceback (most recent call last):
File "D:\my git repo\benchmark\run.py", line 311, in <module>
Model = load_canary_model_by_name(args.model)
File "D:\my git repo\benchmark\torchbenchmark\__init__.py", line 625, in load_canary_model_by_name
module = importlib.import_module(f'.canary_models.{model}', package=__name__)
File "C:\Users\temp\anaconda3\envs\pytorch\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "D:\my git repo\benchmark\torchbenchmark\canary_models\gat\__init__.py", line 1, in <module>
from torchbenchmark.util.framework.gnn.model_factory import GNNModel
File "D:\my git repo\benchmark\torchbenchmark\util\framework\gnn\model_factory.py", line 6, in <module>
from torch_geometric.nn import GAT, GCN, GraphSAGE
ModuleNotFoundError: No module named 'torch_geometric'
```
Test case 4: run a non-exist model
```
(pytorch) PS D:\my git repo\benchmark> python run.py abcde
Warning: The model abcde cannot be found at core set.
Error: The model abcde cannot be found at either core or canary model set.
```
Reviewed By: aaronenyeshi
Differential Revision: D45607559
Pulled By: xuzhao9
fbshipit-source-id: bd9bd1f13dc54dd05e296e224cf64daaf6e845f1
benchmark
4c943538 - Improve error message and print stack backtrace when failing to load models and canary_models (#1615)
