fix device check in op bench (#29918)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/29918
Some of the tests don't specify `device` in the input configs so filter by device won't work for them. This diff fixes that issue.
Test Plan:
```
buck run mode/opt //caffe2/benchmarks/operator_benchmark/pt:qpool_test -- --iterations 1 --device cpu
# ----------------------------------------
# PyTorch/Caffe2 Operator Micro-benchmarks
# ----------------------------------------
# Tag : short
# Benchmarking PyTorch: QAdaptiveAvgPool2dBenchmark
# Mode: Eager
# Name: QAdaptiveAvgPool2dBenchmark_N4_C3_input_size(224,224)_output_size(112,112)_contigTrue_dtypetorch.qint32
# Input: N: 4, C: 3, input_size: (224, 224), output_size: (112, 112), contig: True, dtype: torch.qint32
Forward Execution Time (us) : 2891.172
Reviewed By: hl475
Differential Revision: D18535766
fbshipit-source-id: 09d89cf23b3caab6c0bc3b8a9ae55cc439b98e0f
