do not cast all inputs in benchmarks (#108456)
Fixes why stable diffusion is not showing up in inference dashboard even though it shows up in training dashboard
The reason is stable diffusion in torchbench has a line like `input_tensor = input_tensor.long().to(self.device)` and if you cast this to a bfloat16 the inference will fail
<img width="1705" alt="Screenshot 2023-09-01 at 4 37 49 PM" src="https://github.com/pytorch/pytorch/assets/3282513/ada0d381-1af0-4378-8e8b-2375b39c3713">
Pull Request resolved: https://github.com/pytorch/pytorch/pull/108456
Approved by: https://github.com/cpuhrsch
