fx quant: unbreak Mac OS tests when MKLDNN is available
Summary:
In https://github.com/pytorch/pytorch/pull/74137, the MKLDNN
quantized backend was added to PyTorch.
Sometime in the past couple of days, MKLDNN got enabled on my Mac OS
machine. This uncovered issues in FX graph mode quantization testing,
as we were only testing for fbgemm and qnnpack, and some of the tests
that were assuming fbgemm started silently going through the MKLDNN
path. Since the requirements for MKLDNN are different, the tests started
to fail.
This PR unbreaks the minimal amount of tests to get a clean
test run on my machine.
In the future, it would be great to add testing for MKLDNN specifically,
and also audit all of the current quantization tests which are assuming
fbgemm to set the backend properly.
Test plan:
```
python test/test_quantization.py -k Fx
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79718
Approved by: https://github.com/jerryzh168
