Revert D33024528: [quant][fx][graphmode] Add support for conv add pattern in backend_config_dict
Test Plan: revert-hammer
Differential Revision:
D33024528 (https://github.com/pytorch/pytorch/commit/59000cff9130afb5168b4401b9e6e8affecfbdb5)
Original commit changeset: 5c770c82c8f6
Original Phabricator Diff: D33024528 (https://github.com/pytorch/pytorch/commit/59000cff9130afb5168b4401b9e6e8affecfbdb5)
fbshipit-source-id: 7da6f421ef63f47fbffad8b3ad91f6a31d19d867
