[quant][graphmode][fix] use observed_values_ to check values are observed (#34571)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34571
Previously we added wrong values to observed_values_ and also it is
not used to check if a value is observed or not
Test Plan:
.
Imported from OSS
Differential Revision: D20519605
fbshipit-source-id: 6038b2539bcf7d679b7fe5c5a284b81a979934ee
