benchmark
64f199d5 - Move unbacked-parity regression policy into the benchmark; widen MobileBert (#189813)

Commit
92 days ago
Move unbacked-parity regression policy into the benchmark; widen MobileBert (#189813) Summary: The periodic inductor_huggingface_unbacked_parity job intermittently reds on MobileBertForMaskedLM with no real regression behind it. Its backed and unbacked compiled times are ~equal and land in a bimodal distribution independently, so the backed-vs-unbacked diff is ~+-2% autotune/measurement jitter -- centered at zero over 16 independent runs, and a pre-onset build reproduces the same variance, so it is not a code regression. Move the regression policy out of the CI shell script and into the benchmark (_run_compare_backed_unbacked in benchmarks/dynamo/common.py), next to the comparison itself, as a per-model threshold table (MobileBertForMaskedLM = 3.0%, others 1.0%). The benchmark now emits a canonical UNBACKED_PARITY_REGRESSION / UNBACKED_PARITY_OK verdict line, and .ci/pytorch/test.sh detects that line instead of re-deriving the threshold in bash. X-link: https://github.com/pytorch/pytorch/pull/189813 Approved by: https://github.com/bobrenjc93 Reviewed By: jeanschmidt Differential Revision: D112002277 fbshipit-source-id: 8753da02d79cbf776ab1c82a00339a1c18d7c7dd
Author
Committer
Parents
Loading