benchmark

34ccb0aa - Make uncaught doctest warnings a hard error, after quieting the noise (#191416)

Make uncaught doctest warnings a hard error, after quieting the noise (#191416) Summary: Human: I got annoyed that Dr. CI kept misclassifying doctest problems. Part of it is a huge amount of suppressed error and warning spam. This makes doctests warning clean and then makes warnings in doctest a hard error. AI summary: The doctest CI job's log is cluttered with warning/exception noise that obscures real failures, and nothing prevents new noise from creeping in. This first quiets every warning the job emits, then flips uncaught doctest warnings from silently-reported to a hard failure so the log stays clean. Review in three parts. Part 1 -- self-inflicted warnings (PyTorch's own code trips its own warning, unrelated to what the example demonstrates): - CleanupHook.__call__ did `del self.scope[self.name]`. During interpreter shutdown the module __dict__ holding the installed global is often already torn down, so the del raises KeyError inside a GC weakref callback where it cannot propagate; CPython then prints it via the unraisable hook as "Exception ignored in weakref callback". The method already guards for shutdown (`if CleanupManager is not None`) but not the delete, so switch to `scope.pop(name, None)`. - LeafSpec is deprecated(FutureWarning), so copy.deepcopy/pickle round-trips re-invoke its constructor and leak the "isinstance(treespec, LeafSpec) is deprecated" FutureWarning to code that never wrote an isinstance check (it fired from copyreg during a doctest). Add __reduce__ returning (treespec_leaf, ()) so round-trips rebuild via the factory and reuse the shared singleton, emitting no warning. Part 2 -- by-design warnings, where the example itself exercises a deprecated/prototype/beta API. The rule applied per docstring: if the example documents a deprecated API, silence just its own warning inline; if it documents a legit feature that happens to reach for a discouraged path, fix the example instead. Inline suppression uses a `>>> warnings.filterwarnings(...) # docs: hide` line: xdoctest executes it (so the warning leaves its run-time warning report) but the existing process_docstring hook in conf.py strips `# docs: hide` lines from Sphinx output, exactly as it already strips `# xdoctest:` directives. Suppression is scoped per-doctest by xdoctest's own catch_warnings, so it does not leak. `import warnings` is added to the doctest prologue so hidden lines need no visible import. Suppressed (deprecated/prototype/beta API is the point of the example): library.Library._impl_with_aoti_compile, quantized Embedding and functional.conv{1,2,3}d (eager-quant quint8 creation), nn.utils.weight_norm / remove_weight_norm (the documented fn is itself deprecated), functorch jvp / linearize (benign internal leaks), sparse.as_sparse_gradcheck (CSR beta), functional._lu_impl (torch.lu deprecation). Fixed to use the recommended path: nested.as_nested_tensor and nested_tensor (rewritten to layout=torch.jagged), sparse.sum (wrapped in check_sparse_tensor_invariants, the opt-in the warning points to), nn.Transformer / TransformerEncoder (batch_first=True to silence the enable_nested_tensor notice). The jagged rewrite bumps the process-global ragged-dim counter, so three sibling `jN` wants in torch/nested were made robust with `# xdoctest: +ELLIPSIS` and `j...`. Part 3 -- the gate. run_test.py::run_doctests only failed on n_failed; a doctest that passed but emitted a run-time warning returned success. It now also fails when run_summary reports n_warned > 0, printing a message that points the next contributor at both remedies (fix the example, or add a `# docs: hide` suppression). With Parts 1-2 landed the CI-executed doctest set reports n_warned == 0, so the gate passes today and only trips on regressions. X-link: https://github.com/pytorch/pytorch/pull/191416 Approved by: https://github.com/drisspg Reviewed By: huydhn Differential Revision: D114083573 fbshipit-source-id: 1ca99369f7f5d91edcc5cddd4ab22d80e80063ce