benchmark
24755807 - gate the pregraph profiler marker on the active profiler (#190623)

Commit
102 days ago
gate the pregraph profiler marker on the active profiler (#190623) Summary: Part of #188077 (per-call overhead). Dynamo's codegen brackets the pre-graph arg reconstruction of every compiled function with two record_pregraph_bytecode_enter/exit calls (the "Pregraph bytecode" profiler marker). Those were emitted unconditionally and each wrapped in torch._disable_dynamo, so every compiled call paid ~9,200 instructions for them even with no profiler attached -- the marker is a nullcontext when not profiling. That is ~10% of a trivial compiled call and the single largest avoidable slice of the fixed per-call overhead. Rather than defaulting the feature off (which drops the marker entirely unless you opt in and recompile), keep config.record_runtime_overhead on and emit the marker behind a RUNTIME profiler gate in the generated pre-graph bytecode: cm = None if torch.autograd.profiler._is_profiler_enabled: cm = record_pregraph_bytecode_enter() <reconstruct graph args> if cm is not None: record_pregraph_bytecode_exit(cm) A non-profiled call now pays ~nothing (a flag load + branch): measured ~9,200 -> ~700-1000 instr/call on a trivial compiled function, a >90% reduction. A profiler attached at ANY point -- including after the function was already compiled -- still records, with no recompile, because the gate reads torch.autograd.profiler._is_profiler_enabled live on each call. The two `if` gates are emitted via bytecode_from_template (two small module-level templates) instead of hand-emitted jump opcodes: bytecode_from_template compiles the template on the running interpreter, so the branch lowers to the correct jump on every supported Python version -- a hand-emitted POP_JUMP_IF_FALSE / POP_JUMP_IF_NONE would KeyError on 3.10-3.12, where those opnames differ. The caller pre-loads the template locals (the profiler flag and the torch._dynamo.utils module, reconstructed inline so they are not CSE tempvars that clear_tempvars() would DELETE_FAST on the branch's skip path) and shares one cm var and the module between the enter and exit gates. The marker fns are looked up on the module INSIDE each gate, so their attr loads only run when profiling. The marker fns themselves (record_pregraph_bytecode_enter/exit) are skip_code'd (recursively) rather than torch._disable_dynamo-wrapped, so when a profiler IS attached each marker call is skipped by the eval-frame hook via a static flag instead of paying a per-call disable wrapper. Recursive skip gives the same "never trace the marker's dynamic extent" guarantee the disable wrapper did, but for free -- the recursive skip is just a flag, no per-call cost, so there is no reason to leave callees traceable. skip_code is applied via the C set_code_exec_strategy because torch._dynamo.disable / .skip cannot be imported at utils import time. test_bytecode_debugger disables record_runtime_overhead at the class level: its tests step through / count the exact pre-graph bytecode of a compiled function, and the gated marker's un-executed branch would otherwise perturb those instruction counts and opname sequences. The AOTDispatcher-side marker (in the AOTAutograd runtime wrapper) already gates on _is_profiler_enabled at runtime; this brings the Dynamo pre-graph side to parity, so record_runtime_overhead stays on by default at ~zero idle cost. X-link: https://github.com/pytorch/pytorch/pull/190623 Approved by: https://github.com/anijain2305, https://github.com/guilhermeleobas Reviewed By: atalman Differential Revision: D115607762 fbshipit-source-id: f9746ad439c5caf3b17013e952aa411d95eefa13
Author
Committer
Parents
Loading