gate the pregraph profiler marker on the active profiler (#190623)
Summary:
Part of #188077 (per-call overhead). Dynamo's codegen brackets the pre-graph
arg reconstruction of every compiled function with two
record_pregraph_bytecode_enter/exit calls (the "Pregraph bytecode" profiler
marker). Those were emitted unconditionally and each wrapped in
torch._disable_dynamo, so every compiled call paid ~9,200 instructions for them
even with no profiler attached -- the marker is a nullcontext when not
profiling. That is ~10% of a trivial compiled call and the single largest
avoidable slice of the fixed per-call overhead.
Rather than defaulting the feature off (which drops the marker entirely unless
you opt in and recompile), keep config.record_runtime_overhead on and emit the
marker behind a RUNTIME profiler gate in the generated pre-graph bytecode:
cm = None
if torch.autograd.profiler._is_profiler_enabled:
cm = record_pregraph_bytecode_enter()
<reconstruct graph args>
if cm is not None:
record_pregraph_bytecode_exit(cm)
A non-profiled call now pays ~nothing (a flag load + branch): measured ~9,200 ->
~700-1000 instr/call on a trivial compiled function, a >90% reduction. A profiler
attached at ANY point -- including after the function was already compiled --
still records, with no recompile, because the gate reads
torch.autograd.profiler._is_profiler_enabled live on each call.
The two `if` gates are emitted via bytecode_from_template (two small module-level
templates) instead of hand-emitted jump opcodes: bytecode_from_template compiles
the template on the running interpreter, so the branch lowers to the correct jump
on every supported Python version -- a hand-emitted POP_JUMP_IF_FALSE /
POP_JUMP_IF_NONE would KeyError on 3.10-3.12, where those opnames differ. The
caller pre-loads the template locals (the profiler flag and the
torch._dynamo.utils module, reconstructed inline so they are not CSE tempvars
that clear_tempvars() would DELETE_FAST on the branch's skip path) and shares one
cm var and the module between the enter and exit gates. The marker fns are looked
up on the module INSIDE each gate, so their attr loads only run when profiling.
The marker fns themselves (record_pregraph_bytecode_enter/exit) are skip_code'd
(recursively) rather than torch._disable_dynamo-wrapped, so when a profiler IS
attached each marker call is skipped by the eval-frame hook via a static flag
instead of paying a per-call disable wrapper. Recursive skip gives the same
"never trace the marker's dynamic extent" guarantee the disable wrapper did, but
for free -- the recursive skip is just a flag, no per-call cost, so there is no
reason to leave callees traceable. skip_code is applied via the C
set_code_exec_strategy because torch._dynamo.disable / .skip cannot be imported
at utils import time.
test_bytecode_debugger disables record_runtime_overhead at the class level: its
tests step through / count the exact pre-graph bytecode of a compiled function,
and the gated marker's un-executed branch would otherwise perturb those instruction
counts and opname sequences.
The AOTDispatcher-side marker (in the AOTAutograd runtime wrapper) already gates
on _is_profiler_enabled at runtime; this brings the Dynamo pre-graph side to
parity, so record_runtime_overhead stays on by default at ~zero idle cost.
X-link: https://github.com/pytorch/pytorch/pull/190623
Approved by: https://github.com/anijain2305, https://github.com/guilhermeleobas
Reviewed By: atalman
Differential Revision: D115607762
fbshipit-source-id: f9746ad439c5caf3b17013e952aa411d95eefa13