Seed the recorded import aliases and builtins-dict key into the guard scope on load (#196817)
Summary:
When Dynamo traces a function it mints synthetic names into the TRACING
process's `fn.__globals__` and roots guards at them. Two kinds matter here:
```python
class M(torch.nn.Module):
def __init__(self):
super().__init__(); self.lin = torch.nn.Linear(4, 4)
def forward(self, x):
if not isinstance(x, torch.Tensor): # a builtin reference
raise TypeError(type(x))
return self.lin(x) # a child-module call
torch.compile(M(), backend="eager")(torch.randn(2, 4))
g = M.forward.__globals__
[k for k in g if k.startswith("__import_")] # ['__import_torch_dot_nn_dot_modules_dot_module', ...]
[k for k in g if k.startswith("__builtins_dict__")] # ['__builtins_dict___0']
g["__builtins_dict___0"] is builtins.__dict__ # True: the LIVE builtins dict
```
The guards read those names through `G[...]`:
```
G['__import_torch_dot_nn_dot_modules_dot_module']._global_forward_hooks # DICT_LENGTH == 0
___check_obj_id(G['__builtins_dict___0']['isinstance'], 140012...) # BUILTIN_MATCH, an ID_MATCH into the dict
```
A process that only LOADS an `aot_compile` artifact never traced, so its scope
has none of those names. The default `guard_filter_fn` drops global guards
wholesale, which is why this stayed hidden; keep them and the load dies:
```python
compiled = torch.compile(m, fullgraph=True, backend="eager",
options={"guard_filter_fn": keep_global_guards}).aot_compile(((x,), {}))
compiled.save_compiled_function("m.pt")
# ---- a fresh process ----
loaded = AOTCompiledFunction.deserialize(open("m.pt", "rb").read())
loaded(x)
# RuntimeError: ... KeyError on G['__import_torch_dot_nn_dot_modules_dot_module']
```
In-process tests could not see this: the capture had already leaked the
aliases into the module dict, so the load found them there.
## Who needs a global guard on a loaded artifact
A model that reads module-level state in `forward` is specialized on that
state at trace time, and the global guards are the only record of it:
```python
EPS = 1e-5 # baked into the graph as a constant
USE_FUSED = True # picks a branch at trace time
FREQS = torch.randn(4096, 64) # lifted as a graph input; dtype/shape/device guarded
def forward(x):
if USE_FUSED: ...
return (x / (x.norm() + EPS)) * FREQS[:x.shape[1]]
```
Under the default filter a loading process that sets `EPS = 1e-3`, flips
`USE_FUSED`, or binds a bf16 `FREQS` is served the traced graph anyway, with no
error. That is the right default for an artifact shipped to a different
codebase, and this commit leaves it alone. It is the wrong trade for the other
common deployment, precompile as a warm cache across processes running the SAME
code, where the module layout matches and there is no reason to give up the
safety eager `torch.compile` has; `CompilePackage` already lives there and
keeps every guard. `torch.compile` exposes the choice through
`guard_filter_fn` and ships helpers for it
(`keep_global_context_and_tensor_guards_unsafe`, `keep_tensor_guards`,
`skip_guard_on_globals`). Before this commit the choice did not work: keep a
global guard and the load crashed on the alias, or the guard resolved against
the snapshot the tracing process serialized, which agrees with itself and so
can never fail. An offered option has to either work or refuse.
Some `G[...]` readers survive even the default filter, because `is_global`
looks only at `originating_source`: a `SHAPE_ENV` guard on a dynamic-dim
module-level tensor, and a `DUPLICATE_INPUT` whose `source_b` is a global. The
shape-guard case raised `KeyError` on every default load before this commit, so
the fix below reaches artifacts that never touched the filter.
## Seed the guard scope at load, gated per name
`AOTCompiledFunction.__post_init__` now calls `_seed_guard_scope` before
`load_guard_manager`, writing into whichever dict the guards will resolve
against. In outline:
```python
def _seed_guard_scope(guard_scope, output_graph):
# aliases: the artifact recorded {alias: module_name} in runtime_env.import_sources
for alias, module_name in runtime_env.import_sources.items():
if alias in output_graph.global_scope and alias not in guard_scope:
guard_scope[alias] = importlib.import_module(module_name)
# builtins key: only when a deserialized guard is rooted at exactly that name
roots = {get_global_source_name(g.originating_source) for g in output_graph.guards}
if builtins_key in roots:
CleanupHook.disown(guard_scope, builtins_key)
... # the re-derive below
guard_scope.setdefault("__builtins__", builtins.__dict__)
guard_scope[builtins_key] = get_builtins_dict(guard_scope)
```
Each name is gated on a kept guard being rooted at THAT name, not on the
artifact having some global-rooted guard, because this writes into a dict that
may be a user module's live namespace and installs no `CleanupHook`. A gate
that coarse pays exactly the cost it exists to avoid in the case it admits: an
artifact whose only global guard is on a user global would still write the
builtins key into the caller's namespace, and would `disown` a name a live
compile in that module owns.
The two gates differ because the serializer records the two names differently:
- **Aliases** are gated on the serialized `global_scope`, which the serializer
already pruned to the names the kept guards resolve at check time. That set is
wider than the guards' `originating_source`s: a `DUPLICATE_INPUT` reads its
`source_b` and a `SHAPE_ENV` guard reads the shape-env sources, and a scope
lacking either fails with the same `KeyError on G[...]` (measured). That is
exactly the set an alias must be seeded for. An alias can only be missing from
a caller-supplied scope, since `GraphRuntimeEnv.forward_callable` builds
`fn.__globals__` with every recorded alias already imported.
- **The builtins key** cannot use that gate: the serializer writes it into the
pruned scope whether or not a guard reads it. So it is matched against the
roots of the deserialized guards. The two wider channels never root at the
builtins key: `load_builtin_from_argval` is the only place a source under that
key is minted, and `guard_on_key_order` cannot name it, since every site that
fills it adds the source of a dict or set variable. That is a structural
property, so `test_builtins_key_gate_covers_the_other_serializer_channels`
pins it: one artifact that really populates both channels (a non-empty
`shape_env_sources`, and a `DUPLICATE_INPUT` whose `source_b` is a
`GlobalSource`) and neither yields the builtins key. It never loads, so it
passes unchanged on the parent; it guards the serializer property the gate
relies on.
A binding this process already had is left alone: a wrong binding FAILS a
guard rather than passing it, and a caller-supplied scope may legitimately
carry these names. That deliberately differs from `install()`'s builtins branch
in package.py, which raises on a mismatch. `CleanupHook.disown` still runs on
the builtins key even when its value is kept: a pre-reset compile may own the
name and would delete the binding when its hook is collected.
`test_load_disowns_only_a_builtins_key_a_guard_reads` pins both directions
against a live compile that minted the artifact's exact key name. The aliases
need no `disown`: `import_source` and `CompilePackage._install_global` bind one
by plain dict assignment, and `install_global_unsafe`, the only site that
creates a `CleanupHook`, never installs an alias.
## The builtins key must point at LIVE builtins, not a snapshot
On the DEFAULT load path the key is often already present, and that is the
second bug. When the generated bytecode itself reads a builtin (one that
survived into the output instead of being constant-folded away), the
reconstruction goes through `DictGetItemSource(GlobalSource(key))`, so
`get_runtime_env` records the key in `used_globals` holding a pickle-FILTERED
COPY of the tracing builtins. `forward_callable` spreads `used_globals` into
`fn.__globals__`, which on the default path IS the guard scope. The old
`builtins_key not in guard_scope` gate bailed, and the guard compared
identities inside a frozen snapshot:
```python
def fn(x):
if isinstance(x, torch.Tensor): # constant-folded; its BUILTIN_MATCH is kept via keep_builtin_guards
return len # `len` survives, so the bytecode reads G['__builtins_dict___0']['len']
loaded = AOTCompiledFunction.deserialize(data) # default path
real = builtins.isinstance
builtins.isinstance = lambda o, t: real(o, t) # behaviorally identical wrapper
loaded.guard_check(x)
# parent: True (ID_MATCH against the frozen copy, which still holds the old object)
# now: False (the scope holds builtins.__dict__ itself, so the rebind is seen)
```
A kept guard that cannot fail is the whole bug: the artifact stops noticing
the one change it was kept to detect, on the ORDINARY load. So the key is
re-derived when, and only when, the value bound there `is` the artifact's own
recording (`runtime_env.used_globals.get(builtins_key)`, compared by identity).
Any other binding, a caller's copy of the real builtins included, is a value
this process chose and stays:
```python
scope = {builtins_key: dict(builtins.__dict__)}
loaded = AOTCompiledFunction.deserialize(data, guard_globals=scope)
scope[builtins_key] is builtins.__dict__ # False: the caller's dict is kept
"__builtins__" in scope # False: nothing extra was written
```
`test_load_rederives_a_builtins_key_bound_to_its_own_snapshot` pins both arms
on an artifact whose bytecode really reads the key (it asserts the key is in
`bytecode.co_names` and in `used_globals` before it goes near a load).
Two consequences of the re-derive, both deliberate. First, `__builtins__` is
seeded with the live `builtins.__dict__` rather than a copy, for the same
identity reason: seeding `builtins.__dict__.copy()` leaves `guard_check` True
after the `isinstance` rebind (measured). That branch fires only when the scope
has no `__builtins__` at all, which a real module namespace always has (CPython
binds it to `builtins.__dict__` itself); the scope that lacks it is
`fn.__globals__`, a plain dict literal from `forward_callable`. The
`builtins.__dict__.copy()` in `GuardBuilder.__init__` is not a precedent: it
copies so the torch.package loop below it can write demangled module names
without touching real builtins. Second, the recording was filtered for
picklability alone, never narrowed to what this process has, so a builtin the
tracing process had and the loading one lacks stops being readable: with a
kept guard on that name the load says so, ``KeyError on
G['__builtins_dict___0']['aot_probe']`` out of `guard_check`, and with that
guard filtered out the read raises `KeyError` from the generated bytecode.
`test_load_rederives_over_a_builtin_the_loading_process_lacks` pins both arms.
Binding a dict that merges the recording over live builtins would satisfy both
readers on paper and is the frozen-snapshot bug again for every shared name
(measured: it takes `guard_check` after the rebind back to True).
The seeding runs AFTER `forward_callable` on purpose: `PyFunction_New` caches
`__builtins__` at creation, so writing `__builtins__` afterwards cannot rewire
what the compiled bytecode resolves against (measured on 3.12: rebinding after
creation is inert, binding before creation decides the lookup). The builtins
KEY is the exception, and deliberately so: it is an ordinary global that
`LOAD_GLOBAL` resolves live on every call, so the re-derive also decides what
the generated bytecode subscripts, and it can only fire when that bytecode
reads the key, since that read is what makes `get_runtime_env` record the copy.
## `guard_globals=`: a caller-controlled guard scope
`deserialize` grows a keyword-only parameter whose semantics differ from
`f_globals`:
```python
# f_globals is MERGED over the scope rebuilt from the artifact: a name it omits
# still resolves to the baked-in value.
AOTCompiledFunction.deserialize(data, f_globals={"EPS": 1e-3})
# guard_globals REPLACES the guard scope, no fallback. A missing name fails the
# guard, and an EMPTY dict is an empty scope rather than "no scope".
loaded = AOTCompiledFunction.deserialize(data, guard_globals={})
loaded(x) # RuntimeError: KeyError on G['EPS'] (for an artifact whose kept guard reads EPS)
# The load WRITES its seeds into the dict passed, so pass the one they should land in.
scope = {"__name__": "fresh_module"}
loaded = AOTCompiledFunction.deserialize(data, guard_globals=scope)
builtins_key in scope, "__builtins__" in scope # (True, True) when a kept guard reads them
```
In practice the caller hands over the live namespace that defines those
globals, not a hand-built dict:
```python
# deployment process, in the module that binds EPS / USE_FUSED / FREQS
loaded = AOTCompiledFunction.deserialize(data, guard_globals=sys.modules[__name__].__dict__)
loaded(x) # refuses: the kept guard on EPS reads 1e-3 here, the graph was built for 1e-5
```
The guards check what that module binds NOW, on every call, so a later rebind of
`USE_FUSED` is caught too. The load seeds the synthetic `__import_*` and
`__builtins_dict___N` names into that module dict on the caller's behalf, which
is the case the per-name gating and `CleanupHook.disown` below exist for: a real
namespace is being written to. A hand-built dict works as well, and an omission
is loud rather than silent:
```python
scope = {"EPS": EPS, "USE_FUSED": USE_FUSED} # forgot FREQS
loaded = AOTCompiledFunction.deserialize(data, guard_globals=scope)
loaded(x) # RuntimeError: ... KeyError on G['FREQS']
```
There is no fallback to the baked-in value, so the failure names the missing
global and the fix is adding the key; the synthetic names are never the
caller's to add. Passing nothing leaves a kept global guard vacuous: the
default guard scope is the reconstructed `fn.__globals__`, which holds the
values recorded at trace time, so the guard compares the recording to itself.
A failed guard is a refusal, since `aot_compile` has no recompile path, and
that is the correct outcome: `EPS` is a constant inside the graph, so no scope
can make the loaded artifact compute with `1e-3`; the artifact was built for a
different configuration and says so. `f_globals` is not the tool for this: it
changes what the bytecode reads and is merged into the default guard scope, so
it can fail a guard but cannot change a baked constant.
`torch.compiler.load_compiled_function` does not forward `guard_globals` yet,
so today it is reachable only through `AOTCompiledFunction.deserialize`; the
public plumbing is left for a follow-up (see the FAQ).
It is held by reference and never merged into `fn.__globals__`, so a
caller-supplied scope cannot rewire what the compiled bytecode reads. The load
refuses a `guard_globals`/`f_globals` whose `__builtins__` is neither a dict
nor a module, naming the parameter it arrived by and the offending TYPE:
```python
AOTCompiledFunction.deserialize(data, guard_globals={"__builtins__": None})
# TypeError: guard_globals['__builtins__'] must be a dict or a module, got NoneType
```
The parameter name is conditional because `f_globals` is merged into
`fn.__globals__` before the function is built, so the default scope can carry a
bad binding too; the value is not printed because a repr on a load failure path
runs user code. The check needs only `guard_scope.get("__builtins__")`, so it
is hoisted above the alias writes and above `disown`, and runs whenever a kept
guard roots at the key rather than only when the key is about to be derived.
Otherwise a refused load has already bound every guarded alias into the
caller's dict and stripped a live compile's ownership of the key, and a caller
who pre-binds the key alongside a bad `__builtins__` loads successfully. The
last arm of `test_load_refuses_a_non_dict_builtins_binding` pins both against
a scope where a `CleanupHook` owns the key.
## Three places still resolved `G` against the serialized snapshot
Re-rooting the guards at a caller's dict exposed three sinks that were not
reading the runtime scope.
**A falsy scope was "no scope".** `GuardBuilder.__init__` took
`runtime_global_scope or global_scope`, so `guard_globals={}` was silently
replaced by the globals serialized with the artifact, the baked-in resolution
this path exists to forbid, and whether that happened depended on whether the
seeding above had made the dict truthy. It now tests `is None`, which is what
"no scope supplied" always meant: the eager path passes `None` and is
unchanged, and `CompilePackage.install()` passes `sys.modules[...].__dict__`,
which is never empty. `load_guard_manager` says so at its own signature, since
it is the sink that receives the choice.
`test_load_into_an_empty_scope_refuses_a_lifted_global` pins it: an artifact
whose kept guard reads a global the graph LIFTED, loaded with
`guard_globals={}`, has to refuse on `KeyError on G['EPS']`. The old fallback
holds a FakeTensor for that global, so its `TENSOR_MATCH` fails on the type
instead, which the regex also rejects; the fallback answers on a baked-in value
only for a specialized global the serializer kept a real value for, which is
the `AOT_POOL_MODE` arm of `test_load_seeds_no_name_no_guard_is_rooted_at`.
**Python lambda shape guards.** A `SHAPE_ENV` guard roots at a
`ShapeEnvSource`, which is not global, so the default filter KEEPS it, and with
`enable_cpp_symbolic_shape_guards` off (the default) it installs as one Python
lambda over its code parts, which carry a literal `G['NAME']` for a global with
a dynamic dim. `add_python_lambda_leaf_guard_to_root` exec'd that lambda over
the SERIALIZED `global_scope`, so it never saw `runtime_global_scope`:
```python
# default filter: the global is pruned from the serialized scope, so nothing names it
loaded(x) # KeyError on 'AOT_DYN_ROWS' on EVERY load, the default one included
# kept global guard: the serialized scope holds a FakeTensor with the TRACED sizes
loaded.guard_check(torch.randn(4, 5)) # passed against a traced (4, 3), whatever the caller's dict held
```
The lambda now closes over `runtime_global_scope`, the dict the C++ tree is
rooted at; on the eager path that is `global_scope` itself, so nothing there
changes. `CompilePackage.install()` gets the same fix through the same line,
and there it is the silent arm: a package keeps every guard, so a shape
constraint on a dynamic-dim module-level global was checked against the
pickled FakeTensor and passed whatever the module bound. A one-row global was
served the graph built for `2 <= rows` (measured under `fail_on_recompile`);
the fix turns that stale hit into a recompile.
`test_installed_shape_guard_on_a_global_reads_the_live_module_dict` pins that
half through `DiskDynamoStore` save/load and `install()`, anchored on
`verbose_code_parts=["2 <= G['PKG_DYN_ROWS']` because the stance message
prints the whole guard tree before the parts that failed. Both arms rebind the
global to a fresh tensor because the loaded `TENSOR_MATCH` rejects the marked
original: the pickled FakeTensor carries no `_dynamo_dynamic_indices`, so the
load guards `hasattr(G['PKG_DYN_ROWS'], '_dynamo_dynamic_indices') == False`.
`test_load_resolves_a_python_shape_guard_against_the_guard_scope` pins the
default-filter case and
`test_load_python_shape_guard_cannot_pass_on_the_serialized_scope` the silent
arm; on the parent the first fails its default load with `GuardManager check
failed ... ["'AOT_DYN_ROWS'"]` and the second with `RuntimeError not raised`.
**Fail reasons.** `guard_manager.global_scope` is the dict
`get_guard_fail_reason_helper`, `recompilation_reason_for_no_tensor_aliasing_guard`
and `guard_error_hook` evaluate failed code parts against. None is reachable
from a loaded artifact today (the first two run over eager `cache_entries` and
the `skip_guards_check=True` sanity check a deserialized state disables; the
hook fires from `lookup_in_list` while precompile entries go through
`run_root_guard_manager`), so leaving it on the serialized dict was a latent
split rather than a live bug. But it was the one `G` that disagreed with both
the lambda and the accessor tree, and wiring recompile reasons to precompile
entries would have reintroduced what this commit closes. It is re-pointed the
same way; on the eager path `guard_manager.global_scope["G"] is fn.__globals__`
before and after (measured).
## Interaction with a later compile in the same module
Seeding a captured `__builtins_dict___N` into a live module dict puts a name in
the way of a later compile in that module, whose `unique_id` counter is still
behind and will regenerate it. The mint sites below this commit already skip
forward past a name that is taken (`unique_id_unbound_in`), so that
load-then-compile does not trip `CleanupHook.create`'s "already exists in
scope" assertion. That sequence is pinned here rather than where the skip
lands, since only this commit has a load that does the seeding: the
`guard_reads_builtins` arm of `test_load_disowns_only_a_builtins_key_a_guard_reads`
compiles again, under a rewound counter, in a module dict where the key is
already bound by the earlier live compile and the load only disowned it.
## Review order
Read the guard scope `__post_init__` chooses first, then `_seed_guard_scope`,
then the `is None` test in `GuardBuilder.__init__` that lets an empty scope
survive the trip, then the `G` a Python lambda guard closes over in
`add_python_lambda_leaf_guard_to_root` and the one `guard_manager.global_scope`
keeps for fail reasons, then the tests.
`test_builtins_key_gate_covers_the_other_serializer_channels` runs with
`assume_static_by_default=False` so the shape-env channel carries a
GLOBAL-rooted source (`AOT_DUPE_A`), asserted alongside the dupe half; without
the flag it fails `'AOT_DUPE_A' not found in set()`. The no-overwrite arms of
`test_load_keeps_a_prebound_builtins_scope` and
`test_load_rederives_a_builtins_key_bound_to_its_own_snapshot` pre-bind through
a `CleanupHook` and fire it after the load, so a `_seed_guard_scope` that never
runs on a caller scope fails them with `never disowned` / `the load replaced,
or never disowned, the builtins dict it was handed` (checked by mutation).
X-link: https://github.com/pytorch/pytorch/pull/196817
Approved by: https://github.com/pytorchgreenlight
ghstack dependencies: #196767
Reviewed By: atalman
Differential Revision: D120011425
fbshipit-source-id: 8bfdcc5bb2d486679bed1617894d090a94df299e