jax
11d2dac8 - [state] check vma when reading from a Ref at a varying index

Commit
23 days ago
[state] check vma when reading from a Ref at a varying index Under check_vma=True, reading from a Ref at an index that varies over manual axes the Ref doesn't vary over, e.g. `r[jax.lax.axis_index('x')]` for an invariant Ref, typed the result as invariant even though each device reads a different element. That was only caught at jit lowering, when discharging Refs retypes the jaxpr, with an internal error that doesn't say what's wrong: ```python @jax.jit @jax.shard_map(in_specs=(), out_specs=P()) def f(): r = jax.new_ref(jnp.arange(4.)) return r[jax.lax.axis_index('x')] f() # main: _RepError: ([ManualAxisType(varying=frozenset({'x'}), ...)], PyTreeDef([*])) # this PR: ValueError: Primitive get requires the index's varying manual axes to be a subset of the Ref's for inexact dtypes, but got frozenset({'x'}) for the index and frozenset() for the Ref of dtype float32. To read at this index, read the whole Ref, mark it as varying, then index the resulting array, e.g. `jax.lax.pcast(r[...], ('x',), to='varying')[i]` instead of `r[i]`. ``` Now, for inexact dtypes, get requires the index's vma to be a subset of the Ref's, as swap and addupdate already do (#40997). While we could implicitly compute a varying out aval for the read, its transpose would need a Ref-sized scatter followed by a psum, which we'd rather the user perform explicitly with `jax.lax.pcast(r[...], axes, to='varying')[i]`. For non-differentiable dtypes there's no cotangent to psum, so like indexing an array, the read's out aval varies over both the Ref's and the index's manual axes. PiperOrigin-RevId: 1010019785
Author
Parents
Loading