Standardise Aria's MoE onto the experts interface (#48907)
* Standardise Aria's MoE onto the experts interface
Aria's routed experts predate the standard `use_experts_implementation`
interface. They were a Megatron-style grouped GEMM (`AriaGroupedExpertsGemm`
over `fc1`/`fc2`) that did its own routing inside `AriaExperts.forward` and
synced to host on every call via `tokens_per_expert.cpu()`.
Replace them with `AriaTextTopKRouter` + `AriaExperts`, the same shape
GraniteMoe and Qwen3-MoE use: the router returns
`(top_k_index, top_k_weights, router_logits)` and the experts take them as
arguments. Aria's checkpoints already store each expert's projections as
(in_features, out_features), which is `grouped_mm`'s native layout, so the
experts are declared `is_transposed=True` and the legacy weights map across
by rename alone.
Outputs are bit-identical on existing checkpoints.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Skip Aria's two pre-existing export failures
Standardising the experts removed the blanket
"eager MoE implementation which is not torch exportable" skip that
`_skip_if_not_exportable` applied to Aria, so its export tests run for the first
time. The text model now exports cleanly, but two unrelated pre-existing
failures become visible:
- `AriaProjector` picks its query count by looking the patch dim up in a Python
dict, which raises `TypeError: unhashable type: non-nested SymInt` under
dynamic shapes.
- Exported decode returns only `logits` while eager also surfaces the populated
KV cache, the same shape as the existing Voxtral/Gemma3n entries.
Record both in `EXPORT_SKIPS` with a TODO rather than widening this PR.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Fix Aria's export failures instead of skipping them
Replaces the `EXPORT_SKIPS` entries from the previous commit with fixes for
the two problems they papered over. Both are pre-existing; they only became
visible because standardising the experts removed the blanket
"eager MoE implementation" skip that had been hiding every Aria export test.
`AriaProjector` looked its query count up in a Python dict keyed on the patch
dim, which `torch.export` cannot hash when that dim is symbolic. Comparing the
keys instead lets export specialise on the equality guard. The lookup is over
two entries and the `KeyError` path is unchanged.
`AriaModel.forward` returned `past_key_values` only when `use_cache` was
truthy, so exported decode surfaced just `logits` while eager also returned the
populated cache. Every other Llava-derived model returns it unconditionally,
and the language model already returns `None` when caching is off, so this
aligns Aria with `LlavaModel` rather than changing what callers see.
Aria's export tests now pass in full: 24 subtests, against 19 before this PR
(with the language-model exports failing) and 7 with the skips.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>