Port Llama 4's MoE to the shared experts interface
`Llama4TextExperts` ran every expert densely on every token and had no
`@use_experts_implementation`, so `grouped_mm`/`batched_mm` and expert
parallelism could not apply. It now takes `(hidden_states, top_k_index,
top_k_weights)` like every other experts class and each token runs only
its routed expert.
The router score scales the experts' input, not their output, so
`Llama4TextMoe` scales the input and passes each (token, slot) pair as a
top-1 route with unit weight. The router returns
`(logits, scores, indices)`; the logits recorder follows.
The EP plan is the standard experts plan (`grouped_gemm` +
`ep_dispatch_experts`): main's plan matched no EP mode and was never
applied. The shared-expert add is out of place, since the TP rowwise
output is a view autograd forbids modifying in place.
The fbgemm and sequential (compressed-tensors / bnb) Llama 4 experts take
the same routed signature.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
