transformers
2934a0fe - Reject assisted generation for LFM2 and LFM2-MoE (set _is_stateful) (#46937)

Commit
68 days ago
Reject assisted generation for LFM2 and LFM2-MoE (set _is_stateful) (#46937) LFM2 and LFM2-MoE are conv/attention hybrids that keep recurrent conv state, but inherited the default _is_stateful = False from Llama. Assisted and prompt-lookup decoding were therefore not rejected, and since their conv state cannot be rolled back during speculative verification they silently produced wrong tokens (divergent from greedy) instead of raising the clear "assisted generation is not supported with stateful models" error the other stateful models raise. Set _is_stateful = True on both.
Author
Parents
Loading