transformers
78d52c58 - Exporters: decode a token at a time for linear attention that switches kernels on the query length

Commit
3 days ago
Exporters: decode a token at a time for linear attention that switches kernels on the query length Qwen3.5's (and Qwen3-Next's) gated delta rule runs a chunked kernel beyond one query token and a recurrent one for a single token: a decode graph traced on several tokens keeps the chunked kernel, which a one-token step then runs wrong. Such a model's dynamic export now defaults to a one-token decode graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parents
Loading