DeepSpeed
539598ba - fix(rollout): narrow the padded-prompt fallback to configs that need it

Commit
2 days ago
fix(rollout): narrow the padded-prompt fallback to configs that need it The _ki_prompt_padded guard routed every padded batch to HF generate — a regression for standard-model graph users, whose prefill and decode static_attn carry the padding mask natively (master behavior). Fall back only when hybrid prefill would feed GDN a None mask or injected segKI decode kernels would attend pad slots. Verified on 2xA40 (Qwen2.5-0.5B): standard+padded+no-segki now takes the graph path with master-semantics first token (330 = pad-position argmax, matching the pre-guard behavior); segki and unpadded cases unchanged. Addresses phoeenniixx's review. Signed-off-by: Guokai Ma <guokai.ma@intel.com>
Author
Parents
Loading