transformers
a672eb5f - logits_to_keep bool mask for all ForXXX classes (#48825)

Commit
27 days ago
logits_to_keep bool mask for all ForXXX classes (#48825) * feat(gemma4): bool mask logits_to_keep, expose last_hidden_state logits_to_keep only supported an int (keep the last N) or a flat index tensor for a single packed sequence. SFT training with sequence packing needs to exclude non-contiguous spans per example in a batch, which neither form can express. Accept a 2D bool mask shaped like input_ids (batch_size, sequence_length); logits are computed only for the True positions, flattened in input_ids order. Existing int/tensor behavior is unchanged. Also add last_hidden_state to Gemma4CausalLMOutputWithPast, independent of output_hidden_states, since packed-sequence loss computation needs the full hidden states regardless of that flag. Closes #48784 * feat(gemma4): extend logits_to_keep mask support to all ForXXX classes, fix negative layer indices Extend the bool-mask logits_to_keep support added for Gemma4ForConditionalGeneration to the rest of the family (Gemma4ForCausalLM, Gemma4UnifiedForCausalLM, Gemma4UnifiedForConditionalGeneration), settling on a plain if/else with a single lm_head call per branch to match the convention already used elsewhere in the codebase (Gemma4ForCausalLM itself, biogpt, csm, llava_onevision, etc.). Also fix output_hidden_states selective layer capture (src/transformers/utils/output_capturing.py, #48081) to resolve negative indices (e.g. -1 for the last layer). The hook fires progressively during the forward pass and can't know the total layer count in advance, so count how many hidden_states hooks get installed per model during the one-time hook installation walk, store it on the model, and resolve negative indices against that count before the forward pass runs. No change to the per-layer hook logic itself. Refs #48784 * feat: extend logits_to_keep bool-mask support to all ForCausalLM/ForConditionalGeneration classes * remove last_hidden_state field, rely on output_hidden_states indices instead * revert CSM to original logits_to_keep handling, drop stray gemma4-only docstring * revert output_capturing.py * Add: slice_logits_to_keep to modeling_utils.py * apply to `bit`, `falcon`, `hyperclovax`, `hyperclovax_vision_v2`, `nemotron_h`, `opt` * Add: 2d idx process * Address review feedback: https://github.com/huggingface/transformers/pull/48825#pullrequestreview-5573728421 * Address feedback: https://github.com/huggingface/transformers/pull/48825#issuecomment-6272236980 * Address feedback: https://github.com/huggingface/transformers/pull/48825#issuecomment-6272236980 * Update src/transformers/modeling_utils.py Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Update src/transformers/modeling_utils.py Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> * Apply slice_logits_to_keep to the models that were missed * dont like the newline that was added * more newline * more * remove more newline and nonsense assignment * unrelated * more based on latest main --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com> Co-authored-by: Cyril Vallez <cyril.vallez@huggingface.co>
Author
Parents
Loading