transformers.js

262a1311 - Voxtral Realtime, LFM2-VL, Granite Speech, and modeling type refactoring (#1569)

Voxtral Realtime, LFM2-VL, Granite Speech, and modeling type refactoring (#1569) * [wip] Refactor modeling types * Add more causal qwen tests * Add qwen vl text-only unit tests * Add support for Gemma3nForCausalLM * Add Gemma3n causal test case via pipeline * PaliGemmaForConditionalGeneration can inherit from LlavaForConditionalGeneration * Add initial lfm2_vl support * lfm2_vl support continued * cleanup imports * support sum() on bigint * Create test_image_processing_lfm2_vl.js * Mistral3ForConditionalGeneration should inherit from LlavaForConditionalGeneration * Fix type errors * cleanup + optimize * Formatting * cleanup * formatting * Optimize * Add support for LFM2-VL * Move voxtral to separate folder * Formatting * Use cached audio for test * Add unit tests for VoxtralRealtimeFeatureExtractor * Add voxtral realtime tests * Add support for voxtral realtime * cleanup * Formatting * Formatting * Use new getPastLength method for chatterbox * Create DynamicCache class and refactor internals * Formatting * Fix .gitignore * JSDoc magic * Formatting * Fix dispose type * Add helper types to type tests * Write cache type tests * Move typescript compilation tests * Fix types of Dynamic Cache * Strengthen pipeline type tests * Formatting * Update JSDoc * cleanup + optimize * Refactor to use parent generation logic * Use WeakMap for generation state * Update test_modeling_voxtral_realtime.js * cleanup * correctly dispose old encoder kv cache * Cleanup * cleanup * more cleaning * Add voxtral realtime to docs * Deduplicate smart_resize implementations * cleanup * Add granite speech unit tests * Add helper function for input name selection and session validation * Refactor encode functions (use base class) * Move SmolVLMForConditionalGeneration to its own file * Add support for Granite Speech * Move shared normalization logic to spectrogram function * Use spectrogram parameters to handle custom granite_speech normalization * Support multiple possible modality_input_names * Only pick multimodal inputs if no inputs_embeds are specified * Add text-only granite speech unit test * Add granite speech modeling unit test that uses processor * Add default num_logits_to_keep, if model supports it * Add num_logits_to_keep llama model tests * cleanup * Use num_logits_to_keep=1 during generation * DynamicCache input validation