transformers.js
11db949c - Add support for idefics3 (SmolVLM) (#1059)

Commit
1 year ago
Add support for idefics3 (SmolVLM) (#1059) * [WIP] Add support for idefics3 (SmolVLM) * Cleanup * Update `DataTypeMap` with 4-bit data types * Format the model inputs before logging to console * Use QUInt8 when quantizing models produced by onnxruntime-genai * `auto` dtype selection * Export `load_image` helper function * Add listed support for Idefics3 * Add support for batched 2d images in idefics3 processor * Update unit tests * Add another unit test to ensure correctness of pixel attention mask placement * Move image tokens out of call function * Formatting * Improve auto selection logic * Return correct pixel_attention_mask * Update pixel_attention_mask unit test * Formatting * Add idefics3 unit tests * Increase idefics processor unit test timeout
Author
Parents
Loading