optimum
5dfb4981 - Add IOBinding support to ONNX Runtime module (#421)

Commit
3 years ago
Add IOBinding support to ONNX Runtime module (#421) * Create IOBinding helper and Type helper * Apply IOBinding helper to causallm model * Complete comment * Fix use_io_binding value define bug * np to numpy * Replace asserts with error raise * Add io binding return type * Add is_onnxruntime_training_available in utils * Apply io binding to encoders * Also for custom tasks * Improve use_io_binding args * Apply io binding for seq2seq * seq2seq io binding bug fix * remove zip when outputs are fixed * Check cupy availability * Move io binding preparation to causal model * move io binding preparation for other ort models * Move io binding to ort seq2seq model * Bind output tensors directly for causal lm * Remove useless in IO helper * Direct binding for multi-choice/im-cls/custom * Direct binding for all other ort models * Remove IO binding for ORTModelForCustomModels * IO binding added for seq2seq model * Revert dependencies for anothet PR * Add IO binding tests for Encoder/Decoder * Add config for passing seq2seq modeling tests * Improve seq2seq buffer size * Add test for seq2seq * Add tests for generation * Add warning for TensorRT * Test: allclose -> equal
Author
Parents
Loading