transformers
86119c11 - add VisionTextDualEncoder and CLIP fine-tuning script (#15701)

Commit
3 years ago
add VisionTextDualEncoder and CLIP fine-tuning script (#15701) * begin script * update script * fix features and data args * main * add requirements * add column name args * fix captions * don't jit transforms * fix caption * fix labels, handle attention mask * convert pixel values to numpy * labels => input_ids * transform images on the fly * use AutoModel class, create the hybird model outside of the script * fix version message * add readme * Apply suggestions from code review Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * adderss review comments * add more comments * allow freezing vision and text models Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Author
Parents
Loading