transformers
c82e017a - Misc. fixes for Pytorch QA examples: (#16958)

Comment changes are shownComment changes are hidden
Commit
3 years ago
Misc. fixes for Pytorch QA examples: (#16958) 1. Fixes evaluation errors popping up when you train/eval on squad v2 (one was newly encountered and one that was previously reported Running SQuAD 1.0 sample command raises IndexError #15401 but not completely fixed). 2. Removes boolean arguments that don't use store_true. Please, don't use these: *ANY non-empty string is being converted to True in this case and this clearly is not the desired behavior (and it creates a LOT of confusion). 3. All no-trainer test scripts are now saving metric values in the same way (with the right prefix eval_), which is consistent with the trainer-based versions. 4. Adds forgotten model.eval() in the no-trainer versions. This improved some results, but not everything (see the discussion in the end). Please, see the F1 scores and the discussion below.
Author
Parents
  • examples
    • flax/question-answering
      • File
        utils_qa.py
    • pytorch
      • question-answering
        • File
          run_qa_beam_search_no_trainer.py
        • File
          run_qa_no_trainer.py
        • File
          utils_qa.py
      • File
        test_accelerate_examples.py
    • tensorflow/question-answering
      • File
        utils_qa.py
Loading