transformers
Patch MistralCommonTokenizer
#41439
Merged

Commits
  • Fix token_to_id and add add_generation_prompt
    juliendenize committed 164 days ago
  • Fix spm download
    juliendenize committed 164 days ago
  • Refactor spm
    juliendenize committed 164 days ago
  • Try another possibly non-gated spm
    juliendenize committed 164 days ago
  • Improve get_vocab
    juliendenize committed 164 days ago
  • lint
    juliendenize committed 164 days ago
  • Improve get_vocab
    juliendenize committed 164 days ago
  • Add warn to piece_to_id
    juliendenize committed 164 days ago
  • Improve from_pretrained raise and revert model spm
    juliendenize committed 164 days ago
  • Revert fast
    juliendenize committed 164 days ago
Loading