huggingface/transformers

Pull Requests Commits

check if repo has tokenizer.json file, then load PreTrainedTokenizerFast

Ita Zaporozhets committed 1 year ago

b6ae24fc

check if can index

itazap committed 1 year ago

88b140e8

parse mapping instead

itazap committed 1 year ago

f4d7160d

rm siglip change

itazap committed 1 year ago

3db349e6

itazap committed 1 year ago

7f2e583f

add support for loading a pretrainedfast tokenizer if fast=true and tokenizer.json file exists

itazap committed 1 year ago

d9378690

Update Albumentations Versions (#33704)

vasqu committed 1 year ago

Verified 4f29a60b

fix trainer tr_loss add error (#33651)

Wang-Xiaodong1899 committed 1 year ago

Verified 1ec7a70f

Fix modular model converter unable to generate Processor classes (#33737)

tonywu71 committed 1 year ago

Verified e1b15086

fix: add docstring for `image_size` in Convnextv2 config (#33734)

lucianosrp committed 1 year ago

Verified e32521bf

clean_up_tokenization_spaces=False if unset (#31938)

itazap committed 1 year ago

Verified 6730485b

Generate: `can_generate()` recursive check (#33718)

gante committed 1 year ago

Verified 3557f9a1

Fix position embeddings singular/plural (#33678)

molbap committed 1 year ago

Verified 9f97c393

Fix docs and docstrings Omdet-Turbo (#33726)

yonigozlan committed 1 year ago

Verified 77b47e66

fix: use correct var names for check_tokenizers script (#33702)

niqodea committed 1 year ago

Verified c716fc0e

[`MllamaProcessor`] Update errors and API with multiple image (#33715)

ArthurZucker committed 1 year ago

Verified 46841d3e

Uniformize kwargs for chameleon processor (#32181)

leloykun committed 1 year ago

Verified 0a21381b

Add Idefics 3! (#32473)

andimarafioti committed 1 year ago

Verified f2c388e3

LysandreJik committed 1 year ago

f0eabf6c

adding positional encoder changes and tests (#32600)

manuelsh committed 1 year ago

Verified a55adee8

Add MLLama (#33703)

ArthurZucker committed 1 year ago

Verified 19d58d31

Add OmDet-Turbo (#31843)

yonigozlan committed 1 year ago

Verified 94f18cf2

Corrected max number for bf16 in transformer/docs (#33658)

alankashkash committed 1 year ago

Verified ade9e0fe

Add AdEMAMix optimizer (#33682)

matthewdouglas committed 1 year ago

Verified 196d35cc

Add SDPA support for M2M100 (#33309)

isaac091 committed 1 year ago

Verified 61e98cb9

Fix Megatron-LM tokenizer path (#33344)

hikari-kubota-fixstars committed 1 year ago

Verified 68049b17

HFQuantizer implementation for compressed-tensors library (#31704)

bfineran committed 1 year ago

Verified 574a9e12

fix code quality after merge

ArthurZucker committed 1 year ago

7e638ef2

[Pixtral] Improve docs, rename model (#33491)

NielsRogge committed 1 year ago

Verified 06e27e3d

bump tokenizers, fix added tokens fast (#32535)

ArthurZucker committed 1 year ago

Verified c6379858

Older