Add qwen2 #28436
ArthurZucker
merged
76 commits into
huggingface:main
from
JustinLin610:add_qwen2
add config, modeling, and tokenization
f5bf0189
add auto and init
3226f3a9
update readme
52ab139a
update readme
0fe20cdb
update team name
82992662
fixup
b8a8e229
fixup
55802f3a
update config
6b587b86
update code style
dfe7d2e4
update for fixup
525de40d
update for fixup
7f6f5e28
update for fixup
36fcf653
update for testing
42f561a1
update for testing
105d98f5
fix bug for config and tokenization
47e6e9e2
fix bug for bos token
5061a26f
not doctest
0987dabb
debug tokenizer
bf4e928b
not doctest
2658bafe
debug tokenization
69b03e7d
debug init for tokenizer
725ab32f
fix style
57264f3d
update init
0489d0f3
delete if in token auto
e3b60a62
add tokenizer doc
e1b4a8c2
add tokenizer in init
22d34024
Update dummy_tokenizers_objects.py
c05a7be3
Merge pull request #1 from jklj077/patch-1
783eec87
update
c24ee57d
update
0f83bc1a
debug
e68a65ce
Merge branch 'huggingface:main' into add_qwen2
03ff2286
Update tokenization_qwen2.py
2e004319
Merge pull request #3 from jklj077/patch-1
73695ff3
debug
e041c791
Merge branch 'huggingface:main' into add_qwen2
e96c5c26
Update convert_slow_tokenizer.py
9cda260c
add copies
a81deed9
Merge pull request #4 from jklj077/patch-3
e652aa9b
add copied from and make style
985fe0f1
update files map
f7beb773
update test
35a7480f
Merge pull request #5 from jklj077/patch-4
26a24db2
fix style
f419098f
fix merge reading and update tests
6b3247b3
Merge pull request #6 from jklj077/patch-6
68968f5e
fix tests
9c45e774
fix tests
93bae64c
Merge pull request #7 from jklj077/patch-6
1345aae0
fix style
bd57c639
debug a variable in readme
25d43f04
ArthurZucker
commented
on 2024-01-15
Update src/transformers/models/qwen2/configuration_qwen2.py
e68a1b8e
update test and copied from
4135193b
fix style
8708e50f
update qwen2 tokenization and tests
db82439c
Update tokenization_qwen2.py
e0c7750a
Merge pull request #8 from jklj077/patch-7
7c13f741
delete the copied from after property
8ebacadc
fix style
7b670a87
update tests
304f0dcc
Merge branch 'patch-7' of https://github.com/jklj077/transformers int…
8b144e1e
Merge pull request #9 from jklj077/patch-7
ca4a8c5d
update tests
1ce4e4de
Merge pull request #10 from jklj077/patch-7
04c4a75c
Merge branch 'huggingface:main' into add_qwen2
93cbb5d5
add copied from
1e0b937f
fix bugs
22f7b1e4
update doc
69d3f89e
add warning for sliding window attention
2840f0a5
update qwen2 tokenization
87f6cf70
Merge pull request #11 from jklj077/patch-8
0a47113c
fix style
047e3722
Merge branch 'huggingface:main' into add_qwen2
6b6dae0c
Update src/transformers/models/qwen2/modeling_qwen2.py
a4e80eee
Merge branch 'huggingface:main' into add_qwen2
39cc60b9
fix tokenizer fast
a8b0c3c3
fxmarty
commented
on 2024-04-10
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub