Universal Speculative Decoding `CandidateGenerator` #35029
gante
merged
112 commits into
huggingface:main
from
keyboardAnt:usd
move `TestAssistedCandidateGeneratorDifferentTokenizers` into a new t…
aa7e01a6
refactor
f6b7f204
NOTHING. add space to rerun github actions tests
0ded37c5
remove it...
d48b69b0
`UniversalSpeculativeDecodingGenerator`
b47e33a1
Use `UniversalSpeculativeDecodingGenerator` when `generation_config.d…
8a991299
assistant tokenizes only the target's new suffix
4649bd2f
formatting
f199c94a
fix code
19c0057d
fix code
acf5a4b3
formatting
37121172
add `TestGenerateWithDifferentModels`
63f2f462
`TestGenerateWithDifferentModels` parameterize on `do_sample`
6ac33f14
`AssistantVocabMapping` & `AssistantVocabMappingCache`
69383117
formatting
5a0db3bb
`AssistantToTargetTranslator`: `get_target_input_ids` & `get_target_l…
92f8ad3b
improve `_get_assistant_to_target_input_ids` & formatting
7c8708ed
renaming
880d0aea
WIP: debugging `min_new_tokens`
d9b5e748
fix get_target_ids
25974d5e
`UniversalSpeculativeDecodingGenerator`
b8636ab1
assistant tokenizes only the target's new suffix
1ef46b78
formatting
f8e94eb1
fix code
439db843
fix code
643901de
formatting
77097ffb
`TestGenerateWithDifferentModels` parameterize on `do_sample`
d08b4f08
`AssistantVocabMapping` & `AssistantVocabMappingCache`
f242dc11
formatting
ede11763
`AssistantToTargetTranslator`: `get_target_input_ids` & `get_target_l…
511ee964
improve `_get_assistant_to_target_input_ids` & formatting
5e479456
renaming
25a43497
WIP: debugging `min_new_tokens`
95fe744a
fix get_target_ids
0ad88b27
fix device issue
bc5fa617
fix get_assistant_input_ids
41a5670e
add `TestAssistedCandidateGeneratorDifferentTokenizers`
44f7ba70
formatting
57aafcca
`AssistantVocabTranslatorCache` refactor & tests
6f95c33d
revert changes in `src/transformers/generation/logits_process.py`
078f763e
refactor `AssistedCandidateGenerator`
faac2fcd
refactor `AssistedCandidateGeneratorDifferentTokenizers`
76a2dd39
formatting
43e96e79
refactor `UniversalSpeculativeDecodingGenerator`
e63cb9df
fix negative value for max_new_tokens
8aa6020a
fix generation length target + attention_mask vs. assistant + attent
2169973d
fix device
c6da8273
fix negative max_new_tokens bug
2cf9e8e5
fix UAG
a1c0d051
minor
d8300913
formatting
19d0cced
`AssistedCandidateGeneratorDifferentTokenizers` `lookbehind`s init
5b8217d1
resolve conflict & formatting
9b0126a8
rerun CI tests
578d0b34
remove space...
7db26954
remove old code
fb699001
fix candidate_input_ids device
e40c775c
minor
b5ce873f
Merge pull request #4 from keyboardAnt/fix_device
bfccdeaa
formatting
d34d7eaa
Fix prepare + apply (#7)
9d4d9f9a
Add unittests for Universal Assisted generation
4e92e9ce
Merge branch 'main' into usd
3fe2d31e
fix style
a350b1cb
update tests
e047adf6
Remove unused import and fix `test_speculation_depth` test
011f5956
exclude special and reserved tokens from tokenizer for UAG
26524900
mv `test_universal_assisted_generation.py` to `generation/test_candid…
701edbb5
Merge pull request #8 from keyboardAnt/unit_tests_usd
7088978f
Remove unused imports and fix style using `make style` (#9)
3b89341d
formatting
e43dba87
Swap gated `meta-llama/llama-3.2` with `allenai/llama` (#10)
a5297951
Merge branch 'main' into usd
9025751b
Fix space sign disagreement (#12)
25cd5da8
Default values for some fields of assistant to target translator (#11)
77edae26
Update candidate_generator.py (#15)
a2a2882b
BUG fix in _prepare_assistant_input_ids (#14)
a5569471
jmamou
requested changes
on 2025-01-12
typo (`target_to_assistant_input_ids`)
407d8984
formatting
a24b1934
merge upstream/main
1afdaa3a
Merge branch 'main' into usd
88f6877a
Fix minor review comments (#16)
4e3660a1
Fix: `token_ids.to(torch.int64)` (#18)
c162c885
fix dtype
d0798a08
`assistant_input_ids.to(dtype=torch.long)`
d18d090b
Remove unused import from test_candidate_generator.py
ae2f16ff
Merge branch 'main' of https://github.com/keyboardAnt/transformers in…
02dba319
resolve pr comments (#19)
49a228f4
formatting
7f76fec3
Merge branch 'main' into usd
32335a5c
Merge branch 'main' into usd
1a796471
Merge branch 'main' into usd
78a2a2cf
Fix Joao's comments (#21)
751a0994
Merge branch 'main' into usd
bfb636d3
Merge branch 'main' into usd
00e325d7
fix style (#23)
8a39f5bc
Merge branch 'main' into usd
64c95fe4
Move atm (#24)
7661fc92
fix logit_processor
503ece9c
add atm_translator test
fb7187db
refactor test
dedcf982
Merge branch 'main' into usd
7e3f3dcd
Merge branch 'main' into usd
c9fc5a61
Merge branch 'main' into usd
94e8a317
remove threading from test
4e234706
Merge branch 'main' into usd
eae175c8
add require_torch in tests
67849311
Merge branch 'main' into usd
d20f07b7
gante
commented
on 2025-02-26
move AssistantVocabTranslatorCache + add tests
be79a153
Merge branch 'main' into usd
9cb0a3aa
gante
approved these changes
on 2025-02-26
ruff fix
b0e7a16b
Merge branch 'main' into usd
683bbee8
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub