Add support for stop words in TRTLLM #2678
Narsil
merged
18 commits into
main
from
trtllm-stop-words
feat(trtllm): rewrite health to not account for current state
7f383bf4
chore(looper): cleanup a bit more
3af45189
feat(post_processing): max_new_tokens is const evaluated now
17073267
chore(ffi):formatting
e711947e
feat(trtllm): add stop words handling
171a5638
chore(trtllm): create specific parallelconfig factory and logging ini…
16bb4b67
chore(trtllm): define a macro for SizeType cast
c90680ed
chore(trtllm): use GetParallelConfig
703c26ec
chore(trtllm): minor refactoring
c2bb199f
chore(trtllm): validate there are enough GPus on the system for the d…
d659cb01
chore(trtllm): ensure max throughput scheduling policy is selected
cdba16fd
chore(trtllm): minor fix
b939a0f7
chore(router): minor refactorings
50a19aa3
feat(docker): build with-slurm ompi
17573d42
feat(docker): add python3.10 dev to runtime deps
e4f67f70
chore(docker): add mpi to ld_library_path
38126233
chore(docker): install transformers
13a68e22
feat(trtllm): detect stop_words from generation_config.json
84f14a14
Login to write a write a comment.
Login via GitHub