text-generation-inference
TensorRT-LLM backend bump to latest version + misc fixes
#2791
Merged
Merged
TensorRT-LLM backend bump to latest version + misc fixes #2791
mfuntowicz
merged
30 commits into
main
from
trtllm/cancellation
misc(cmake) update dependencies
0f17415d
feat(hardware) enable new hardware.hpp and unittests
7a81040d
test(ctest) enable address sanitizer
1830fe88
feat(backend): initial rewrite of the backend for simplicity
3a2698fb
feat(backend): remove all the logs from hardware.hpp
6d356575
feat(backend): added some logging
9bb63097
feat(backend): enable compiler warning if support for RVO not applying
87272ffe
feat(backend): missing return statement
702dc9cd
feat(backend): introduce backend_workspace_t to store precomputed inf…
25c6bbe1
feat(backend): delete previous backend impl
df99164d
feat(backend): more impl
fd7e2b5b
feat(backend): use latest trtllm main version to have g++ >= 13 compa…
71e700a6
feat(backend): allow overriding which Python to use
879e1a41
feat(backend): fix backend_exception_t -> backend_error_t naming
a7bad25c
feat(backend): impl missing generation_step_t as return value of pull…
2f8634ec
feat(backend): make backend_workspace_t::engines_folder constexpr
874bc28d
feat(backend): fix main.rs retrieving the tokenizer
16ba2f5a
feat(backend): add guard to multiple header definitions
c94b9de4
test(backend): add more unittest
ad3ed0d1
feat(backend): remove constexpr from par
881527a5
feat(backend): remove constexpig
62530649
test(backend): more test coverage
cc6bc339
chore(trtllm): update dependency towards 0.15.0
b6dbf605
effectively cancel the request on the executor
460f290d
feat(backend) fix moving backend when pulling
300f6c6f
feat(backend): make sure we can easily cancel request on the executor
b3cd5ea0
feat(backend): fix missing "0" field access
049f4acd
misc(backend): fix reborrowing Pin<&mut T> as described in the doc ht…
f0cd4742
chore: Add doc and CI for TRTLLM (#2799)
ab6591e7
misc(backend): indent
1640da7c
Login to write a write a comment.
Login via GitHub