huggingface/text-generation-inference

Pull Requests Commits

feat(trtllm): fix logits retrieval

mfuntowicz committed 1 year ago

b653605e

feat(trtllm): expose finish reason to Rust

mfuntowicz committed 1 year ago

60059b69

misc(backend): fix reborrowing Pin<&mut T> as described in the doc https://doc.rust-lang.org/stable/std/pin/struct.Pin.html#method.as_mut

mfuntowicz committed 1 year ago

f0cd4742

feat(backend): fix missing "0" field access

mfuntowicz committed 1 year ago

049f4acd

feat(backend): make sure we can easily cancel request on the executor

mfuntowicz committed 1 year ago

b3cd5ea0

feat(backend) fix moving backend when pulling

mfuntowicz committed 1 year ago

300f6c6f

effectively cancel the request on the executor

mfuntowicz committed 1 year ago

460f290d

chore(trtllm): update dependency towards 0.15.0

mfuntowicz committed 1 year ago

b6dbf605

test(backend): more test coverage

mfuntowicz committed 1 year ago

cc6bc339

feat(backend): remove constexpig

mfuntowicz committed 1 year ago

62530649

feat(backend): remove constexpr from par

mfuntowicz committed 1 year ago

881527a5

test(backend): add more unittest

mfuntowicz committed 1 year ago

ad3ed0d1

feat(backend): add guard to multiple header definitions

mfuntowicz committed 1 year ago

c94b9de4

feat(backend): fix main.rs retrieving the tokenizer

mfuntowicz committed 1 year ago

16ba2f5a

feat(backend): make backend_workspace_t::engines_folder constexpr

mfuntowicz committed 1 year ago

874bc28d

feat(backend): impl missing generation_step_t as return value of pull_tokens

mfuntowicz committed 1 year ago

2f8634ec

feat(backend): fix backend_exception_t -> backend_error_t naming

mfuntowicz committed 1 year ago

a7bad25c

feat(backend): allow overriding which Python to use

mfuntowicz committed 1 year ago

879e1a41

feat(backend): use latest trtllm main version to have g++ >= 13 compatibility

mfuntowicz committed 1 year ago

71e700a6

feat(backend): more impl

mfuntowicz committed 1 year ago

fd7e2b5b

feat(backend): delete previous backend impl

mfuntowicz committed 1 year ago

df99164d

feat(backend): introduce backend_workspace_t to store precomputed information from the engine folder

mfuntowicz committed 1 year ago

25c6bbe1

feat(backend): missing return statement

mfuntowicz committed 1 year ago

702dc9cd

feat(backend): enable compiler warning if support for RVO not applying

mfuntowicz committed 1 year ago

87272ffe

feat(backend): added some logging

mfuntowicz committed 1 year ago

9bb63097

feat(backend): remove all the logs from hardware.hpp

mfuntowicz committed 1 year ago

6d356575

feat(backend): initial rewrite of the backend for simplicity

mfuntowicz committed 1 year ago

3a2698fb

test(ctest) enable address sanitizer

mfuntowicz committed 1 year ago

1830fe88

feat(hardware) enable new hardware.hpp and unittests

mfuntowicz committed 1 year ago

7a81040d

misc(cmake) update dependencies

mfuntowicz committed 1 year ago

0f17415d

Older