huggingface/text-generation-inference

Pull Requests Commits

misc(deps): update ompi from 4.1.6 to 4.1.7rc1 to avoid strange deadlock

mfuntowicz committed 1 year ago

3bb78a82

misc(deps): add pyo3 to dependencies

mfuntowicz committed 1 year ago

51222547

feat(trtllm): detect stop_words from generation_config.json

mfuntowicz committed 1 year ago

9cee00ee

chore(docker): install transformers

mfuntowicz committed 1 year ago

6376fecc

chore(docker): add mpi to ld_library_path

mfuntowicz committed 1 year ago

ef003118

feat(docker): add python3.10 dev to runtime deps

mfuntowicz committed 1 year ago

5f81550a

feat(docker): build with-slurm ompi

mfuntowicz committed 1 year ago

ba2618eb

chore(router): minor refactorings

mfuntowicz committed 1 year ago

56106b4c

chore(trtllm): minor fix

mfuntowicz committed 1 year ago

84f3bf90

chore(trtllm): ensure max throughput scheduling policy is selected

mfuntowicz committed 1 year ago

47d8c53d

chore(trtllm): validate there are enough GPus on the system for the desired model

mfuntowicz committed 1 year ago

a6ac2741

chore(trtllm): minor refactoring

mfuntowicz committed 1 year ago

848b8ad5

chore(trtllm): use GetParallelConfig

mfuntowicz committed 1 year ago

60a08a28

chore(trtllm): define a macro for SizeType cast

mfuntowicz committed 1 year ago

d5c8bdc5

chore(trtllm): create specific parallelconfig factory and logging init methods

mfuntowicz committed 1 year ago

7217cafa

feat(trtllm): add stop words handling

mfuntowicz committed 1 year ago

421a1754

chore(ffi):formatting

mfuntowicz committed 1 year ago

c1a43a6c

feat(post_processing): max_new_tokens is const evaluated now

mfuntowicz committed 1 year ago

9ac26ed7

chore(looper): cleanup a bit more

mfuntowicz committed 1 year ago

cdac4b00

feat(trtllm): rewrite health to not account for current state

mfuntowicz committed 1 year ago

04c6f512

chore(router): add python dependency

mfuntowicz committed 1 year ago

18b473b0

chore(rebase): fix invalid references

mfuntowicz committed 1 year ago

d73401ac

Revert "chore(trtllm): remove unused method"

mfuntowicz committed 1 year ago

f5b9ee36

feat(trtllm): do not tokenize twice

mfuntowicz committed 1 year ago

8d1c3c8a

misc(router): remove SchedulingError

mfuntowicz committed 1 year ago

1a3da05f

feat(trtllm): cache maxNumTokens to avoid calling JSON everytime

mfuntowicz committed 1 year ago

e6da2124

chore(trtllm): remove unused method

mfuntowicz committed 1 year ago

31747163

chore(trtllm): post-rebase commit

mfuntowicz committed 1 year ago

fb00f985

chore(trtllm): fmt

mfuntowicz committed 1 year ago

85c03e33

chore(trtllm): disable tokenizer parallelism by default

mfuntowicz committed 1 year ago

e3bce407

Older