huggingface/text-generation-inference

Pull Requests Commits
add some more basic info in README.md
mfuntowicz committed 1 year ago
8fad7ae5
look for cuda 12.5
mfuntowicz committed 1 year ago
b665e2fa
make sure variable live long enough...
mfuntowicz committed 1 year ago
6b74f5b4
use std::env::const::ARCH
mfuntowicz committed 1 year ago
69a5804e
fix envvar CARGO_CFG_TARGET_ARCH set at runtime vs compile time
mfuntowicz committed 1 year ago
fcbf2fc1
add some custom stuff for nccl linkage
mfuntowicz committed 1 year ago
dda015f2
install to decoder_attention target
mfuntowicz committed 1 year ago
0a8c9d3d
clean up a bit
mfuntowicz committed 1 year ago
48315e26
add missing dependant libraries for linking
mfuntowicz committed 1 year ago
9c60c9ca
update build.rs to link to cuda 12.5
mfuntowicz committed 1 year ago
09bcca6a
update TensorRT install script to latest
mfuntowicz committed 1 year ago
e4fc0ebc
update TensorRT-LLM to latest version
mfuntowicz committed 1 year ago
03935f67
refactor the compute capabilities detection along with num gpus
mfuntowicz committed 1 year ago
ef187634
fix typo
mfuntowicz committed 1 year ago
3c39ab5a
make docker linter happy with same capitalization rule
mfuntowicz committed 1 year ago
4c657ca1
move to TensorRT-LLM v0.11.0
mfuntowicz committed 1 year ago
d9decb4c
refactored docker image
mfuntowicz committed 1 year ago
ff151b73
commenting out Python part for TensorRT installation
mfuntowicz committed 1 year ago
3db1be41
update tgi entrypoint
mfuntowicz committed 1 year ago
805e584b
adding missing ld_library_path for cuda stubs in Dockerfile
mfuntowicz committed 1 year ago
d0a34a95
fix missing / before tgi lib path
mfuntowicz committed 1 year ago
3fd2bb70
correctly setup linking search path for runtime layer
mfuntowicz committed 1 year ago
a32ef3b8
add missing pkgconfig folder for MPI in Dockerfile
mfuntowicz committed 1 year ago
fd06ca6e
align all the linker search dependency
mfuntowicz committed 1 year ago
40330c73
fix bad copy/past missing nvinfer linkage direction
mfuntowicz committed 1 year ago
6a9e925e
leverage pkg-config to probe libraries paths and reuse new install structure from cmake
mfuntowicz committed 1 year ago
3597beef
do the same name definition stuff for tensorrt_llm_executor_static
mfuntowicz committed 1 year ago
2aac2ff2
simplify prebuilt trtllm libraries name definition
mfuntowicz committed 1 year ago
da079df4
add some more information in CMakeLists.txt to correctly find and install nvrtc wrapper
mfuntowicz committed 1 year ago
20bcaea5
add some more information in CMakeLists.txt to correctly install executorWorker
mfuntowicz committed 1 year ago
84153702
Terms Of Use Privacy Policy Refund Policy Impressum