huggingface/text-generation-inference

Pull Requests Commits
working tunable
fxmarty committed 2 years ago
ff5e16b0
wip fix tunableop
fxmarty committed 2 years ago
2677bf85
trying to update to ROCm 6.1
fxmarty committed 2 years ago
a5093606
working & cached tunableop
fxmarty committed 2 years ago
17f5c307
use released torch 2.3
fxmarty committed 2 years ago
193dbb68
run integration tests on rocm
fxmarty committed 2 years ago
b8da9024
Merge branch 'main' into mi300-compat
fxmarty committed 2 years ago
75023670
Use the generation config. (#1808)
Narsil committed 2 years ago
Verified ee47973a
Update guidance docs to reflect grammar support in API (#1775)
dr3s committed 2 years ago
Verified eb08b9fa
Updating the benchmarks so everyone uses openai compat layer. (#1800)
Narsil committed 2 years ago
Verified fccf5edf
feat: improve temperature logic in chat (#1749)
drbh committed 2 years ago
Verified 0acac5cb
Adding support for `HF_HUB_OFFLINE` support in the router. (#1789)
Narsil committed 2 years ago
Verified 4c698fa6
fix: avoid frequency and repetition penalty on padding tokens (#1765)
drbh committed 2 years ago
Verified 23d82b8f
Idefics2. (#1756)
Narsil committed 2 years ago
Verified bfddfa59
Phi3 support (#1797)
Narsil committed 2 years ago
Verified 986b4044
black
mht-sharma committed 2 years ago
e7289700
add LLMM_Silu
mht-sharma committed 2 years ago
fbc5a6a1
feat: allow null eos and bos tokens in config (#1791)
drbh committed 2 years ago
Verified 9be1db31
Add attribute descriptions for `GenerateParameters` (#1798)
Wauplin committed 2 years ago
Verified 455cada5
fix typos in docs and add small clarifications (#1790)
MoritzLaurer committed 2 years ago
Verified ed72e921
Make `--cuda-graphs` work as expected (bis) (#1768)
fxmarty committed 2 years ago
Verified 26b39166
fix fa2 triton kernel not working with MQA/GQA
fxmarty committed 2 years ago
aef931ea
reenable _custom_C.LLMM1 as the culprit was FA2 triton
fxmarty committed 2 years ago
325f9774
disable _custom_C.LLMM1 as it is broken for TP>=2
fxmarty committed 2 years ago
81c27ba9
fix
fxmarty committed 2 years ago
562cd4b0
revert dev only changes
fxmarty committed 2 years ago
6d59eb2e
add missing files
fxmarty committed 2 years ago
8eacae01
cleaning
fxmarty committed 2 years ago
ec5343ec
_custom_C.LLMM1 and HIP_FORCE_DEV_KERNARG=1
fxmarty committed 2 years ago
1b4c8b4b
working
fxmarty committed 2 years ago
f723e5cc
Terms Of Use Privacy Policy Refund Policy Impressum