huggingface/text-generation-inference

Pull Requests Commits

working tunable

fxmarty committed 1 year ago

ff5e16b0

wip fix tunableop

fxmarty committed 1 year ago

2677bf85

trying to update to ROCm 6.1

fxmarty committed 1 year ago

a5093606

working & cached tunableop

fxmarty committed 1 year ago

17f5c307

use released torch 2.3

fxmarty committed 1 year ago

193dbb68

run integration tests on rocm

fxmarty committed 1 year ago

b8da9024

Merge branch 'main' into mi300-compat

fxmarty committed 1 year ago

75023670

Use the generation config. (#1808)

Narsil committed 1 year ago

Verified ee47973a

Update guidance docs to reflect grammar support in API (#1775)

dr3s committed 1 year ago

Verified eb08b9fa

Updating the benchmarks so everyone uses openai compat layer. (#1800)

Narsil committed 1 year ago

Verified fccf5edf

feat: improve temperature logic in chat (#1749)

drbh committed 1 year ago

Verified 0acac5cb

Adding support for `HF_HUB_OFFLINE` support in the router. (#1789)

Narsil committed 1 year ago

Verified 4c698fa6

fix: avoid frequency and repetition penalty on padding tokens (#1765)

drbh committed 1 year ago

Verified 23d82b8f

Idefics2. (#1756)

Narsil committed 1 year ago

Verified bfddfa59

Phi3 support (#1797)

Narsil committed 1 year ago

Verified 986b4044

mht-sharma committed 1 year ago

e7289700

mht-sharma committed 1 year ago

fbc5a6a1

feat: allow null eos and bos tokens in config (#1791)

drbh committed 1 year ago

Verified 9be1db31

Add attribute descriptions for `GenerateParameters` (#1798)

Wauplin committed 1 year ago

Verified 455cada5

fix typos in docs and add small clarifications (#1790)

MoritzLaurer committed 1 year ago

Verified ed72e921

Make `--cuda-graphs` work as expected (bis) (#1768)

fxmarty committed 1 year ago

Verified 26b39166

fix fa2 triton kernel not working with MQA/GQA

fxmarty committed 1 year ago

aef931ea

reenable _custom_C.LLMM1 as the culprit was FA2 triton

fxmarty committed 1 year ago

325f9774

disable _custom_C.LLMM1 as it is broken for TP>=2

fxmarty committed 1 year ago

81c27ba9

fxmarty committed 1 year ago

562cd4b0

revert dev only changes

fxmarty committed 1 year ago

6d59eb2e

add missing files

fxmarty committed 1 year ago

8eacae01

fxmarty committed 1 year ago

ec5343ec

_custom_C.LLMM1 and HIP_FORCE_DEV_KERNARG=1

fxmarty committed 1 year ago

1b4c8b4b

fxmarty committed 1 year ago

f723e5cc

Older