huggingface/text-generation-inference

Pull Requests Commits

Fix local load for peft (#1373)

Narsil committed 2 years ago

Verified 529d7c25

feat: update exllamav2 kernels (#1370)

OlivierDehaene committed 2 years ago

Verified 564199ba

docs: Change URL for Habana Gaudi support in doc (#1343)

regisss committed 2 years ago

Verified 987c959f

Peft safetensors. (#1364)

Narsil committed 2 years ago

Verified eb8923a9

fix: fix gpt-q with groupsize = -1 (#1358)

OlivierDehaene committed 2 years ago

Verified d077150e

fix: fix offline (#1341) (#1347)

OlivierDehaene committed 2 years ago

Verified 8428ed10

fix: fix logic if sliding window key is not present in config (#1352)

OlivierDehaene committed 2 years ago

Verified 1b1bfa49

feat: relax mistral requirements (#1351)

OlivierDehaene committed 2 years ago

Verified 9b56d3fb

OlivierDehaene committed 2 years ago

f3aea78f

fix: max_past default value must be -1, not 0 (#1348)

OlivierDehaene committed 2 years ago

Verified 37555cf4

fix: only keep stop sequence buffer if we have some

OlivierDehaene committed 2 years ago

9b78a6ee

fix: slice stopping criteria buffer

OlivierDehaene committed 2 years ago

80a69204

fix: fix quant linear autotune

OlivierDehaene committed 2 years ago

083c2de9

fix: fix triton OutOfResources import

OlivierDehaene committed 2 years ago

773aabdd

feat: add more latency metrics in forward (#1346)

OlivierDehaene committed 2 years ago

Verified 50b495f3

fix: fix gpt-q params loading

OlivierDehaene committed 2 years ago

44b267ab

fix: default max_new_tokens to 100

OlivierDehaene committed 2 years ago

28821bfd

OlivierDehaene committed 2 years ago

88aae259

feat: add quant to mixtral (#1337)

OlivierDehaene committed 2 years ago

Verified 82670d97

OlivierDehaene committed 2 years ago

ec6d4592

OlivierDehaene committed 2 years ago

d0841cc8

chore: formatting

OlivierDehaene committed 2 years ago

72ee382d

feat: mixtral (#1328)

OlivierDehaene committed 2 years ago

Verified 3a521c92

Speculative (#1308)

Narsil committed 2 years ago

Verified 9ecfa16b

Add a stale bot. (#1313)

Narsil committed 2 years ago

Verified 3238c491

Fix AMD documentation (#1307)

fxmarty committed 2 years ago

Verified 25b5f819

OlivierDehaene committed 2 years ago

ccd5725a

Make GPTQ test less flaky (#1295)

Narsil committed 2 years ago

Verified 624800c4

Let each model resolve their own default dtype. (#1287)

Narsil committed 2 years ago

Verified ba552e1a

`make install-flash-attn-v2-cuda` should work like `make install-flash-attn-v2` used to work. (#1294)

Narsil committed 2 years ago

Verified 3c71c656

Newer Older