huggingface/text-generation-inference

Pull Requests Commits

fix typo for dynamic rotary (#745)

flozi00 committed 2 years ago

Verified 7766fee9

Typo fix. (#746)

Narsil committed 2 years ago

Verified d3d8f1bd

fix(server): Failing quantize config after local read. (#743)

Narsil committed 2 years ago

Verified 15fc6466

chore: fix typo in mpt_modeling.py (#737)

eltociear committed 2 years ago

Verified 2a13f1a0

Adding Rope scaling. (#741)

Narsil committed 2 years ago

Verified 932bdd93

Fix typing in `Model.generate_token` (#733)

jaywonchung committed 2 years ago

Verified b9633c46

Local gptq support. (#738)

Narsil committed 2 years ago

Verified 92bb56b0

OlivierDehaene committed 2 years ago

Verified 3ef5ffbc

chore: update license to HFOIL (#725)

OlivierDehaene committed 2 years ago

Verified bde25e62

feat(server): update vllm version (#723)

OlivierDehaene committed 2 years ago

Verified afd04dc7

docs: Add hardware section to TOC in README (#721)

regisss committed 2 years ago

Verified f848dece

Add section about TGI on other AI hardware accelerators in README (#715)

regisss committed 2 years ago

Verified 5a1cccbb

OlivierDehaene committed 2 years ago

Verified 9f18f4c0

feat(server): support new falcon config (#712)

OlivierDehaene committed 2 years ago

Verified ab96b9ae

fix(server): fix missing datasets in quantize

OlivierDehaene committed 2 years ago

2efd46ef

fix(server): fix quantization python requirements (#708)

OlivierDehaene committed 2 years ago

Verified 8bd0adb1

docs(README): update readme

OlivierDehaene committed 2 years ago

e64a6589

feat(server): Using `quantize_config.json` instead of GPTQ_BITS env variables. (#671)

Narsil committed 2 years ago

Verified a0d55358

fix(server): fix exllama buffers (#689)

OlivierDehaene committed 2 years ago

Verified 37df6df3

feat: add cuda memory fraction (#659)

OlivierDehaene committed 2 years ago

Verified 73a4d65d

feat(server): add local prom and health routes if running w/ ngrok

OlivierDehaene committed 2 years ago

1da642bd

Directly load GPTBigCode to specified device (#618)

Atry committed 2 years ago

Verified 15b3e9ff

feat(server): Add exllama GPTQ CUDA kernel support #553 (#666)

Narsil committed 2 years ago

Verified d5b5bc75

fix(server): use mem_get_info to get kv cache size (#664)

OlivierDehaene committed 2 years ago

Verified bf94df3c

fix(server): Fixing non parameters in quantize script `bigcode/starcoder` was an example. (#661)

Narsil committed 2 years ago

Verified 08b8eec1

fix(server): llama v2 GPTQ (#648)

fxmarty committed 2 years ago

Verified 362883f2

Add trust_remote_code to quantize script (#647)

ChristophRaab committed 2 years ago

Verified 214c06f5

docs: Update README.md (#643)

Narsil committed 2 years ago

Verified 5a1512c0

docs: Update README.md (#639)

Narsil committed 2 years ago

Verified 1c81df15

feat(router): ngrok edge (#642)

OlivierDehaene committed 2 years ago

Verified b66b1904

Older