huggingface/text-generation-inference

Pull Requests Commits

docs: Add hardware section to TOC in README (#721)

regisss committed 2 years ago

Verified f848dece

Add section about TGI on other AI hardware accelerators in README (#715)

regisss committed 2 years ago

Verified 5a1cccbb

OlivierDehaene committed 2 years ago

Verified 9f18f4c0

feat(server): support new falcon config (#712)

OlivierDehaene committed 2 years ago

Verified ab96b9ae

fix(server): fix missing datasets in quantize

OlivierDehaene committed 2 years ago

2efd46ef

fix(server): fix quantization python requirements (#708)

OlivierDehaene committed 2 years ago

Verified 8bd0adb1

docs(README): update readme

OlivierDehaene committed 2 years ago

e64a6589

feat(server): Using `quantize_config.json` instead of GPTQ_BITS env variables. (#671)

Narsil committed 2 years ago

Verified a0d55358

fix(server): fix exllama buffers (#689)

OlivierDehaene committed 2 years ago

Verified 37df6df3

feat: add cuda memory fraction (#659)

OlivierDehaene committed 2 years ago

Verified 73a4d65d

feat(server): add local prom and health routes if running w/ ngrok

OlivierDehaene committed 2 years ago

1da642bd

Directly load GPTBigCode to specified device (#618)

Atry committed 2 years ago

Verified 15b3e9ff

feat(server): Add exllama GPTQ CUDA kernel support #553 (#666)

Narsil committed 2 years ago

Verified d5b5bc75

fix(server): use mem_get_info to get kv cache size (#664)

OlivierDehaene committed 2 years ago

Verified bf94df3c

fix(server): Fixing non parameters in quantize script `bigcode/starcoder` was an example. (#661)

Narsil committed 2 years ago

Verified 08b8eec1

fix(server): llama v2 GPTQ (#648)

fxmarty committed 2 years ago

Verified 362883f2

Add trust_remote_code to quantize script (#647)

ChristophRaab committed 2 years ago

Verified 214c06f5

docs: Update README.md (#643)

Narsil committed 2 years ago

Verified 5a1512c0

docs: Update README.md (#639)

Narsil committed 2 years ago

Verified 1c81df15

feat(router): ngrok edge (#642)

OlivierDehaene committed 2 years ago

Verified b66b1904

feat(server): auto max_batch_total_tokens for flash att models (#630)

OlivierDehaene committed 2 years ago

Verified fe80f536

fix(server): fix llamav2 config (#635)

OlivierDehaene committed 2 years ago

Verified 5e6ddfd6

OlivierDehaene committed 2 years ago

Verified cf83f9b6

feat(server): add support for llamav2 (#633)

Narsil committed 2 years ago

Verified 211b211e

feat(server): flash attention v2 (#624)

OlivierDehaene committed 2 years ago

Verified 3b71c385

feat(server): Reworking the quantization script so it's still universal (not llama specific) (#587)

Narsil committed 2 years ago

Verified 4d38a1c4

fea(launcher): debug logs (#623)

OlivierDehaene committed 2 years ago

Verified 44acf72a

fix(launcher): Rename `b-float16` to `bfloat16` in the launcher arg (#621)

Narsil committed 2 years ago

Verified bc287324

fix(server): empty_cache when stopped

OlivierDehaene committed 2 years ago

a2cf1bdb

OlivierDehaene committed 2 years ago

Verified c58a0c18

Newer Older