huggingface/text-generation-inference

Pull Requests Commits

Prefix caching WIP

danieldk committed 1 year ago

7735b385

Using an enum for flash backens (paged/flashdecoding/flashinfer) (#2385)

Narsil committed 1 year ago

Verified 7a48a847

flake: use rust-overlay (#2390)

danieldk committed 1 year ago

Verified 6e127dcc

Update documentation for Supported models (#2386)

Vaibhavs10 committed 1 year ago

Verified b2b9c427

flake: add fmt and clippy (#2389)

danieldk committed 1 year ago

Verified 977534bc

Using HF_HOME instead of CACHE to get token read in addition to models. (#2288)

Narsil committed 1 year ago

Verified 952b450a

Add experimental flake (#2384)

danieldk committed 1 year ago

Verified c6d5039c

Add FlashInfer support (#2354)

danieldk committed 1 year ago

Verified 7830de15

Pr 2352 ci branch (#2382)

drbh committed 1 year ago

Verified 6d06473c

Update Quantization docs and minor doc fix. (#2368)

Vaibhavs10 committed 1 year ago

Verified cb3ae302

fix: prefer hidden_activation over hidden_act in gemma2 (#2381)

drbh committed 1 year ago

Verified f8521900

Pr 2337 ci branch (#2379)

drbh committed 1 year ago

Verified 2ca59806

fix EleutherAI/gpt-neox-20b does not work in tgi (#2346)

sywangyi committed 1 year ago

Verified 689b1abb

Pr 2374 ci branch (#2378)

drbh committed 1 year ago

Verified 82d19d77

Fix the prefix for OPT model in opt_modelling.py #2370 (CI RUN) (#2371)

drbh committed 1 year ago

Verified a379d553

add gptj modeling in TGI #2366 (CI RUN) (#2372)

drbh committed 1 year ago

Verified 21267f3c

fix: fix num_ln_in_parallel_attn attribute name typo in RWConfig (#2350)

almersawi committed 1 year ago

Verified 8094ecfc

fix: prefer original layernorm names for 180B (#2365)

drbh committed 1 year ago

Verified 133015f4

fix: default num_ln_in_parallel_attn to one if not supplied (#2364)

drbh committed 1 year ago

Verified a64d407d

feat: return the generated text when parsing fails (#2353)

drbh committed 1 year ago

Verified 1768c00b

feat: prefer stop over eos_token to align with openai finish_reason (#2344)

drbh committed 1 year ago

Verified f8a5b381

feat: implement a templated endpoint for visibility into chat requests (#2333)

drbh committed 1 year ago

Verified e11f5f1c

fix: return the out tensor rather then the functions return value (#2361)

drbh committed 1 year ago

Verified 29b8d19c

feat: include local lora adapter loading docs (#2359)

drbh committed 1 year ago

Verified dd47a3da

fix: attempt forward on flash attn2 to check hardware support (#2335)

drbh committed 1 year ago

Verified 215ed3ad

Unify attention output handling (#2343)

danieldk committed 1 year ago

Verified 47447ef0

Fix cache block size for flash decoding (#2351)

danieldk committed 1 year ago

Verified 22fb1be5

enable HuggingFaceM4/idefics-9b in intel gpu (#2338)

sywangyi committed 1 year ago

Verified 9ab99374

refactor usage stats (#2339)

ErikKaum committed 1 year ago

Verified 7451041e

Pr 2290 ci run (#2329)

drbh committed 1 year ago

Verified f7f61876

Older