huggingface/text-generation-inference

Pull Requests Commits

fix: prefer recent gptq changes

drbh committed 1 year ago

b84bb19e

fix: superseed gptq changes with main

drbh committed 1 year ago

7e773b0f

fix: marlin repeat scale for fp8 and bump snapshots

drbh committed 1 year ago

3f12750a

fix: include correct exllama methods based on version

drbh committed 1 year ago

df9eb387

fix: move GPTQWeight into file to avoid circular import

drbh committed 1 year ago

e99dd84b

fix: update mamba snap and run other release tests

drbh committed 1 year ago

700e64c5

fix: update mt0, mamba and grammar tests

drbh committed 1 year ago

add7908f

fix: update deepseek and gemma tests

drbh committed 1 year ago

c3e358e8

fix: run bloom in non release and update snapshots

drbh committed 1 year ago

57efa7ab

fix: improve completions to send a final chunk with usage details (#2336)

drbh committed 1 year ago

Verified 30395b09

fix: allocate tmp based on sgmv kernel if available (#2345)

drbh committed 1 year ago

Verified 4c3f8a70

feat: validate template variables before apply and improve sliding wi… (#2403)

drbh committed 1 year ago

Verified 155f9c98

Keeping the benchmark somewhere (#2401)

Narsil committed 1 year ago

Verified 136bcc81

Add support for prefix caching to the v3 router (#2392)

danieldk committed 1 year ago

Verified 8deeaca4

Cpu dockerimage (#2367)

sywangyi committed 1 year ago

Verified b6bb1d51

Fixing import exl2 (#2399)

Narsil committed 1 year ago

Verified 84bc3d7b

Adding launcher to build. (#2397)

Narsil committed 1 year ago

Verified 730fa00e

Upgrade fbgemm (#2398)

Narsil committed 1 year ago

Verified 9c739651

nix: add router to the devshell (#2396)

danieldk committed 1 year ago

Verified 01a515de

Update flake for 9.0a capability in Torch (#2394)

danieldk committed 1 year ago

Verified 8dcc7d3f

feat: add guideline to chat request and template (#2391)

drbh committed 1 year ago

Verified 0d06aed0

Using an enum for flash backens (paged/flashdecoding/flashinfer) (#2385)

Narsil committed 1 year ago

Verified 7a48a847

flake: use rust-overlay (#2390)

danieldk committed 1 year ago

Verified 6e127dcc

Update documentation for Supported models (#2386)

Vaibhavs10 committed 1 year ago

Verified b2b9c427

flake: add fmt and clippy (#2389)

danieldk committed 1 year ago

Verified 977534bc

Using HF_HOME instead of CACHE to get token read in addition to models. (#2288)

Narsil committed 1 year ago

Verified 952b450a

Add experimental flake (#2384)

danieldk committed 1 year ago

Verified c6d5039c

Add FlashInfer support (#2354)

danieldk committed 1 year ago

Verified 7830de15

Pr 2352 ci branch (#2382)

drbh committed 1 year ago

Verified 6d06473c

Update Quantization docs and minor doc fix. (#2368)

Vaibhavs10 committed 1 year ago

Verified cb3ae302

Older