huggingface/text-generation-inference

Pull Requests Commits

fix: Fix PR comments

Hugoch committed 1 year ago

Verified a258e8f6

Merge branch 'main' into feat/add-load-test

Hugoch committed 1 year ago

Verified 345d4736

fix: Fix PR comments (remove Jinja)

Hugoch committed 1 year ago

Verified 4e78b006

Tied embeddings in MLP speculator. (#2473)

Narsil committed 1 year ago

Verified d9fbbaaf

update doc with intel cpu part (#2420)

sywangyi committed 1 year ago

Verified 9883f3b4

feat: add /v1/models endpoint (#2433)

drbh committed 1 year ago

Verified d5202c46

Lots of improvements (Still 2 allocators) (#2449)

Narsil committed 1 year ago

Verified e415b690

nix: build Torch against MKL and various other improvements (#2469)

danieldk committed 1 year ago

Verified 4e821c00

fix: improve regex expression (#2468)

drbh committed 1 year ago

Verified 8f99f165

fix: bump minijinja version and add test for llama 3.1 tools (#2463)

drbh committed 1 year ago

Verified 21187c27

Fixing CI. (#2462)

Narsil committed 1 year ago

Verified 2788d41a

Pr 2451 ci branch (#2454)

drbh committed 1 year ago

Verified cfa73b5c

Fix: don't apply post layernorm in SiglipVisionTransformer (#2459)

drbh committed 1 year ago

Verified 30be1884

nix: add default package (#2453)

danieldk committed 1 year ago

Verified f3c5d7d9

nix: add awq-inference-engine as server dependency (#2442)

danieldk committed 1 year ago

Verified 358ceb67

Adding eetq to flake. (#2438)

Narsil committed 1 year ago

Verified 310778e0

nix: add `text-generation-benchmark` to pure devshell (#2431)

danieldk committed 1 year ago

Verified 94744150

nix: add pure server to flake, add both pure and impure devshells (#2430)

danieldk committed 1 year ago

Verified f5f11b79

Prefix caching (#2402)

Narsil committed 1 year ago

Verified b70ae096

nix: update to CUDA 12.4 (#2429)

danieldk committed 1 year ago

Verified 38773453

All integration tests back everywhere (too many failed CI). (#2428)

Narsil committed 1 year ago

Verified e4201f44

doc: Add metrics documentation and add a 'Reference' section (#2230)

Hugoch committed 1 year ago

Verified 53729b74

Narsil committed 1 year ago

Verified cb0a2948

Further fixes. (#2426)

Narsil committed 1 year ago

Verified c7ab1810

Improve the Consuming TGI + Streaming docs. (#2412)

Vaibhavs10 committed 1 year ago

Verified 99b662f8

nix: try to reduce the number of Rust rebuilds (#2424)

danieldk committed 1 year ago

Verified 1411bfb9

Upgrading the tests to match the current workings. (#2423)

Narsil committed 1 year ago

Verified 1b0aa062

Fixing exl2 and other quanize tests again. (#2419)

Narsil committed 1 year ago

Verified 57b34958

nix: build router incrementally (#2422)

danieldk committed 1 year ago

Verified 9aaa12e7

More fixes trtllm (#2342)

mfuntowicz committed 1 year ago

Verified 3f385991

Older