huggingface/text-generation-inference

Pull Requests Commits

Adding A100 compute. (#2806)

Narsil committed 1 year ago

Verified d96dcb17

Auto max prefill (#2797)

Narsil committed 1 year ago

Verified 5df80590

feat: auto max_new_tokens (#2803)

OlivierDehaene committed 1 year ago

Verified 8c3669b2

use oneapi 2024 docker image directly for xpu (#2793)

sywangyi committed 1 year ago

Verified 6685e8fc

fix: avoid setting use_sgmv if no kernels present (#2796)

drbh committed 1 year ago

Verified e0db6333

Saving some VRAM. (#2790)

Narsil committed 1 year ago

Verified b57f3703

Sync (most) server dependencies with Nix (#2782)

danieldk committed 1 year ago

Verified 2003d8be

fix: only use eos_token_id as pad_token_id if int (#2774)

dvrogozh committed 1 year ago

Verified 535149d8

fix: add merge-lora arg for model id (#2788)

drbh committed 1 year ago

Verified 2c74c556

Removing ../ that broke the link (#2789)

Getty committed 1 year ago

Verified a35d1e6f

Fix doc. (#2792)

Narsil committed 1 year ago

Verified 1d2cb356

Support continue final message (#2733)

drbh committed 1 year ago

Verified d4718051

Fix: docs typo (#2777)

jp1924 committed 1 year ago

Verified caff779d

upgrade ipex cpu to fix coredump in tiiuae/falcon-7b-instruct (pageat… (#2778)

sywangyi committed 1 year ago

Verified 892a26e5

Use FP8 KV cache when specified by compressed-tensors (#2761)

danieldk committed 1 year ago

Verified 72ab60fd

Move JSON grammar -> regex grammar conversion to the router (#2772)

danieldk committed 1 year ago

Verified 289aa485

feat: concat the adapter id to the model id in chat response (#2779)

drbh committed 1 year ago

Verified c637d68d

chore: prepare 2.4.1 release (#2773)

OlivierDehaene committed 1 year ago

Verified 780531ec

chore: Update to marlin-kernels 0.3.6 (#2771)

danieldk committed 1 year ago

Verified e87893d3

feat: add payload limit (#2726)

OlivierDehaene committed 1 year ago

Verified ab7ccf5b

feat: Add automatic nightly benchmarks (#2591)

Hugoch committed 1 year ago

Verified d5bc6a20

Remove guideline from API (#2762)

Wauplin committed 1 year ago

Verified d012f229

docs: Add a README section about using Nix (#2767)

danieldk committed 1 year ago

Verified c5b5b3a1

fix: tweak grammar test response (#2769)

drbh committed 1 year ago

Verified faa10ad0

fix: incomplete generations w/ single tokens generations and models that did not support chunking (#2770)

OlivierDehaene committed 1 year ago

Verified 8e0c161d

nix: downgrade to outlines 0.1.3 (#2768)

danieldk committed 1 year ago

Verified 3c544886

fix: set outlines version to 0.1.3 to avoid caching serialization issue (#2766)

drbh committed 1 year ago

Verified 6ee8d6dd

nix: build and cache impure devshells (#2765)

danieldk committed 1 year ago

Verified 07bed530

Add support for wNa16 int 2:4 compressed-tensors checkpoints (#2758)

danieldk committed 1 year ago

Verified 46a5a7e7

nix: update for outlines 0.1.4 (#2764)

danieldk committed 1 year ago

Verified 2fda8845

Newer Older