huggingface/text-generation-inference

Pull Requests Commits

Use FP8 KV cache when specified by compressed-tensors (#2761)

danieldk committed 1 year ago

Verified 72ab60fd

Move JSON grammar -> regex grammar conversion to the router (#2772)

danieldk committed 1 year ago

Verified 289aa485

feat: concat the adapter id to the model id in chat response (#2779)

drbh committed 1 year ago

Verified c637d68d

chore: prepare 2.4.1 release (#2773)

OlivierDehaene committed 1 year ago

Verified 780531ec

chore: Update to marlin-kernels 0.3.6 (#2771)

danieldk committed 1 year ago

Verified e87893d3

feat: add payload limit (#2726)

OlivierDehaene committed 1 year ago

Verified ab7ccf5b

feat: Add automatic nightly benchmarks (#2591)

Hugoch committed 1 year ago

Verified d5bc6a20

Remove guideline from API (#2762)

Wauplin committed 1 year ago

Verified d012f229

docs: Add a README section about using Nix (#2767)

danieldk committed 1 year ago

Verified c5b5b3a1

fix: tweak grammar test response (#2769)

drbh committed 1 year ago

Verified faa10ad0

fix: incomplete generations w/ single tokens generations and models that did not support chunking (#2770)

OlivierDehaene committed 1 year ago

Verified 8e0c161d

nix: downgrade to outlines 0.1.3 (#2768)

danieldk committed 1 year ago

Verified 3c544886

fix: set outlines version to 0.1.3 to avoid caching serialization issue (#2766)

drbh committed 1 year ago

Verified 6ee8d6dd

nix: build and cache impure devshells (#2765)

danieldk committed 1 year ago

Verified 07bed530

Add support for wNa16 int 2:4 compressed-tensors checkpoints (#2758)

danieldk committed 1 year ago

Verified 46a5a7e7

nix: update for outlines 0.1.4 (#2764)

danieldk committed 1 year ago

Verified 2fda8845

Install compressed-tensors in Docker CPU builds

danieldk committed 1 year ago

45013b60

fix: adjust llama MLP name from dense to mlp to correctly apply lora (#2760)

drbh committed 1 year ago

Verified bd6e8b3c

PR 2634 CI - Fix the tool_choice format for named choice by adapting OpenAIs scheme (#2645)

drbh committed 1 year ago

Verified 5489406c

Update to moe-kernels 0.7.0 (#2720)

danieldk committed 1 year ago

Verified 2007a947

Simplify two ipex conditions (#2755)

danieldk committed 1 year ago

Verified b4ec427a

feat: support flash attention 2 in qwen2 vl vision blocks (#2721)

drbh committed 1 year ago

Verified 38cff84a

Add support for compressed-tensors w8a8 int checkpoints (#2745)

danieldk committed 1 year ago

Verified 3c9df21f

add ipex moe implementation to support Mixtral and PhiMoe (#2707)

sywangyi committed 1 year ago

Verified a5ecd6e5

fix: improve find_segments via numpy diff (#2686)

drbh committed 1 year ago

Verified fea62e92

Remove vLLM dependency for CUDA (#2751)

danieldk committed 1 year ago

Verified 52e48739

feat: return streaming errors as an event formatted for openai's client (#2668)

drbh committed 1 year ago

Verified 6489f852

Upgrading our deps. (#2750)

Narsil committed 1 year ago

Verified 34a3bded

Upgrade outlines to 0.1.1 (#2742)

aW3st committed 1 year ago

Verified 4580ced0

fix response type of document for Text Generation Inference (#2743)

jitokim committed 1 year ago

Verified 003eaec0

Newer Older