huggingface/text-generation-inference

Pull Requests Commits

Improvments within mamba.

Narsil committed 2 years ago

a3c45da0

feat: add optimization and first pass of integration test

drbh committed 2 years ago

5b6f9259

feat: use fused kernel in forward pass

drbh committed 2 years ago

966f3ba3

feat: build custom selective-scan kernels

drbh committed 2 years ago

c2681b2b

feat: prefer custom model and produce correct output

drbh committed 2 years ago

1c32d53f

feat: mvp single inference and explore integration

drbh committed 2 years ago

35939a28

feat: initial weight load

drbh committed 2 years ago

bcf14573

feat: conditionally toggle chat on invocations route (#1454)

drbh committed 2 years ago

Verified 98e5faff

chore: bump rust version and annotate/fix all clippy warnings (#1455)

drbh committed 2 years ago

Verified becd0997

feat: support raise_exception, bos and eos tokens (#1450)

drbh committed 2 years ago

Verified 3ccb3bb0

feat: supports openai chat completions API (#1427)

drbh committed 2 years ago

Verified 0eabc835

Return prompt vs generated tokens. (#1436)

Narsil committed 2 years ago

Verified ac08b4ef

Fix local load for Medusa (#1420)

PYNing committed 2 years ago

Verified da27fbdf

fix: follow base model for tokenizer in router (#1424)

OlivierDehaene committed 2 years ago

Verified fbeb1c44

Fix missing make target platform for local install: 'install-flash-attention-v2' (#1414)

deepily committed 2 years ago

Verified 91d72675

fix: fix local loading for .bin models (#1419)

OlivierDehaene committed 2 years ago

Verified 564f2a3b

docs: update required CUDA version to 12.2

OlivierDehaene committed 2 years ago

3f9b3f45

OlivierDehaene committed 2 years ago

630800ee

Fix local load for peft (#1373)

Narsil committed 2 years ago

Verified 529d7c25

feat: update exllamav2 kernels (#1370)

OlivierDehaene committed 2 years ago

Verified 564199ba

docs: Change URL for Habana Gaudi support in doc (#1343)

regisss committed 2 years ago

Verified 987c959f

Peft safetensors. (#1364)

Narsil committed 2 years ago

Verified eb8923a9

fix: fix gpt-q with groupsize = -1 (#1358)

OlivierDehaene committed 2 years ago

Verified d077150e

fix: fix offline (#1341) (#1347)

OlivierDehaene committed 2 years ago

Verified 8428ed10

fix: fix logic if sliding window key is not present in config (#1352)

OlivierDehaene committed 2 years ago

Verified 1b1bfa49

feat: relax mistral requirements (#1351)

OlivierDehaene committed 2 years ago

Verified 9b56d3fb

OlivierDehaene committed 2 years ago

f3aea78f

fix: max_past default value must be -1, not 0 (#1348)

OlivierDehaene committed 2 years ago

Verified 37555cf4

fix: only keep stop sequence buffer if we have some

OlivierDehaene committed 2 years ago

9b78a6ee

fix: slice stopping criteria buffer

OlivierDehaene committed 2 years ago

80a69204

Older