huggingface/text-generation-inference

Pull Requests Commits

feat(server): auto max_batch_total_tokens for flash att models (#630)

OlivierDehaene committed 2 years ago

Verified fe80f536

fix(server): fix llamav2 config (#635)

OlivierDehaene committed 2 years ago

Verified 5e6ddfd6

OlivierDehaene committed 2 years ago

Verified cf83f9b6

feat(server): add support for llamav2 (#633)

Narsil committed 2 years ago

Verified 211b211e

feat(server): flash attention v2 (#624)

OlivierDehaene committed 2 years ago

Verified 3b71c385

feat(server): Reworking the quantization script so it's still universal (not llama specific) (#587)

Narsil committed 2 years ago

Verified 4d38a1c4

fea(launcher): debug logs (#623)

OlivierDehaene committed 2 years ago

Verified 44acf72a

fix(launcher): Rename `b-float16` to `bfloat16` in the launcher arg (#621)

Narsil committed 2 years ago

Verified bc287324

fix(server): empty_cache when stopped

OlivierDehaene committed 2 years ago

a2cf1bdb

OlivierDehaene committed 2 years ago

Verified c58a0c18

fix(server): blacklist local files (#609)

OlivierDehaene committed 2 years ago

Verified 5b9de4a1

docs: README: Add logo + baseline (#611)

gary149 committed 2 years ago

Verified c8b077be

feat(router): explicit warning if revision is not set (#608)

OlivierDehaene committed 2 years ago

Verified 982ce322

feat(launcher): add arg validation and drop subprocess (#595)

OlivierDehaene committed 2 years ago

Verified b7327205

GPTQ Env vars: catch correct type of error (#596)

ssmi153 committed 2 years ago

Verified 36285595

feat(server): empty cache on errors

OlivierDehaene committed 2 years ago

f2f0289f

feat(server): Implements sharding for non divisible `vocab_size`. (#583)

Narsil committed 2 years ago

Verified 67347950

fix(server): Bug fixes for GPTQ_BITS environment variable passthrough (#590)

ssmi153 committed 2 years ago

Verified 2c4bf882

fix(server): Adding logger import to t5_modeling.py (#585)

akowalsk committed 2 years ago

Verified 7f907222

fix(server): T5 weights names. (#582)

Narsil committed 2 years ago

Verified db4efbf4

chore: migrate ci region for more availability. (#581)

Narsil committed 2 years ago

Verified f063ebde

feat(server): Support for env value for GPTQ_BITS and GPTQ_GROUPSIZE. (#580)

Narsil committed 2 years ago

Verified 5bd2ab65

fix(server): Fixing RW code (it's remote code so the Arch checking doesn't work to see which weights to keep). (#579)

Narsil committed 2 years ago

Verified f0181436

feat: better errors for warmup and TP (#575)

OlivierDehaene committed 2 years ago

Verified b4024edd

fix(server): harden the weights choice to save on disk. (#561)

Narsil committed 2 years ago

Verified e943a294

OlivierDehaene committed 2 years ago

Verified 31b36cca

fix(server): decrease memory fragmentation (#557)

OlivierDehaene committed 2 years ago

Verified c4bb5264

feat(router): add argument for hostname in router (#545) (#550)

OlivierDehaene committed 2 years ago

Verified 6f429427

feat(server): use latest flash attention commit (#543)

OlivierDehaene committed 2 years ago

Verified 31e2253a

fix(server): avoid errors for very small top_p values (#544)

njhill committed 2 years ago

Verified e4b26aa1

Newer Older