huggingface/text-generation-inference

Pull Requests Commits

feat(server): batch tokenization for flash causal lm (#411)

OlivierDehaene committed 2 years ago

Verified 6abec14a

feat(server): only compute prefill logprobs when asked (#406)

OlivierDehaene committed 2 years ago

Verified 895c5f15

feat(launcher): parse oom signal (#404)

OlivierDehaene committed 2 years ago

Verified 83b84486

feat(sagemaker): add trust remote code to entrypoint (#394)

OlivierDehaene committed 2 years ago

Verified 62fc4010

OlivierDehaene committed 2 years ago

e7248fe9

feat(server): load santacoder/starcoder models with safetensors (#393)

OlivierDehaene committed 2 years ago

Verified 95d35469

feat(server): remove trust_remote_code requirement for falcon models (#396)

OlivierDehaene committed 2 years ago

Verified c0928e6f

fix(server): fix has_position_ids (#395)

OlivierDehaene committed 2 years ago

Verified d69a0633

OlivierDehaene committed 2 years ago

db2ebe39

fix(server): fix bnb quantization for CausalLM models (#385)

OlivierDehaene committed 2 years ago

Verified 337afb28

feat(server): add retry on download (#384)

OlivierDehaene committed 2 years ago

Verified 87dc034b

increase health checks

OlivierDehaene committed 2 years ago

444400b4

OlivierDehaene committed 2 years ago

081b9265

feat(server): support RefinedWeb models (#379)

OlivierDehaene committed 2 years ago

Verified b8b950b3

fix(server): fix quantization

OlivierDehaene committed 2 years ago

bf7f1d54

fix(launcher): parse num cuda devices from CUDA_VISIBLE_DEVICES and NVIDIA_VISIBLE_DEVICES

OlivierDehaene committed 2 years ago

49a6c8c1

fix(launcher): parse num cuda devices from CUDA_VISIBLE_DEVICES and NVIDIA_VISIBLE_DEVICES

OlivierDehaene committed 2 years ago

146e72c3

Fix issue when load AutoModelForSeq2SeqLM model (#370)

CL-Shang committed 2 years ago

Verified 5fde8d99

feat(server): support vectorized warpers in flash causal lm (#317)

OlivierDehaene committed 2 years ago

Verified 62f91f78

feat(benchmarker): add summary tables (#368)

OlivierDehaene committed 2 years ago

Verified 951930fb

feat: decrease IPC proto size (#367)

OlivierDehaene committed 2 years ago

Verified 218c9ada

OlivierDehaene committed 2 years ago

Verified d31562f3

feat(router): log input/ouput at debug level (#364)

OlivierDehaene committed 2 years ago

Verified 94200538

feat(server): support trust_remote_code (#363)

OlivierDehaene committed 2 years ago

Verified e3e487dc

feat(server): do not use device_map auto on single GPU (#362)

OlivierDehaene committed 2 years ago

Verified e9669a40

feat(server): support fp16 for t5 (#360)

OlivierDehaene committed 2 years ago

Verified cfaa8580

chore(sever): update requirements (#357)

OlivierDehaene committed 2 years ago

Verified 94377efa

feat: add nightly load testing (#358)

OlivierDehaene committed 2 years ago

Verified 5f67923c

fix(ci): fix security group (#359)

oOraph committed 2 years ago

Verified 0a649478

fix(server): t5 cannot run in f16 (#356)

OlivierDehaene committed 2 years ago

Verified 4f4c9c16

Newer Older