Go
Home
Pricing
FAQ
Install
Home
Pricing
FAQ
Install
Login
via GitHub
huggingface/text-generation-inference
Pull Requests
Commits
feat/parse_logs
20250708-ci-fixes
add_L4
add_api_key
add_batch_dimension
add_chunked_atn
add_chunked_attn
add_deepseekv3
add_gptq_docs
add_integration_test
add_readme_dashboard
add_tunable_prefill
add_vlm_chunking
add-chat-response-format
add-google-cloud-provider
add-quickstart-script
add-rotary-embed-tests
add-small-ttft-script
add-test-for-warmup-and-kvcache
adding_docs
adjust-mllama-test-output
adjust-where-request-max-tokens-is-defaulted
aiter_kernels
amd-ci-fx
auto_length
automodel-supports-flash-paged-attention
avoid-cuda-graph-during-warmup-if-oom
avoid-zero-seed
backends/trtllm
backends/trtllm-executor
baichuan2-13b
bnb4
bugfix/add_tools_prompt
bugfix/moe-kernels-imports
bugfix/phi-exl2
bump-client-0.6.2
bump-kernel-versions
bump-poetry-and-requirements
chunked_attn_l4
ci_amd
ci_amd2
ci_amd3
ci_amd4
ci2
ci-amihalik-update-chat-completion-messages
ci-new-cluster
ci-patch
ci-run-openai-function-calling-compatible-support
ci-update_xpu_image
ci-xpu
ci-xpu2
close_dl_thread
compat_logger
compile-grammar-in-router
cuda_ipc_allreduce
debug/gemma2
debug-gpt2
debug-request-id
debug-torch-23
debugging-timeouts
deploy/aml
dev
development-guide
dummy
enable_non_divisible_embeddings
enable-non-grammar-constrained-tools
enable-qwen2vl-video
enable-transformers-vlm
exl2
experiment/moe
explore-static-triton-kernels
explore-t4-gemma-issues
feat/add-load-test
feat/attention_sinks
feat/backend_abstraction
feat/backend_feature
feat/better_tokens
feat/cuda_12
feat/flash_decoding
feat/improve_max_tokens
feat/max_queue_size
feat/page_re_alloc
feat/parse_logs
feat/support_deepspeed
feat-backend-llamacpp
feature/machete
feature/moe-kernels
feature/no_repeat_ngram_size_ci
feature/no_repeat_ngram_size
feature/phi-3-small
feature/prefix
feature/radix-prefix-cache
feature/radix-prefix-cache-bench
feature/vlm-prefix-caching
fix/allow-top-p-0
fix/avoid_record_streams
fix_default_arg
fix_exl2
fix_fp8_llama3.2
fix_leak
fix_mistral2
fix_neox_rotary_emb
fix/op-trace-id
fix/parse-mamba-config
fix_phi3
fix-cudagraph-bug
fix-gemma-tokenization
fix-grammar-cleanup-bug
fix-grammar-fsm-batching
fix-mixtral-adapter-loading
fix-release-tests
fix-repack-for-marlin
fix-tool-call-def
fix-tp
fix-version-install
flashinfer
flashinfer-0.2.5
fp8_kvcache
fp8_rocm
gaudi_llama4_tmp
git_v2.1.0
git_v2.1.1
git_v2.2.0
git_v2.3.0
git_v2.3.1
git_v2.4.0
git_v2.4.1
git_v3.0.0
git_v3.0.1
git_v3.0.2
git_v3.1.0
git_v3.2.2
git_v3.2.3
git_v3.3.3
git_v3.3.4
git_v3.3.5
git_2.0.4
git_3.1.1
git_3.2.0
git_3.2.1
git_3.3.0
git_3.3.1
git_3.3.2
improve_defaults
improve_launcher_defaults
improve-docs
improve-dynamic-message-content
improve-json-schema-field
improve-tool-call-and-response-ids
inlcude-latest-release-on-commit-builds-tags
ipex-moe
kvrouter
kvrouter-endpoints
llama-fused-compiled-mlp
main
maintenance/docker-network
maintenance/merge-vlm-input-prep
mamba2
martinigoyanes-fix-frequency-penalty
medusa
megatron
message-more-info
mi300-temp
mllama
model_compat_log
more_logs
multi-lora
new_minor_version
nix/cargo-clippy
nix/docker2
nix_integration_tests
nix/pytorch-2.5.1
nix_test2
no_root_user
no_root_user2
op-compilation-benchmarking
origin/slind_window_fix
osanseviero-patch-1
pip-installable
pr-1869-ci-run
pr-2076-ci-run
pr-2290-ci-runner
pr-2366-ci-branch
pr-2444-ci-branch
pr-2517-ci-branch
pr-2711-ci-branch
pr-2784-ci-branch
pr-2840-ci-branch
pr-2954-ci-branch
pr-3002-ci-branch
pr-3004-ci-branch
pr-3018-ci-branch
precompile-kernels-workflow
prefix_chunk
prefix_default
proxy_sse_engine_state
quantization
quantization-0.1
refactor-lora-linear
release-3.2.4
remove_post_load_weights
response-header-metrics
revert
rocm_6.2_fixes
rocm-ci-build
router-grammar-compile
s3-cache
self-generating-docs
set-num-blocks
simpler_exllama
skip-mistral-test
speculative
streaming_conceptual
support-granite-vision
support-logit-bias-in-chat
support-phi3-small
support-phi-model
support-pre-compile-kernels
temp_work
test_docs
test_rocm
test-batch-speedup-amount
tmp_invariants
tmp_medusa
tmp_torch_compile
transformers-ci
triton_fix
trtllm/executor_stats
trtllm-stop-words
tuna
update_docs2
update_internal_version
update_peft
update_readme
update-flake-deps-and-logit-processor
update-jsonschema
upgrade_mlp_speculator
upgrade-outlines
use_g6
use_updated_kernels
vllm/setup
zstd
feat(launcher): parse all shard logs
OlivierDehaene
committed
2 years ago
7caea425
fix(docker): revert dockerfile changes
OlivierDehaene
committed
2 years ago
4096ca4e
fix(docker): fix image (#185)
OlivierDehaene
committed
2 years ago
Verified
f9047562
fix(docker): fix docker image (#184)
OlivierDehaene
committed
2 years ago
Verified
1bb39463
fix(ci): fix cosign error (#183)
OlivierDehaene
committed
2 years ago
Verified
01c0e368
fea(dockerfile): better layer caching (#159)
OlivierDehaene
committed
2 years ago
Verified
53ee09c0
fix(ci): fix ci permissions (#181)
OlivierDehaene
committed
2 years ago
Verified
12e5633c
feat(ci): faster scanning (#180)
OlivierDehaene
committed
2 years ago
Verified
c1e2ea3b
feat(ci): use large runners (#179)
OlivierDehaene
committed
2 years ago
Verified
13f1cd02
feat(ci): add Trivy and scan docker image (#178)
OlivierDehaene
committed
2 years ago
Verified
9683c37b
feat(ci): add image signing with cosign (#175)
OlivierDehaene
committed
2 years ago
Verified
643a39d5
fix(ci): fix CVE in github-slug-action (#174)
OlivierDehaene
committed
2 years ago
Verified
64347b05
fix(launcher): revert change on shard errors (#173)
OlivierDehaene
committed
2 years ago
Verified
e3a63b6f
feat(server): support sharded santacoder (#167)
OlivierDehaene
committed
2 years ago
Verified
880a76ee
feat(server): optimize decode for sane tokenizers (#170)
OlivierDehaene
committed
2 years ago
Verified
5fa8ae04
v0.5.0 (#168)
OlivierDehaene
committed
2 years ago
Verified
6f0f1d70
feat(server): support OPT models (#55)
OlivierDehaene
committed
2 years ago
Verified
f26dfd0d
feat(server): add flash attention llama (#144)
OlivierDehaene
committed
2 years ago
Verified
299217c9
feat(router): make router input validation optional (#164)
OlivierDehaene
committed
2 years ago
Verified
99879600
fix(router): use buckets for metrics histograms (#163)
OlivierDehaene
committed
2 years ago
Verified
7dec65a2
fix(rust-client): use join_all instead of select_all to hopefully fix nccl issues (#162)
OlivierDehaene
committed
2 years ago
Verified
5cddc055
feat(launcher): allow disabling hf_transfer (#161)
OlivierDehaene
committed
2 years ago
Verified
e63a21eb
feat(docker): improve flash_attention caching (#160)
OlivierDehaene
committed
2 years ago
Verified
1883d8ec
fix(server): fix escape characters in stop sequence (#155)
OlivierDehaene
committed
2 years ago
Verified
3f2542bb
docs(readme): provide link Logits Warper README (#154)
guspan-tanadi
committed
2 years ago
Verified
9122e7bd
feat(server): flash santacoder (#153)
OlivierDehaene
committed
2 years ago
Verified
c0aeb325
v0.4.3 (#152)
OlivierDehaene
committed
2 years ago
Verified
fef1a1c3
v0.4.2 (#151)
OlivierDehaene
committed
2 years ago
Verified
84722f3e
fix(server): fix flash neox rotary embeddings (#150)
OlivierDehaene
committed
2 years ago
Verified
08b7e4a2
feat(benchmark): tui based benchmarking tool (#149)
OlivierDehaene
committed
2 years ago
Verified
610bb1f9
Older