text-generation-inference
Gaudi: clean cuda/rocm code in hpu backend, enable flat_hpu
#3113
Merged
Merged
Gaudi: clean cuda/rocm code in hpu backend, enable flat_hpu #3113
Narsil
merged
27 commits into
huggingface:main
from
sywangyi:gaudi_backend_pa
clean cuda/rocm code in hpu backend, enable flat_hpu
201dc629
fix TP in pageattn
b7fea6fc
adjust block table in hpu to improve performance
5d365394
enable all the model. not testet yet
a07e7437
use tensor cache in hpu graph to avoid replay issue
6bbe24d9
baptistecolle
commented
on 2025-03-17
baptistecolle
commented
on 2025-03-17
baptistecolle
commented
on 2025-03-17
add moe support, fix qwen/mistral/mixtral crash
5cd1c93c
fix phimoe issue
073f7939
gpt_bigcode could also go pageattn
2cde30de
enable dbrx remove some unused code
2074d051
Merge branch 'main' into gaudi_backend_pa
d5b78ba1
multi-modality initial PR
f95aa426
adjust warmup and enable vlm
36b6612f
fix incorrect output in qwen2 idefics if hpu graph is used
fdf0733f
remove unused quantization code and enable awq/gptq int4
9914ffe1
fix gptq issue
8d221b7b
enable fp8
69773767
warmup prefill
fd70ad70
add warmup_decode
ba7a131e
warmup decode
7900be5a
remove block_tables and prefill_cache_indices which will lead to dyna…
1508ee8d
Merge branch 'main' into gaudi_backend_pa
7914e980
fix comment
787dbe98
missing gptj change...
376e0507
fix some issue
f0e5faec
remove torch.where to fix incorrect output in hpu graph model
c55a8cae
baptistecolle
changed the title
Merge branch 'main' into gaudi_backend_pa
610dd200
match the latest vllm_extension ops
4cdc34ec
regisss
approved these changes
on 2025-04-14
Reviewers
Assignees
No one assigned
Labels
gaudi
Milestone
No milestone
Login to write a write a comment.
Login via GitHub