vllm
[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments
#26315

Merged

[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments #26315

vllm-bot merged 73 commits into vllm-project:main from MatthewBonanni:add-attention-config

MatthewBonanni requested a review from

simon-mo 91 days ago

MatthewBonanni requested a review from

WoosukKwon 91 days ago

MatthewBonanni requested a review from

youkaichao 91 days ago

MatthewBonanni requested a review from

robertgshaw2-redhat 91 days ago

MatthewBonanni requested a review from

mgoin 91 days ago

MatthewBonanni requested a review from

tlrmchlsmth 91 days ago

MatthewBonanni requested a review from

houseroad 91 days ago

MatthewBonanni requested a review from

hmellor 91 days ago

MatthewBonanni requested a review from

yewentao256 91 days ago

MatthewBonanni requested a review from

ProExpertProg 91 days ago

gemini-code-assist commented on 2025-10-06

chatgpt-codex-connector commented on 2025-10-06

MatthewBonanni requested a review from

gshtras 91 days ago

MatthewBonanni requested a review from

LucasWilkinson 91 days ago

mergify added v1

MatthewBonanni requested a review from

jikunshang 91 days ago

mergify added rocm

mergify added needs-rebase

hmellor requested changes on 2025-10-09

mergify added nvidia

Add AttentionConfig and --attention-backend CLI argument

4bc2ee95

Add documentation for environment variable compatibility

cc130f5f

Add deprecation warnings for attention environment variables

a6161f54

Limit deprecation warning to VLLM_ATTENTION_BACKEND only

13727f39

Replace attention env var references with AttentionConfig

cd720ddd

Replace remaining VLLM_ATTENTION_BACKEND env var usages

3cd41b73

Remove forced_attn_backend global, use AttentionConfig directly

92c15d16

Replace remaining attention env var references with AttentionConfig

bdeab7ab

Apply suggestion from @gemini-code-assist[bot]

8af62383

MatthewBonanni force pushed from 52faefd4 to 8af62383 46 days ago

MatthewBonanni requested a review from

pavanimajety 46 days ago

mergify removed needs-rebase

fix pre-commit

f605638f

use get_kwargs

ebadc2c3

handle envs differently

30bc9e39

backend handled elsewhere

96c914cc

remove old comments

53db82fb

remove old comments

92c673b9

change comment back

af5148ef

update hash

22ba47cd

hmellor commented on 2025-11-21

remove comment

b7bc45e9

Update vllm/config/attention.py

5b296dc4

comment

9f7522e3

move to __post_init__

592b6438

hmellor commented on 2025-11-21

remove unnecessary cache

2d017ae0

Update vllm/engine/arg_utils.py

5c2aa2da

Update vllm/engine/arg_utils.py

5b6f35dc

use enum instead of str

eb25b97b

Update vllm/config/attention.py

395a9ba0

convert str to enum

c0e7fe39

Update vllm/attention/utils/fa_utils.py

93090802

handle everything in attention config

e08f21fe

simplify

be4395ec

Merge branch 'main' into add-attention-config

46aa50f0

MatthewBonanni requested a review from

hmellor 45 days ago

remove --attention-backend

178e903f

update comment

aa39faeb

use config

4ba23673

MatthewBonanni requested a review from

tjtanaa 45 days ago

clean up env variable usage

2a42d293

MatthewBonanni requested a review from

DarkLight1337 45 days ago

MatthewBonanni requested a review from

ywang96 45 days ago

clean up env variable usage

affffe17

MatthewBonanni changed the title ~~[Attention][UX] Add AttentionConfig and change attention backend to CLI argument~~ [Attention][UX][1/N] Add AttentionConfig and change attention backend to CLI argument 45 days ago

Merge branch 'main' into add-attention-config

2746c5c0

MatthewBonanni changed the title ~~[Attention][UX][1/N] Add AttentionConfig and change attention backend to CLI argument~~ [Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments 45 days ago

Add --attention-backend back

0495e61c

mergify added needs-rebase

hmellor commented on 2025-11-24

Update vllm/config/attention.py

e60bdc52

Update vllm/platforms/cuda.py

ac28e3be

WIP: 4d01b64284 [Bugfix] - Add Trace Headers to Beam Search Path (#29…

c0646e11

mergify removed needs-rebase

hmellor approved these changes on 2025-11-24

hmellor enabled auto-merge (squash) 42 days ago

github-actions added ready

mergify added needs-rebase

Merge branch 'main' into add-attention-config

6ca97a9e

disabled auto-merge 42 days ago
Head branch was pushed to by a user without write access

mergify removed needs-rebase

mgoin commented on 2025-11-24

Rename flashinfer_disable_q_quantization to disable_flashinfer_q_quan…

10bd4315

Merge branch 'main' into add-attention-config

d9347bd2

lazy import

49051f06

fix missed reference

9f981113

typo

40e2a647

rebuild dataclass

d0d910b0

Revert "rebuild dataclass"

614a0a1f

undo TYPE_CHECKING import

ef70b675

ProExpertProg commented on 2025-11-25

mergify added needs-rebase

Merge branch 'main' into add-attention-config

bd0a5786

mergify removed needs-rebase

add -ac shorthand

40b60d9f

update test to use AttentionConfig directly

941be4f5

Merge branch 'main' into add-attention-config

be696453

validate during __post_init__ (env variable handling)

c93d923c

fix import

d3016e9d

update test

bc44febe

mergify added needs-rebase

tjtanaa commented on 2025-11-29

Merge branch 'main' into add-attention-config

1fe94ff1

mergify removed needs-rebase

Merge branch 'main' into add-attention-config

48f67953

Merge branch 'main' into add-attention-config

6c1d59b7

Merge branch 'main' into add-attention-config

154c466e

fix cache behavior

86282173

Merge branch 'main' into add-attention-config

8d78f1f0

Fix test_nixl_connector

a1bc043b

MatthewBonanni requested a review from

ApostaC 34 days ago

mergify added kv-connector

Merge branch 'main' into add-attention-config

ce60e37e

Fix Blackwell compile test

132bd749

MatthewBonanni requested a review from

tdoublep 33 days ago

MatthewBonanni requested a review from

zhuohan123 33 days ago

MatthewBonanni requested a review from

alexm-redhat 33 days ago

MatthewBonanni requested a review from

njhill 33 days ago

Merge branch 'main' into add-attention-config

ecf68f26

Merge branch 'main' into add-attention-config

d20c9fcb

update version to allow 1 release

d0abdba9

Fix test_attention_backends

39d65a69

fix test_gpu_model_runner

e8c7e780

hmellor commented on 2025-12-04

Fix FlashInferMetadataBuilder.__init__ and revert "Fix test_attention…

a971d7c2

Merge branch 'main' into add-attention-config

99544358

LucasWilkinson added ready-run-all-tests

vllm-bot merged 66e674cd into main 31 days ago

MatthewBonanni deleted the add-attention-config branch 31 days ago

Reviewers

hmellor

tjtanaa

ProExpertProg

mgoin

chatgpt-codex-connector

gemini-code-assist

simon-mo

WoosukKwon

youkaichao

robertgshaw2-redhat

tlrmchlsmth

houseroad

yewentao256

gshtras

LucasWilkinson

jikunshang

pavanimajety

DarkLight1337

ywang96

ApostaC

tdoublep

zhuohan123

alexm-redhat

njhill

Assignees

No one assigned

Labels

rocm ready v1 kv-connector nvidia ready-run-all-tests

Milestone

No milestone

vllm [Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments #26315 Merged

[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments #26315

vllm
[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments
#26315

Merged