vllm
[Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id`
#18979

Merged

[Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id` #18979

vllm-bot merged 22 commits into vllm-project:main from eicherseiji:flash_mla_ray_fix

eicherseiji changed the title ~~Fix FlashMLA detection in ray environment~~ Avoid a crash in is_flashmla_supported() by handling Platform.get_device_capability()'s optional return value 266 days ago

eicherseiji requested a review from

WoosukKwon 265 days ago

eicherseiji requested a review from

robertgshaw2-redhat 265 days ago

eicherseiji requested a review from

njhill 265 days ago

eicherseiji requested a review from

ywang96 265 days ago

eicherseiji requested a review from

comaniac 265 days ago

eicherseiji requested a review from

alexm-redhat 265 days ago

mergify added v1

eicherseiji changed the title ~~Avoid a crash in is_flashmla_supported() by handling Platform.get_device_capability()'s optional return value~~ Avoid a crash in is_flashmla_supported() by moving block_size fixup to GPU worker 265 days ago

mergify added needs-rebase

eicherseiji force pushed 265 days ago

mergify removed needs-rebase

eicherseiji changed the title ~~Avoid a crash in is_flashmla_supported() by moving block_size fixup to GPU worker~~ [Regression][Bugfix] Avoid a crash in is_flashmla_supported() by moving block_size fixup to GPU worker 265 days ago

eicherseiji changed the title ~~[Regression][Bugfix] Avoid a crash in is_flashmla_supported() by moving block_size fixup to GPU worker~~ [Regression][Bugfix] Avoid a crash in is_flashmla_supported() by moving FlashMLA block_size fixup to GPU worker 265 days ago

njhill requested a review from

LucasWilkinson 265 days ago

njhill commented on 2025-06-04

eicherseiji force pushed 264 days ago

eicherseiji requested a review from

hmellor 263 days ago

eicherseiji requested a review from

mgoin 263 days ago

eicherseiji requested a review from

DarkLight1337 263 days ago

eicherseiji requested a review from

tlrmchlsmth 263 days ago

eicherseiji requested a review from

simon-mo 263 days ago

eicherseiji requested a review from

aarnphm 263 days ago

eicherseiji requested a review from

zhuohan123 263 days ago

eicherseiji requested a review from

youkaichao 263 days ago

mergify added documentation

mergify added ci/build

mergify added frontend

mergify added multi-modality

mergify added tpu

mergify added tool-calling

eicherseiji force pushed 263 days ago

mergify removed tpu

eicherseiji force pushed 263 days ago

eicherseiji force pushed 261 days ago

eicherseiji force pushed 260 days ago

mergify added needs-rebase

eicherseiji force pushed to eea7dcbe 260 days ago

mergify removed needs-rebase

eicherseiji force pushed to 9f24ef5b 260 days ago

ruisearch42 commented on 2025-06-11

eicherseiji changed the title ~~[Regression][Bugfix] Avoid a crash in is_flashmla_supported() by moving FlashMLA block_size fixup to GPU worker~~ [Bugfix] Move hardware-dependent configuration resolution (FlashMLA, `dtype: auto`) to worker 258 days ago

eicherseiji changed the title ~~[Bugfix] Move hardware-dependent configuration resolution (FlashMLA, `dtype: auto`) to worker~~ [Bugfix] Move hardware-dependent configuration resolution (FlashMLA capability, `dtype: 'auto'`) to worker 258 days ago

kouroshHakha commented on 2025-06-12

eicherseiji force pushed 257 days ago

kouroshHakha added ready

kouroshHakha commented on 2025-06-16

eicherseiji force pushed 253 days ago

kouroshHakha approved these changes on 2025-06-16

eicherseiji force pushed 253 days ago

eicherseiji force pushed 252 days ago

kouroshHakha commented on 2025-06-18

ruisearch42 commented on 2025-06-18

eicherseiji force pushed 247 days ago

kouroshHakha approved these changes on 2025-06-23

ruisearch42 commented on 2025-06-24

eicherseiji changed the title ~~[Bugfix] Move hardware-dependent configuration resolution (FlashMLA capability, `dtype: 'auto'`) to worker~~ [Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id` 246 days ago

Fix FlashMLA detection in ray environment

22fdba14

Move FlashMLA capability check to GPU worker

979129b7

Unit test

5055e5bd

reate helper function _resolve_hardware_dependent_config

9e943ea0

Add V0 support under env flag

b812cd17

Parameterize _get_and_verify_dtype by defer_to_worker

4e4e092b

Change parameter name to 'defer_auto_to_worker'

baadec88

Add _resolve_hardware_dependent_config for V0 Worker

8df72535

Resolve lora dtype

b2fea3c6

Move V1 supported type check into auto resolution's if block

6500644d

Move config resolution to WorkerBase

8c308410

Fix kwargs list

7ff5daea

Inline init_config with init_worker

ceccc0d1

Testing without V0-specific flag

c1912995

Remove whitespace changes

190f9f75

Avoid modifying _get_and_verify_dtype signature for the sake of testing

ab584319

Only check V1 supported dtypes in V1

5f87e531

Add V0 support under env flag

b812cd17

Change parameter name to 'defer_auto_to_worker'

baadec88

Add _resolve_hardware_dependent_config for V0 Worker

8df72535

Resolve lora dtype

b2fea3c6

Move V1 supported type check into auto resolution's if block

6500644d

Move config resolution to WorkerBase

8c308410

Fix kwargs list

7ff5daea

Inline init_config with init_worker

ceccc0d1

Testing without V0-specific flag

c1912995

Avoid modifying _get_and_verify_dtype signature for the sake of testing

ab584319

Move config fixup logic to VllmConfig.resolve_config_with_hardware, f…

e77592cc

Support inplace model weights loading

8e0f77b6

Try resolving dtype on worker in ray

d5f88bb4

Treat empty device control env var as unset

afe0e010

Allow empty string CUDA_VISIBLE_DEVICES

66bd1fb2

eicherseiji force pushed to 66bd1fb2 245 days ago

ruisearch42 approved these changes on 2025-06-25

kouroshHakha approved these changes on 2025-06-25

aarnphm approved these changes on 2025-06-25

vllm-bot merged 65397e40 into main 244 days ago

Reviewers

aarnphm

ruisearch42

kouroshHakha

njhill

WoosukKwon

robertgshaw2-redhat

ywang96

comaniac

alexm-redhat

LucasWilkinson

hmellor

mgoin

DarkLight1337

tlrmchlsmth

simon-mo

zhuohan123

youkaichao

Assignees

No one assigned

Labels

documentation frontend ready ci/build v1 multi-modality tool-calling

Milestone

No milestone

vllm [Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id` #18979 Merged

[Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id` #18979

vllm
[Bugfix] Allow `CUDA_VISIBLE_DEVICES=''` in `Platform.device_id_to_physical_device_id`
#18979

Merged