[SGLang CI] Fix the VLM teardown error on CPU (follow-up of #50913) (#50914)
[SGLang CI] Fix the VLM teardown error on CPU instead of deselecting a test
The vision-language error comes from TestQwenVLUnderstandsImage.tearDownClass calling
empty_gpu_cache() (torch.accelerator.empty_cache(), which raises without an accelerator), not
from test_colliding_low_hash_bits_do_not_reuse_image_kv itself (it passes); pytest reports it
on the class's last test. The deselect from #50913 also never matched (pytest's rootdir is
sglang/test, so node ids don't start with test/). Revert it and make empty_gpu_cache() a no-op
without an accelerator in the conftest the workflow already writes.
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>