transformers
a61d5f9e - [DeepSeekV2] Fix integration tests OOM: use device_map=auto instead of 8-bit quantization (#47991)

Commit
90 days ago
[DeepSeekV2] Fix integration tests OOM: use device_map=auto instead of 8-bit quantization (#47991) * [DeepSeekV2] Fix integration tests: use device_map=auto, remove 8-bit quantization The model (DeepSeek-V2-Lite) exceeds A10G VRAM even with 8-bit quantization. Switch to device_map="auto" (bfloat16, no quantization) which offloads excess weights to CPU. Update expected logit values accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [DeepSeekV2] Update expected values for device_map=auto bfloat16 loading Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [DeepSeekV2] Fix curly apostrophe in batch_fa2 expected text, add tearDown cleanup Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [DeepSeekV2] Round expected logit values to 4 decimal places Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
Parents
Loading