optimize infer_auto_device_map for multi-GPU allocation #3321
Nech-C
wants to merge
5 commits into
huggingface:main
from
Nech-C:feature/infer-auto-device-map-multi-gpu-alloc-optim
feat: allow infer_auto_device_map to avoid unnecessary memory reserva…
00dbeffa
Merge branch 'main' into feature/infer-auto-device-map-multi-gpu-allo…
f4de3c0f
Merge remote-tracking branch 'upstream/main' into feature/infer-auto-…
bb22ffc8
fix: let function recompute device_map when it is {}
6a3ff7eb
test: add tests for reserve_max_layer
0af58995
Nech-C
changed the title
Reviewers
No reviews
Assignees
No one assigned
Labels
wip
Milestone
No milestone
Login to write a write a comment.
Login via GitHub