vllm
f03d82ef - [UX][Bugfix] Fix OOM by setting PyTorch `max_split_size_mb` during model loading (#41268)

Commit

22 days ago

[UX][Bugfix] Fix OOM by setting PyTorch `max_split_size_mb` during model loading (#41268) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

References

Author

MatthewBonanni

Parents