text-generation-inference
53214633
- feat(router): add max_batch_size (#1542)
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Commit
View On
GitHub
Commit
1 year ago
feat(router): add max_batch_size (#1542) Some hardware require a maximum batch size.
References
#1542 - feat(router): add max_batch_size
Author
OlivierDehaene
Parents
a4e58016
Loading
