transformers
69d0e232 - feat(serve): add memory admission param to `add_request`

Commit

111 days ago

feat(serve): add memory admission param to `add_request`

References

feat/kv_cache_retention_across_convos

Author

McPatate

McPatate

Committer

McPatate

McPatate

Parents

Loading