PR #41847 docs: add continuous batching page

docs: add continuous batching page

McPatate committed 212 days ago

docs(cb): add `generate_batch` example

McPatate committed 212 days ago

docs(cb): add `opentelemtry` and `serving` section

McPatate committed 212 days ago

feat: add `TODO` note about opentelemetry dependency

McPatate committed 212 days ago

docs(cb): add supported features

McPatate committed 212 days ago

docs(cb): add unsupported features

McPatate committed 212 days ago

docs(cb): add `ContinuousBatchingManager` example

McPatate committed 205 days ago

docs(cb): x reference CB in optimizing inference

McPatate committed 205 days ago

transformers docs: add continuous batching page #41847 Merged