transformers
[docs] continuous batching (offloading behavior, max batch tokens, block size minimum)
#46925

Merged

[docs] continuous batching (offloading behavior, max batch tokens, block size minimum) #46925

stevhliu merged 3 commits into huggingface:main from stevhliu:cb-stuff

docs

a86b6e8b

block size

363c7499

prefill batch size

351ea84a

stevhliu requested a review from

remi-or 17 days ago

remi-or commented on 2026-07-07

remi-or approved these changes on 2026-07-07

stevhliu merged 4af901d5 into main 7 days ago

stevhliu deleted the cb-stuff branch 7 days ago

Reviewers

remi-or

Assignees

No one assigned

Labels

None yet

Milestone

No milestone